View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0003888 | unreal | installing | public | 2010-01-25 05:48 | 2010-02-08 17:56 |
| Reporter | ohnobinki | Assigned To | syzop | ||
| Priority | normal | Severity | minor | Reproducibility | have not tried |
| Status | resolved | Resolution | fixed | ||
| Platform | amd64 | OS | Gentoo | OS Version | 2.0 |
| Product Version | 3.2.8 | ||||
| Fixed in Version | 3.2.9-RC1 | ||||
| Summary | 0003888: cURL not handled well by buildsystem | ||||
| Description | tons of problems. ./configure doesn't fail early when it should ./configure nor ./Config have sane defaults concerning --enable-libcurl using old system curl + bundled new cares causes strange problems: `` mode/#unreal-support [+v slushey] by [email protected] <slushey> * Loading IRCd configuration .. <slushey> [error] unrealircd.conf:9: include: error downloading '(url taken out)': Could not resolve host: www.synirc.net (Successful completion) <slushey> [error] IRCd configuration failed to load <slushey> $ nslookup www.synirc.net <slushey> Server: 64.85.160.4 <slushey> Address: 64.85.160.4#53 <slushey> Non-authoritative answer: <slushey> www.synirc.net canonical name = synirc.net. <slushey> Name: synirc.net <slushey> Address: 66.179.9.246 mode/#unreal-support [+b *!*@9CE2AB91.8D9E3B4C.8A7D5246.IP] by [email protected] <slushey> $ slushey has been kicked by [email protected] [Banned: You are sending text too fast. DO NOT PASTE! Use http://unrealircd.pastebin.com - Otherwise, please try to put more text on each line you send so you aren't filling the channel with your nonsense. - 1 minute ban] <binki> lol <Stealth> I need to add that to the bot too '' ... `` 18-01-2010 22:29:08 > binki: slushey: pastebin your server's /etc/resolv.conf [email protected] has joined #unreal-support mode/#unreal-support [+o Nazca] by [email protected] mode/#unreal-support [+o Nazca] by [email protected] <binki> oh, nvmd <slushey> its 2 lines so <slushey> domain slushey.net <slushey> nameserver 64.85.160.4 <binki> slushey: try getting the same URL using curl's CLI <slushey> it worked <binki> try upgrading curl <slushey> worked <slushey> thanks [email protected] has left #unreal-support '' | ||||
| Additional Information | My patch adds some sane defaults and brings errors to light more quickly. I.e., the user will hopefully trigger an error in ./configure when messing with --enable-libcurl than at buildtime. I also added a warning about system curl + bundled c-ares that nobody will read. Maybe I'll write a patch for ./Config to make system-cares and system-tre more available to users. | ||||
| Tags | No tags attached. | ||||
| Attached Files | |||||
| 3rd party modules | |||||
|
|
Did you see 0003857 and the related solution, and the new stuff in CVS? I know your patch is against CVS, but from your talk I can't make up you really looked at the new CVS stuff (quite the contrary). Especially since you talk about not having sane defaults, well.. with CVS if you say 'yes' during ./Config to remote includes and use the defaults, it will download and build curl if necessary, and use that, so not the "system curl", which solved all the problems 0003857 and others are having with regards to ABI incompatibility and such. (Unless something goes wrong in that download & build process, if so, please report in a new issue). |
|
|
Objectives: 1) Should work, no crash (due to ABI incompatability issues) 2) Should not stall the ircd when doing /REHASH (due to curl not being compiled with c-ares) 3) Don't use an insecure old c-ares This means: * System curl without c-ares is no go (conflicts with 1) * System curl with c-ares is only a go if: * The c-ares it uses is equal to or newer than unreal's (makes 1&2&3 ok. needs to use --system-c-ares then) However there are still some issues (see further down) Is not a go if: * Well any other case, but in particular.. version too old (3), and if not using system-c-ares (1) The argument in favor of system curl & c-ares is: * Managability: curl & c-ares get upgraded automatically by the package manager etc in case of issues & updates The argument against it is: * Curl without c-ares is a no go * Due to the automatic upgrading, you'll need to recompile Unreal whenever system c-ares or curl updates (and is ABI incompatible.. which has been frequent with c-ares..). In my view this basically means: * If Unreal is installed via a package manager, system curl & system c-ares (assuming they are too) are a good choice. Unreal package will also be upgraded if needed due to the new curl/c-ares. * If Unreal is self-compiled, then it is debatable... a tradeoff. As Unreal will crash or misbehave whenever something in the system changes. This is where our automatic download & install fix from 0003857 helps and takes care of all that. The only downside with the auto-install aproach is that curl updates don't take place automatically, and that it takes some more compile time and space. As you can see, it's not a simple issue. To my knowledge package mantainers are already using system-c-ares and curl. And as for self-compiled, the solution to that is in CVS. I don't mind applying parts of your patch though, there are some good parts in it. However, it should not warn the same way for system curl as the auto-download-and-install curl. |
|
|
I do not know how much I have paid attention to 0003857 , but I did notice the changes to CVS concerning assisting the user in invoking ./curlinstall. In fact, that is why I filed this bug ;-). Sorry. I now realize that I went overboard in my patch's comments, etc. But I will never be able to understand why people will opt for manually installing cURL into their ${HOME}---unless if they're forced to because they're using a shell account. I have observed that people do install curl locally regardless of how silly it is (in my understanding) to do so. So I suppose it'd be OK to support them even if I can't understand their motivation. But may the ./configure script at least issue a warning when system and local versions of curl & c-ares (respectively) are requested by the user? I am pretty sure that my patch will not warn the user if he chose ./curlinstall... I tested to make sure that the cURL prefix was not ${HOME}/curl (which is where ./curlinstall places its files afaik). FYI, and I forgot about this until now, the ``proper'' solution for preventing conflict between a bundled c-ares and system curl is symbol visibility: http://blog.flameeyes.eu/2008/02/10/why-would-an-executable-export-symbols . There should be some way to hide the c-ares symbols in the final ircd executable, but I'm not sure this is worth the work since using a system-installed c-ares is the ideal and avoids this problem altogether :-/ Thanks for considering even pieces of my patch, I look forward to seeing ``cvs update'' output some `U' lines :-). |
|
|
Concerning visibility, here is the problem: Run this command in a compiled UnrealIRCd-3.2.8.1 directory: $ nm -D src/ircd | grep ares_version You should see something like ``21341234 T ares_version'' (iirc). See the following output from my machine. --with-system-cares: $ nm -D $(which unrealircd) | grep ares_version U ares_version libcares.so: $ nm -D /usr/lib64/libcares.so | grep ares_version 000000000000ce00 T ares_version with the patch I have attached and --without-system-cares (uses bundled): $ nm -D src/ircd | grep ares_version $ (no output). The output from the last one should be possible with unrealircd-3.2.8.1CVS-hidden-cares.patch . This will mean that if unrealircd links against a system curl, that curl will look where it normally does for c-ares, even when c-ares is bundled into src/ircd. Also, unrealircd will use the bundled version of c-ares. Something similar could be done for TRE, but there have been no issues so far. |
|
|
Patch applied, mostly unharmed :). Sorry, I overlooked the $HOME check indeed! Changed a warning in an error. Added a warning when using a curl without c-ares (which, as you too pointed out, probably nobody reads :P). Etc. - Added patch from ohnobinki (0003888), only slightly edited, which improves curl detection, added checks to see if curl actually works (print out a clear curl error during configure, instead of getting an error during 'make'), and we now error when using --enable-libcurl without --with-system-cares if the system curl depends on c-ares. This is because this can cause ABI incompatability between curl's c-ares and our c-ares, which leads to odd issues such as: Could not resolve host: www.example.net (Successful completion) And possibly other weird issues, perhaps even crashes. Thanks for the patch, definitely appreciated. Improving ./Config is probably not a bad idea either. I'll check your other patch on Wednesday. |
|
|
Seems you posted while I was committing & writing my comment. Ehm... yeah is interesting, but.. I think we already solved the problem this way? It's probably a good idea to use system c-ares if it's available, anyway. And when curl is compiled with c-ares (which it should) then c-ares should be available too. (and if curl isn't compiled with c-ares then there's no problem). Only thing I can think of is if you have some old c-ares on your system, which is insecure or does not support the features we require. Then you would need to upgrade your system c-ares first, before you can compile Unreal (as we require version X.YY). But that shouldn't be too much of a problem. If it is, like you don't have root and the admin refuses, then you can always build your own. Like.. the example pointed out that unreal didn't work because the system c-ares was a version 1.4.0 and unreal used 1.6.0 is not much of an issue to me, as <1.6.0 had a security issue in it and shouldn't be used anyway (incorrect memory allocation of multiple PTR records, in case you wonder, and possibly other issues). Of course, if we at some point depend on some really new feature in c-ares, then it might be an issue. Then again, if you choose system c-ares, then it's your job to keep it recent. |
|
|
I'll think again about it on Wednesday :p |
|
|
This patch seems to break building Unreal *without* cURL. ./configure --with-showlistmodes --enable-hub --enable-prefixaq --with-listen=5 --with-dpath=/home/syzop/Unreal3.2 --with-spath=/home/syzop/Unreal3.2/src/ircd --with-nick-history=2000 --with-sendq=3000000 --with-bufferpool=18 --with-hostname=vulnscan --with-permissions=0600 --with-fd-setsize=1024 --enable-dynamic-linking gcc -I../include -I/home/syzop/Unreal3.2/extras/regexp/include -I/home/syzop/Unreal3.2/extras/c-ares/include -pipe -g -O2 -funsigned-char -fno-strict-aliasing -Wno-pointer-sign -export-dynamic -c timesynch.c gcc -I../include -I/home/syzop/Unreal3.2/extras/regexp/include -I/home/syzop/Unreal3.2/extras/c-ares/include -pipe -g -O2 -funsigned-char -fno-strict-aliasing -Wno-pointer-sign -export-dynamic -c res.c --snip-- gcc -I../include -I/home/syzop/Unreal3.2/extras/regexp/include -I/home/syzop/Unreal3.2/extras/c-ares/include -pipe -g -O2 -funsigned-char -fno-strict-aliasing -Wno-pointer-sign -export-dynamic -o ircd timesynch.o res.o s_bsd.o auth.o aln.o channel.o cloak.o crule.o dbuf.o events.o fdlist.o hash.o help.o ircd.o ircsprintf.o list.o lusers.o match.o modules.o packet.o parse.o s_auth.o s_conf.o s_debug.o s_err.o s_extra.o s_kline.o s_misc.o s_numeric.o s_serv.o s_svs.o socket.o ssl.o s_user.o charsys.o scache.o send.o support.o umodes.o version.o whowas.o zip.o cidr.o random.o extcmodes.o extbans.o md5.o api-isupport.o api-command.o url.o -lcrypt -lnsl -ldl -L/home/syzop/Unreal3.2/extras/regexp/lib -ltre -L../extras/c-ares/lib -L/home/syzop/Unreal3.2/extras/c-ares/lib -lcares -lrt url.o: In function `url_do_transfers_async': /home/syzop/Unreal3.2/src/url.c:302: undefined reference to `curl_multi_perform' /home/syzop/Unreal3.2/src/url.c:317: undefined reference to `curl_multi_fdset' /home/syzop/Unreal3.2/src/url.c:333: undefined reference to `curl_multi_info_read' /home/syzop/Unreal3.2/src/url.c:341: undefined reference to `curl_easy_getinfo' /home/syzop/Unreal3.2/src/url.c:342: undefined reference to `curl_easy_getinfo' /home/syzop/Unreal3.2/src/url.c:343: undefined reference to `curl_easy_getinfo' /home/syzop/Unreal3.2/src/url.c:344: undefined reference to `curl_easy_getinfo' /home/syzop/Unreal3.2/src/url.c:372: undefined reference to `curl_multi_remove_handle' /home/syzop/Unreal3.2/src/url.c:377: undefined reference to `curl_easy_cleanup' /home/syzop/Unreal3.2/src/url.c:327: undefined reference to `curl_multi_perform' url.o: In function `url_init': /home/syzop/Unreal3.2/src/url.c:221: undefined reference to `curl_global_init' /home/syzop/Unreal3.2/src/url.c:222: undefined reference to `curl_multi_init' url.o: In function `download_file_async': /home/syzop/Unreal3.2/src/url.c:242: undefined reference to `curl_easy_init' /home/syzop/Unreal3.2/src/url.c:264: undefined reference to `curl_easy_setopt' /home/syzop/Unreal3.2/src/url.c:265: undefined reference to `curl_easy_setopt' /home/syzop/Unreal3.2/src/url.c:266: undefined reference to `curl_easy_setopt' /home/syzop/Unreal3.2/src/url.c:267: undefined reference to `curl_easy_setopt' /home/syzop/Unreal3.2/src/url.c:272: undefined reference to `curl_easy_setopt' url.o:/home/syzop/Unreal3.2/src/url.c:273: more undefined references to `curl_easy_setopt' follow url.o: In function `download_file': /home/syzop/Unreal3.2/src/url.c:144: undefined reference to `curl_easy_init' /home/syzop/Unreal3.2/src/url.c:170: undefined reference to `curl_easy_setopt' /home/syzop/Unreal3.2/src/url.c:171: undefined reference to `curl_easy_setopt' /home/syzop/Unreal3.2/src/url.c:172: undefined reference to `curl_easy_setopt' /home/syzop/Unreal3.2/src/url.c:173: undefined reference to `curl_easy_setopt' /home/syzop/Unreal3.2/src/url.c:174: undefined reference to `curl_easy_setopt' url.o:/home/syzop/Unreal3.2/src/url.c:175: more undefined references to `curl_easy_setopt' follow url.o: In function `download_file': /home/syzop/Unreal3.2/src/url.c:188: undefined reference to `curl_easy_perform' /home/syzop/Unreal3.2/src/url.c:200: undefined reference to `curl_easy_getinfo' /home/syzop/Unreal3.2/src/url.c:201: undefined reference to `curl_easy_cleanup' /home/syzop/Unreal3.2/src/url.c:209: undefined reference to `curl_easy_cleanup' url.o: In function `download_file_async': /home/syzop/Unreal3.2/src/url.c:289: undefined reference to `curl_multi_add_handle' collect2: ld returned 1 exit status make[1]: *** [ircd] Error 1 make[1]: Leaving directory `/home/syzop/Unreal3.2/src' make: *** [build] Error 2 |
|
|
I'm quite busy today, and, I can't quickly spot the error to be honest (which isn't unusual with autoconf) :P. I've reverted the patch in .793 until we fix this: - Patch from above is (temp.) reverted, Unreal wouldn't compile without curl. |
|
|
Sorry, it looks like I ended the AS_IF([test "x$enable_curl" != "no"]) with a comment that looks like ``dnl for emac's sh-mode ])''. This is now removed. Its presence caused url.o to be added to the list of files that's compiled and linked into UnrealIRCd and we somehow didn't see a warning about too many closing ``])'' sequences... which confuses me still :-/ Oh, disregard version 2, the ([]) are stil unbalanced I think... |
|
|
Oh, I think that version 2 is properly balanced, just emacs was confused and thus confusing me ;-). Compiling with and without remote includes now works for me. |
|
|
ok, redid the fix. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2010-01-25 05:48 | ohnobinki | New Issue | |
| 2010-01-25 05:48 | ohnobinki | File Added: unrealircd-3.2.9_rc-curl-sanity.patch | |
| 2010-01-25 12:00 | syzop | Note Added: 0016002 | |
| 2010-01-25 12:29 | syzop | Note Added: 0016003 | |
| 2010-01-25 17:34 | ohnobinki | Note Added: 0016004 | |
| 2010-01-25 21:12 | ohnobinki | Note Added: 0016005 | |
| 2010-01-25 21:13 | ohnobinki | File Added: unrealircd-3.2.8.1CVS-hidden-cares.patch | |
| 2010-01-25 21:26 | syzop | QA | => Not touched yet by developer |
| 2010-01-25 21:26 | syzop | U4: Need for upstream patch | => No need for upstream InspIRCd patch |
| 2010-01-25 21:26 | syzop | Note Added: 0016006 | |
| 2010-01-25 21:26 | syzop | Status | new => resolved |
| 2010-01-25 21:26 | syzop | Fixed in Version | => 3.2.9-RC1 |
| 2010-01-25 21:26 | syzop | Resolution | open => fixed |
| 2010-01-25 21:26 | syzop | Assigned To | => syzop |
| 2010-01-25 21:37 | syzop | Note Added: 0016007 | |
| 2010-01-25 21:38 | syzop | Note Added: 0016008 | |
| 2010-01-27 09:58 | syzop | Note Added: 0016009 | |
| 2010-01-27 09:58 | syzop | Status | resolved => feedback |
| 2010-01-27 09:58 | syzop | Resolution | fixed => reopened |
| 2010-01-27 10:01 | syzop | Note Edited: 0016009 | |
| 2010-01-27 10:14 | syzop | Note Added: 0016010 | |
| 2010-01-27 23:50 | ohnobinki | File Added: unrealircd-3.2.9_pre-curl-sanity-2.patch | |
| 2010-01-27 23:57 | ohnobinki | Note Added: 0016011 | |
| 2010-01-28 00:01 | ohnobinki | Note Added: 0016012 | |
| 2010-02-08 17:56 | syzop | Note Added: 0016014 | |
| 2010-02-08 17:56 | syzop | Status | feedback => resolved |
| 2010-02-08 17:56 | syzop | Resolution | reopened => fixed |
