View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0001798 | unreal | ircd | public | 2004-05-08 12:14 | 2004-05-19 16:53 |
| Reporter | trystanscott | Assigned To | |||
| Priority | normal | Severity | crash | Reproducibility | always |
| Status | closed | Resolution | open | ||
| Product Version | 3.2 | ||||
| Summary | 0001798: Remote Include with Rehash causes crash | ||||
| Description | Attempting to use Remote Include, when rehash is called, the ircd crashes libcurl/7.10.2 OpenSSL/0.9.6b ipv6 zlib/1.1.4 | ||||
| Additional Information | #0 0x08095705 in url_do_transfers_async () at url.c:332 332 fclose(handle->fd); (gdb) bt #0 0x08095705 in url_do_transfers_async () at url.c:332 #1 0x0806e132 in read_message (delay=0, listp=0x814f5c0) at s_bsd.c:1657 #2 0x0806336b in main (argc=135589385, argv=0x2) at ircd.c:1431 #3 0x400ab1c4 in __libc_start_main () from /lib/libc.so.6 (gdb) backtrace full #0 0x08095705 in url_do_transfers_async () at url.c:332 handle = (FileHandle *) 0x248 url = 0x81a15d0 "http://www.nomadirc.net/ircd/spam.conf" code = 304 last_mod = -1 easyhand = (CURL *) 0x8214b38 cont = 0 msgs_left = 0 msg = (CURLMsg *) 0x81e4c60 #1 0x0806e132 in read_message (delay=0, listp=0x814f5c0) at s_bsd.c:1657 cptr = (aClient *) 0xbffff950 nfds = 32 wait = {tv_sec = 1, tv_usec = 980000} read_set = {__fds_bits = {0 <repeats 32 times>}} write_set = {__fds_bits = {0 <repeats 32 times>}} j = 1084036022 delay2 = 0 res = 0 length = 32 fd = 32 i = 584 sockerr = 584 #2 0x0806336b in main (argc=135589385, argv=0x2) at ircd.c:1431 oldtimeofday = 2 uid = 513 euid = 513 delay = 2 corelim = {rlim_cur = 4294967295, rlim_max = 4294967295} nextfdlistcheck = 1084036023 #3 0x400ab1c4 in __libc_start_main () from /lib/libc.so.6 No symbol table info available. | ||||
| 3rd party modules | |||||
|
|
Are you able to reproduce this problem? |
|
|
Every time with a test server that I am building to test remote includes and various other options on before I put it live. |
|
|
You looking into this? Or was this already fixed? (I noticed a change but I don't know if it was related ;p). Just wondering. |
|
|
I haven't looked too much, I can't see anything that would cause the fd to become NULL. And no, that bug fix was unrelated (at least as far as I know). |
|
|
I just checked it over again, I don't see anything that could be screwing it up. We initialize the FileHandle, send it to libcurl (libcurl never touches it) then when the transfer finishes, we have curl send it back to us. |
|
|
I see. trystanscott: you said you were able to reproduce this every time... Could you send your configfiles [with passwords/cloak-keys changed] to [email protected] so I could try to reproduce this issue? Also let me know if you have any 3rd party modules loaded and with which options you compiled the ircd (ssl, zip, remote includes with or without ssl, ..). |
|
|
codemastr: from the information above it says 'handle = (FileHandle *) 0x248' which looks a very wrong pointer to me ;).. So I suspect some kind of memory corruption (perhaps not in unreal, we'll see). So once I receive the configfile(s) from trystanscott I hope that mpatrol will catch it. |
|
|
Yeah, not sure why I said NULL, I meant mem corruption :P The thing is though, if you look in libcurl, it never touches that handle. It just stores the pointer, and returns it when we ask for it. So if there is memory corruption, it would seem as though it is most likely something we are causing, and I don't see anything that would cause it. |
|
|
[long story but I've traced it and provide a solution/workaround :p] Was unable to reproduce it on my machine, now got an account on that box. I removed curl and installed a new one (latest) without SSL... Still crashing. Then I disable SSL (in unreal)... still crashing. Then I disable zlib... no crash! A closer look shows that even if zlib is compiled it for some reason magically adds SSL support to curl. I traced this back to the -I/usr/local/include and -L/usr/local/lib parameters... If I compile with them [=normal] (zip+curl) I get: _ _ _ ___________ _____ _ | | | | | |_ _| ___ \/ __ \ | | | | | |_ __ _ __ ___ __ _| | | | | |_/ /| / \/ __| | | | | | '_ \| '__/ _ \/ _` | | | | | / | | / _` | | |_| | | | | | | __/ (_| | |_| |_| |\ \ | \__/\ (_| | \___/|_| |_|_| \___|\__,_|_|\___/\_| \_| \____/\__,_| v3.2 using zlib 1.1.4 using libcurl/7.10.2 OpenSSL/0.9.6b ipv6 zlib/1.1.4 then if I drop those 2 -I and -L thingies I get: _ _ _ ___________ _____ _ | | | | | |_ _| ___ \/ __ \ | | | | | |_ __ _ __ ___ __ _| | | | | |_/ /| / \/ __| | | | | | '_ \| '__/ _ \/ _` | | | | | / | | / _` | | |_| | | | | | | __/ (_| | |_| |_| |\ \ | \__/\ (_| | \___/|_| |_|_| \___|\__,_|_|\___/\_| \_| \____/\__,_| v3.2 using zlib 1.1.4 using libcurl/7.11.2 zlib/1.1.4 c-ares/1.2.0 fun isn't it? Now, trying a closer look to the case w/-I and -L: [ircdtest@denora Unreal3.2]$ ldd src/ircd libcrypt.so.1 => /lib/libcrypt.so.1 (0x4001d000) libnsl.so.1 => /lib/libnsl.so.1 (0x4004b000) libz.so.1 => /usr/lib/libz.so.1 (0x4005f000) libcurl.so.2 => /usr/lib/libcurl.so.2 (0x4006d000) libdl.so.2 => /lib/libdl.so.2 (0x40091000) libc.so.6 => /lib/libc.so.6 (0x40094000) libssl.so.2 => /lib/libssl.so.2 (0x401bb000) libcrypto.so.2 => /lib/libcrypto.so.2 (0x401e9000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) [ircdtest@denora Unreal3.2]$ ldd -v src/ircd libcrypt.so.1 => /lib/libcrypt.so.1 (0x4001d000) libnsl.so.1 => /lib/libnsl.so.1 (0x4004b000) libz.so.1 => /usr/lib/libz.so.1 (0x4005f000) libcurl.so.2 => /usr/lib/libcurl.so.2 (0x4006d000) libdl.so.2 => /lib/libdl.so.2 (0x40091000) libc.so.6 => /lib/libc.so.6 (0x40094000) libssl.so.2 => /lib/libssl.so.2 (0x401bb000) libcrypto.so.2 => /lib/libcrypto.so.2 (0x401e9000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) Version information: src/ircd: libcrypt.so.1 (GLIBC_2.0) => /lib/libcrypt.so.1 libdl.so.2 (GLIBC_2.1) => /lib/libdl.so.2 libdl.so.2 (GLIBC_2.0) => /lib/libdl.so.2 libc.so.6 (GLIBC_2.1) => /lib/libc.so.6 libc.so.6 (GLIBC_2.2) => /lib/libc.so.6 libc.so.6 (GLIBC_2.0) => /lib/libc.so.6 /lib/libcrypt.so.1: libc.so.6 (GLIBC_2.1.3) => /lib/libc.so.6 libc.so.6 (GLIBC_2.0) => /lib/libc.so.6 /lib/libnsl.so.1: libc.so.6 (GLIBC_2.1.3) => /lib/libc.so.6 libc.so.6 (GLIBC_2.2) => /lib/libc.so.6 libc.so.6 (GLIBC_2.2.3) => /lib/libc.so.6 libc.so.6 (GLIBC_2.0) => /lib/libc.so.6 libc.so.6 (GLIBC_2.1) => /lib/libc.so.6 /usr/lib/libz.so.1: libc.so.6 (GLIBC_2.1.3) => /lib/libc.so.6 libc.so.6 (GLIBC_2.1) => /lib/libc.so.6 libc.so.6 (GLIBC_2.0) => /lib/libc.so.6 /usr/lib/libcurl.so.2: libdl.so.2 (GLIBC_2.1) => /lib/libdl.so.2 libdl.so.2 (GLIBC_2.0) => /lib/libdl.so.2 libc.so.6 (GLIBC_2.2) => /lib/libc.so.6 libc.so.6 (GLIBC_2.1.3) => /lib/libc.so.6 libc.so.6 (GLIBC_2.1) => /lib/libc.so.6 libc.so.6 (GLIBC_2.0) => /lib/libc.so.6 /lib/libdl.so.2: libc.so.6 (GLIBC_2.1.3) => /lib/libc.so.6 libc.so.6 (GLIBC_2.1) => /lib/libc.so.6 libc.so.6 (GLIBC_PRIVATE) => /lib/libc.so.6 libc.so.6 (GLIBC_2.0) => /lib/libc.so.6 /lib/libc.so.6: ld-linux.so.2 (GLIBC_2.1) => /lib/ld-linux.so.2 ld-linux.so.2 (GLIBC_2.0) => /lib/ld-linux.so.2 ld-linux.so.2 (GLIBC_PRIVATE) => /lib/ld-linux.so.2 /lib/libssl.so.2: libc.so.6 (GLIBC_2.1.3) => /lib/libc.so.6 libc.so.6 (GLIBC_2.0) => /lib/libc.so.6 /lib/libcrypto.so.2: libdl.so.2 (GLIBC_2.1) => /lib/libdl.so.2 libdl.so.2 (GLIBC_2.0) => /lib/libdl.so.2 libc.so.6 (GLIBC_2.1.3) => /lib/libc.so.6 libc.so.6 (GLIBC_2.1) => /lib/libc.so.6 libc.so.6 (GLIBC_2.2) => /lib/libc.so.6 libc.so.6 (GLIBC_2.0) => /lib/libc.so.6 not really helpful. Then, just to test SSL + zlib + curl support WITHOUT -L/usr/local/lib and -I/usr/local/include: _ _ _ ___________ _____ _ | | | | | |_ _| ___ \/ __ \ | | | | | |_ __ _ __ ___ __ _| | | | | |_/ /| / \/ __| | | | | | '_ \| '__/ _ \/ _` | | | | | / | | / _` | | |_| | | | | | | __/ (_| | |_| |_| |\ \ | \__/\ (_| | \___/|_| |_|_| \___|\__,_|_|\___/\_| \_| \____/\__,_| v3.2 using OpenSSL 0.9.7d 17 Mar 2004 using zlib 1.1.4 using libcurl/7.11.2 zlib/1.1.4 c-ares/1.2.0 and it works fine. trystanscott: Anyway, the reason that it goes wrong on your box is because you have multiple versions of openssl installed.. $ locate opensslv.h /usr/include/openssl/opensslv.h /usr/local/include/openssl/opensslv.h /usr/local/ssl/include/openssl/opensslv.h /home/ircdtest/openssl-0.9.7d/crypto/opensslv.h /home/ircdtest/openssl-0.9.7d/include/openssl/opensslv.h [ircdtest@denora Unreal3.2]$ grep OPENSSL_VERSION_TEXT /usr/include/openssl/opensslv.h #define OPENSSL_VERSION_TEXT "OpenSSL 0.9.6b [engine] 9 Jul 2001" #define OPENSSL_VERSION_PTEXT " part of " OPENSSL_VERSION_TEXT [ircdtest@denora Unreal3.2]$ grep OPENSSL_VERSION_TEXT /usr/local/include/openssl/opensslv.h #define OPENSSL_VERSION_TEXT "OpenSSL 0.9.7d 17 Mar 2004" #define OPENSSL_VERSION_PTEXT " part of " OPENSSL_VERSION_TEXT [ircdtest@denora Unreal3.2]$ grep OPENSSL_VERSION_TEXT /home/ircdtest/openssl-0.9.7d/include/openssl/opensslv.h #define OPENSSL_VERSION_TEXT "OpenSSL 0.9.7d 17 Mar 2004" #define OPENSSL_VERSION_PTEXT " part of " OPENSSL_VERSION_TEXT [ircdtest@denora Unreal3.2]$ ls -al /lib/libssl* /usr/lib/libssl* -rwxr-xr-x 1 root root 211067 Mar 19 2003 /lib/libssl.so.0.9.6b lrwxrwxrwx 1 root root 16 Jun 21 2003 /lib/libssl.so.2 -> libssl.so.0.9.6b -rw-r--r-- 1 root root 259046 Mar 19 2003 /usr/lib/libssl.a lrwxrwxrwx 1 root root 26 Jun 21 2003 /usr/lib/libssl.so -> ../../lib/libssl.so.0.9.6b ..And since unreal is (for whatever reason) adding include/libdirs for /usr/local/[etc] you get both mixed causing these weird crashes/behavior. Note that with the configuration above (zip+ssl+w/-I/-L for /usr/local/[..] removed) ldd looks like this: [ircdtest@denora Unreal3.2]$ ldd src/ircd libcrypt.so.1 => /lib/libcrypt.so.1 (0x4001d000) libnsl.so.1 => /lib/libnsl.so.1 (0x4004b000) libz.so.1 => /usr/lib/libz.so.1 (0x4005f000) libdl.so.2 => /lib/libdl.so.2 (0x4006d000) libc.so.6 => /lib/libc.so.6 (0x40070000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) and SSL seems to be compiled statically in the ircd. Oh and when we are at it, the same applies to zlib: [ircdtest@denora Unreal3.2]$ grep ZLIB_VERSION /usr/include/zlib.h #define ZLIB_VERSION "1.1.4" [ircdtest@denora Unreal3.2]$ grep ZLIB_VERSION /usr/local/include/zlib.h #define ZLIB_VERSION "1.2.1" [note that I still have no idea why if the -I/-L stuff is added for /usr/local/* suddenly openssl gets put into curl {without touching curl itself}.. pretty weird] *dinner*. |
|
|
What about adding a ./configure check that would compare the runtime and header version of openssl, something like this: -- #include <openssl/opensslv.h> #include <openssl/crypto.h> int main(int argc, char *argv[]) { printf("Runtime version: %s\n", SSLeay_version(SSLEAY_VERSION)); printf("Headers version: %s\n", OPENSSL_VERSION_TEXT); exit(EXIT_SUCCESS); } -- (but then obviously a compare and not printing ;p). [or use SSLEAY_VERSION_NUMBER instead of OPENSSL_VERSION_TEXT] This is mentioned at http://www.openssl.org/docs/crypto/OPENSSL_VERSION_NUMBER.html quoting: "For backward compatibility, SSLEAY_VERSION_NUMBER is also defined. SSLeay() returns this number. The return value can be compared to the macro to make sure that the correct version of the library has been loaded, especially when using DLLs on Windows systems. SSLeay_version() returns different strings depending on t: SSLEAY_VERSION The text variant of the version number and the release date. For example, ``OpenSSL 0.9.5a 1 Apr 2000''." And perhaps at runtime a WARNING too if it mismatches.. as some people reported the ircd could crash after upgrading openssl.. simply recompiling solves it then. Or just only at runtime... [=easier ;pp] |
|
|
hmk, I think that's the way to go... the runtime check -> WARNING I mean.. perhaps also taint the unreal version # in a certain way so we can spot such issues easily. *working on it* ;) [ircdtest@denora Unreal3.2]$ killall -9 ircd; src/ircd _ _ _ ___________ _____ _ | | | | | |_ _| ___ \/ __ \ | | | | | |_ __ _ __ ___ __ _| | | | | |_/ /| / \/ __| | | | | | '_ \| '__/ _ \/ _` | | | | | / | | / _` | | |_| | | | | | | __/ (_| | |_| |_| |\ \ | \__/\ (_| | \___/|_| |_|_| \___|\__,_|_|\___/\_| \_| \____/\__,_| v3.2 using OpenSSL 0.9.7d 17 Mar 2004 using zlib 1.1.4 using libcurl/7.10.2 OpenSSL/0.9.6b ipv6 zlib/1.1.4 [!!!] OpenSSL version mismatch: compiled for 'OpenSSL 0.9.7d 17 Mar 2004', library is 'OpenSSL 0.9.6b [engine] 9 Jul 2001' [!!!] Zlib version mismatch: compiled for '1.2.1', library is '1.1.4' [!!!] Header<->library mismatches can make UnrealIRCd *CRASH*! Make sure you don't have multiple versions of openssl or zlib installed (eg: one in /usr and one in /usr/local). And, if you recently upgraded them, be sure to recompile Unreal. * Loading IRCd configuration .. * Configuration loaded without any problems .. * Loading tunefile.. * Initializing SSL. * Dynamic configuration initialized .. booting IRCd. --------------------------------------------------------------------- Oooooooo :P edited on: 2004-05-19 16:09 |
|
|
Committed version check thingy (@runtime) in CVS. I think that's enough :p. *tv for 1h* |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2004-05-08 12:14 | trystanscott | New Issue | |
| 2004-05-08 12:29 |
|
Note Added: 0006131 | |
| 2004-05-08 12:36 | trystanscott | Note Added: 0006132 | |
| 2004-05-17 21:18 | syzop | Note Added: 0006309 | |
| 2004-05-18 17:04 |
|
Note Added: 0006324 | |
| 2004-05-18 17:13 |
|
Note Added: 0006326 | |
| 2004-05-18 17:14 | syzop | Note Added: 0006327 | |
| 2004-05-18 17:19 | syzop | Note Added: 0006329 | |
| 2004-05-19 00:01 |
|
Note Added: 0006332 | |
| 2004-05-19 12:56 | syzop | Note Added: 0006335 | |
| 2004-05-19 15:39 | syzop | Note Added: 0006338 | |
| 2004-05-19 15:52 | syzop | Note Added: 0006339 | |
| 2004-05-19 15:58 | syzop | Note Edited: 0006339 | |
| 2004-05-19 15:59 | syzop | Note Edited: 0006339 | |
| 2004-05-19 16:00 | syzop | Note Edited: 0006339 | |
| 2004-05-19 16:09 | syzop | Note Edited: 0006339 | |
| 2004-05-19 16:53 | syzop | Status | new => closed |
| 2004-05-19 16:53 | syzop | Note Added: 0006340 |
