View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0002503unrealircdpublic2005-04-26 19:242005-05-02 12:41
Reporterjesterpm Assigned To 
PrioritynormalSeveritycrashReproducibilityalways
Status closedResolutionopen 
Product Version3.2.3 
Summary0002503: Segmentation Fault while reading config file
DescriptionUnreal crashes with a segmentation fault halfway through loading the configuration file.
I have also built 3.2.3 under Mandrake 9.1 and it runs fine on this same system.
I've included the output of the unreal startup script and a backtrace. If you need anything else please email me. [email protected]

Output from ./unreal start:

Starting UnrealIRCd
 _ _ _ ___________ _____ _
| | | | | |_ _| ___ \/ __ \ | |
| | | |_ __ _ __ ___ __ _| | | | | |_/ /| / \/ __| |
| | | | '_ \| '__/ _ \/ _` | | | | | / | | / _` |
| |_| | | | | | | __/ (_| | |_| |_| |\ \ | \__/\ (_| |
 \___/|_| |_|_| \___|\__,_|_|\___/\_| \_| \____/\__,_|
                           v3.2.3
                     using TRE 0.7.2 (GPL)

* Loading IRCd configuration ..
* unrealircd.conf:153: Ignoring extra data
* unrealircd.conf:153: Ignoring extra data
./unreal: line 39: 28410 Segmentation fault (core dumped) /home/ircd/Unreal
3.2/src/ircd
Possible error encountered (IRCd seemily not started)
=====================================================
Check above for possible errors, and this output of
ircd.log. If you cannot solve the problem, read
Unreal.nfo on where to get support
=====================================================
tail: cannot open `/home/ircd/Unreal3.2/ircd.log' for reading: No such file or d
irectory
ircd@Krypton:~/Unreal3.2$

--------------------------------------------

gdb Backtrace:

#0 0x400c7c41 in _int_realloc () from /lib/libc.so.6
#1 0x400c894d in realloc () from /lib/libc.so.6
#2 0x400c8a61 in realloc_hook_ini () from /lib/libc.so.6
#3 0x400c889c in realloc () from /lib/libc.so.6
#4 0x400ef6e8 in glob_in_dir () from /lib/libc.so.6
#5 0x400eff6b in glob () from /lib/libc.so.6
#6 0x0807acae in load_conf (filename=0x80a1c00 "unrealircd.conf") at s_conf.c:7636
#7 0x0807e2ce in init_conf (rootconf=0xfffffffc <Address 0xfffffffc out of bounds>, rehash=0) at s_conf.c:1602
0000008 0x0805e30a in main (argc=135472360, argv=0xbffffb44) at ircd.c:1259
3rd party modules

Activities

syzop

2005-04-26 21:35

administrator   ~0009818

Last edited: 2005-04-26 21:36

 what are the loadmodule lines you use? the one with wildcards to be exact (it crashes on that).
[clarification: yes, wildcard are accepted, no.. it shouldn't crash, it didn't before either]

jesterpm

2005-04-26 21:40

reporter   ~0009820

I wish it was that simple... but I don't have any wild cards in my loadmodule lines. Here they are though:

# Load Modules
loadmodule "src/modules/commands.so";
loadmodule "src/modules/cloak.so";

syzop

2005-04-26 22:00

administrator   ~0009824

hm ok.. could you type the following on the gdb prompt (with the core file loaded etc, just like you did before):
--
p *ce
p files
--
it shouldn't crash there anyway, which makes me wonder if this really is the "real cause", when I look at your output I also see "init_conf (rootconf=0xfffffffc <Address 0xfffffffc out of bounds>," which is quite odd.

Is this an official .tar.gz downloaded ./Config'ed and compiled?
Could you double check if you have any CFLAGS present that might cause odd things to happen? (type 'set|grep CFLAGS' at the shell)

Also, it shouldn't be related, but.. there are 2 'ignoring extra data' messages during loading of the conf, perhaps you can take a look at fixing that.

PS: will be gone for today.

jesterpm

2005-04-26 22:18

reporter   ~0009825

Here's the gdb stuff:
(gdb) p *ce
No symbol "ce" in current context.
(gdb) p files
No symbol "files" in current context.

No CFLAGS are set.

As for the extra data, I tried a known good conf and it had the same problem.
If I remember right with 2 out of 3 confs I tried it crashed around the listen blocks.

syzop

2005-04-27 00:03

administrator   ~0009827

Hi, sorry I forgot a cmd, try this:
--
frame 6
p *ce
p files
--

jesterpm

2005-04-27 00:11

reporter   ~0009828

(gdb) frame 6
#6 0x0807acae in load_conf (filename=0x80a1c00 "unrealircd.conf")
    at s_conf.c:7636
7636 glob(ce->ce_vardata, GLOB_NOSORT|GLOB_NOCHECK, NULL, &files);
(gdb) p *ce
$1 = {ce_fileptr = 0x8147200, ce_varlinenum = 6, ce_fileposstart = 112,
  ce_fileposend = 148, ce_sectlinenum = 0,
  ce_varname = 0x8147258 "loadmodule",
  ce_vardata = 0x8147268 "src/modules/commands.so", ce_entries = 0x0,
  ce_prevlevel = 0x0, ce_next = 0x8147288}
(gdb) p files
No symbol "files" in current context.

---------
I haven't done much C programming or debuging but is that saying it crashed on the loadmodule for commands.so?

syzop

2005-04-27 18:01

administrator   ~0009836

Last edited: 2005-04-27 18:03

 hm.. the glob manpage isn't totally clear to me, but try this:
open src/s_conf.c and go to line 7632 which reads:
#ifdef GLOBH
and add below that the following line:
memset(&files, 0, sizeof(files));
save it, and do a 'make && make install' and try to boot up again.

Let me know if that helps.

*edit* typo/grammar fix, no code edit */edit*

jesterpm

2005-04-27 20:27

reporter   ~0009837

That fixed part of it :)

Here's a new backtrace:
(gdb) bt
#0 0x0807ccb3 in load_conf (filename=0x80a4408 "unrealircd.conf")
    at s_conf.c:7647
#1 0x080802ee in init_conf (rootconf=0x0, rehash=0) at s_conf.c:1602
#2 0x0805fae3 in main (argc=135485452, argv=0xbffffaf4) at ircd.c:1259


Looking at what you had me do last time, I ran a few other commands:
(gdb) frame 0
#0 0x0807ccb3 in load_conf (filename=0x80a4408 "unrealircd.conf")
    at s_conf.c:7647
7647 if ((ret = Module_Create(files.gl_pathv[i]))) {
(gdb) p *ce
$1 = {ce_fileptr = 0x814a1e8, ce_varlinenum = 6, ce_fileposstart = 112,
  ce_fileposend = 148, ce_sectlinenum = 0,
  ce_varname = 0x814a240 "loadmodule",
  ce_vardata = 0x814a250 "src/modules/commands.so", ce_entries = 0x0,
  ce_prevlevel = 0x0, ce_next = 0x814a270}


Hope that helps.

syzop

2005-04-27 20:40

administrator   ~0009838

a 'p files' and 'p files.gl_pathv[i]' would be helpful as well (if it returns anything, that is ;p).

jesterpm

2005-04-27 20:43

reporter   ~0009839

Haha, already tried p files :P
Here ya go anyways

(gdb) p files
No symbol "files" in current context.
(gdb) p files.gl_pathv[i]
No symbol "files" in current context.

syzop

2005-04-27 22:03

administrator   ~0009840

Last edited: 2005-04-27 22:04

 Yeah that doesn't help much indeed :(.
Could you open up Makefile (the Makefile from your main unreal dir, not from src/) and change the XCFLAGS line slightly:
- remove the -O2
- add -fno-inline
(this are all edits of just 1 line)

And then do: 'make clean; make && make install'
And try to boot and do all the 'bt' and 'p files' etc commands again, then this 'files variable' info should be available. I think (hope) we are finally getting to the problem then..

Thanks.

jesterpm

2005-04-28 00:35

reporter   ~0009841

Here you go.
Would you mind explaining what the XCFLAG changes did? and why zerofilling files fixed the first problem? (sorry if I sound stupid :P I never finished learning C)

(gdb) bt
#0 0x08083783 in _conf_loadmodule (conf=0x81561e8, ce=0x8156210)
    at s_conf.c:7647
#1 0x08073903 in load_conf (filename=0x80aeb61 "unrealircd.conf")
    at s_conf.c:1722
#2 0x08073640 in init_conf (rootconf=0x80aeb61 "unrealircd.conf", rehash=0)
    at s_conf.c:1602
#3 0x0806198c in main (argc=0, argv=0xbffffaf4) at ircd.c:1259
(gdb) frame 0
#0 0x08083783 in _conf_loadmodule (conf=0x81561e8, ce=0x8156210)
    at s_conf.c:7647
7647 if ((ret = Module_Create(files.gl_pathv[i]))) {
(gdb) p *ce
$5 = {ce_fileptr = 0x81561e8, ce_varlinenum = 6, ce_fileposstart = 112,
  ce_fileposend = 148, ce_sectlinenum = 0,
  ce_varname = 0x8156240 "loadmodule",
  ce_vardata = 0x8156250 "src/modules/commands.so", ce_entries = 0x0,
  ce_prevlevel = 0x0, ce_next = 0x8156270}
(gdb) p files
$6 = {gl_pathc = 1, gl_matchc = 135615096, gl_offs = 0, gl_flags = 48,
  gl_pathv = 0x0, gl_errfunc = 0, gl_closedir = 0, gl_readdir = 0,
  gl_opendir = 0, gl_lstat = 0, gl_stat = 0}
(gdb) p files.gl_pathv[i]
Cannot access memory at address 0x0

Thanks for all the work you've put into this :)

syzop

2005-04-28 12:05

administrator   ~0009844

Last edited: 2005-04-28 12:06

 I don't know if zerofilling is required, the manpage isn't that clear about that. But if it is then it would fix glibc reading from unitialized memory. In any case, it makes it easier for debugging (because you go from an <all 0> state to 'after the function call').

Could you download http://www.vulnscan.org/tmp/s_conf.c and drop it in your src/ folder (be sure to overwrite the original src/s_conf.c), and then do a 'make' and run the ircd again...
It should print out various debug output that will help me with what is going on (so just paste it to me).
Also, if it crashes, the usual gdb commands would be helpful again.
Oh and, the output of 'ls -al /lib/libc.so*' would be helpful as well (in other words, your glibc version).

Thanks.

-
Just for reference, this is what happens on my Linux GNU/Debian box w/glibc 2.3.2:
* glob() item #0 (1 avail.) for 'src/modules/commands.so': 'src/modules/commands.so'
* glob() item #0 (1 avail.) for 'src/modules/cloak.so': 'src/modules/cloak.so'
* glob() item #0 (1 avail.) for 'badwords.channel.conf': 'badwords.channel.conf'
* glob() item #0 (1 avail.) for 'badwords.message.conf': 'badwords.message.conf'
* glob() item #0 (1 avail.) for 'help.conf': 'help.conf'


*edit* url 403'd, fixed */edit*

jesterpm

2005-04-28 12:30

reporter   ~0009846

This doesn't look like it's gonna help much.

Here's the glibc stuff:
ircd@Krypton:~$ ls -la /lib/libc.so*
lrwxr-xr-x 1 root root 13 Jan 27 13:00 /lib/libc.so.6 -> libc-2.3.4.so

Here's ./unreal start's output:
ircd@Krypton:~/Unreal3.2$ ./unreal start
Starting UnrealIRCd
 _ _ _ ___________ _____ _
| | | | | |_ _| ___ \/ __ \ | |
| | | |_ __ _ __ ___ __ _| | | | | |_/ /| / \/ __| |
| | | | '_ \| '__/ _ \/ _` | | | | | / | | / _` |
| |_| | | | | | | __/ (_| | |_| |_| |\ \ | \__/\ (_| |
 \___/|_| |_|_| \___|\__,_|_|\___/\_| \_| \____/\__,_|
                           v3.2.3
                     using TRE 0.7.2 (GPL)
                     using OpenSSL 0.9.7e 25 Oct 2004

* Loading IRCd configuration ..
* unrealircd.conf:134: Ignoring extra data
* unrealircd.conf:134: Ignoring extra data
./unreal: line 39: 8185 Segmentation fault (core dumped) /home/ircd/Unreal3.2/src/ircd
Possible error encountered (IRCd seemily not started)


GDB Stuff:
(gdb) bt
#0 0x0808388e in _conf_loadmodule (conf=0x81561e8, ce=0x8156210)
    at s_conf.c:7661
#1 0x08073903 in load_conf (filename=0x80aecc1 "unrealircd.conf")
    at s_conf.c:1722
#2 0x08073640 in init_conf (rootconf=0x80aecc1 "unrealircd.conf", rehash=0)
    at s_conf.c:1602
#3 0x0806198c in main (argc=0, argv=0xbffffaf4) at ircd.c:1259
(gdb) frame 0
#0 0x0808388e in _conf_loadmodule (conf=0x81561e8, ce=0x8156210)
    at s_conf.c:7661
7661 config_status("glob() item #%d (%d avail.) for '%s': '%s'",
(gdb) p *ce
$6 = {ce_fileptr = 0x81561e8, ce_varlinenum = 6, ce_fileposstart = 112,
  ce_fileposend = 148, ce_sectlinenum = 0,
  ce_varname = 0x8156240 "loadmodule",
  ce_vardata = 0x8156250 "src/modules/commands.so", ce_entries = 0x0,
  ce_prevlevel = 0x0, ce_next = 0x8156270}
(gdb) p files
$7 = {gl_pathc = 1, gl_matchc = 135615096, gl_offs = 0, gl_flags = 48,
  gl_pathv = 0x0, gl_errfunc = 0, gl_closedir = 0, gl_readdir = 0,
  gl_opendir = 0, gl_lstat = 0, gl_stat = 0}
(gdb) frame 1
#1 0x08073903 in load_conf (filename=0x80aecc1 "unrealircd.conf")
    at s_conf.c:1722
1722 ret = _conf_loadmodule(cfptr, ce);
(gdb) p *ce
$8 = {ce_fileptr = 0x81561e8, ce_varlinenum = 6, ce_fileposstart = 112,
  ce_fileposend = 148, ce_sectlinenum = 0,
  ce_varname = 0x8156240 "loadmodule",
  ce_vardata = 0x8156250 "src/modules/commands.so", ce_entries = 0x0,
  ce_prevlevel = 0x0, ce_next = 0x8156270}
(gdb) p files
No symbol "files" in current context.

syzop

2005-04-28 15:50

administrator   ~0009848

This is more and more looking like a glibc bug, I'll code a small prog that uses glob and give it to you (here on the bugtracker) -- if it then still crashes then I guess we'll have to report it to the glibc guys.

(btw there's a workaround, I'll give it to you later on as well, so you can at least run unreal ;p)

syzop

2005-04-28 16:19

administrator   ~0009849

Ok, got it finished, just do:
-
wget http://www.vulnscan.org/tmp/globtest.c
gcc -o globtest globtest.c -g -Wall
./globtest
-

This is the output on my machine, which is correct:
-
$ ./globtest
GLOBTEST
Testfile exists, good.
Performing glob() for 'globtest.c'
glob() retcode: 0
files.gl_pathc = 1
files.gl_pathv = 0x804d900
glob match #0: 'globtest.c'
End of glob test
-

Let me know yours (including any warnings/errors generated by gcc if they are present, I had one on my amd64 but that one was harmless) :)

jesterpm

2005-04-28 16:47

reporter   ~0009851

Here's the output:

ircd@Krypton:~$ ./globtest
GLOBTEST
Testfile exists, good.
Performing glob() for 'globtest.c'
glob() retcode: 0
files.gl_pathc = 1
files.gl_pathv = (nil)
Segmentation fault

syzop

2005-04-28 18:26

administrator   ~0009854

Thanks, that looks indeed like a libc issue then. I suggest you take it up to your distributor or glibc team then (they'll probably want this small demonstration program as well, feel free to distribute it).
The weird thing is that it works fine on libc 2.3.2 and 2.3.5 here, so I don't know if there was an issue with 2.3.4 or if it's some other kind of issue.
I've also googled and checked bugzilla on glob() bugs, but nothing like this appeared.

As for a workaround (note that this disabled wildcard supports in includes and loadmodule lines) you can disable globbing support in unrealircd by editting include/setup.h and comment out (add // in front) the' #define GLOBH 1' line in it and recompile. Note that whenever you rerun ./Config you will have to redo this change.

jesterpm

2005-05-02 12:35

reporter   ~0009878

That fixed it :)
Thanks Syzop for all your help

syzop

2005-05-02 12:41

administrator   ~0009880

sure :)

Issue History

Date Modified Username Field Change
2005-04-26 19:24 jesterpm New Issue
2005-04-26 21:35 syzop Note Added: 0009818
2005-04-26 21:36 syzop Note Edited: 0009818
2005-04-26 21:40 jesterpm Note Added: 0009820
2005-04-26 22:00 syzop Note Added: 0009824
2005-04-26 22:18 jesterpm Note Added: 0009825
2005-04-27 00:03 syzop Note Added: 0009827
2005-04-27 00:11 jesterpm Note Added: 0009828
2005-04-27 18:01 syzop Note Added: 0009836
2005-04-27 18:02 syzop Note Edited: 0009836
2005-04-27 18:03 syzop Note Edited: 0009836
2005-04-27 20:27 jesterpm Note Added: 0009837
2005-04-27 20:40 syzop Note Added: 0009838
2005-04-27 20:43 jesterpm Note Added: 0009839
2005-04-27 22:03 syzop Note Added: 0009840
2005-04-27 22:04 syzop Note Edited: 0009840
2005-04-28 00:35 jesterpm Note Added: 0009841
2005-04-28 12:05 syzop Note Added: 0009844
2005-04-28 12:06 syzop Note Edited: 0009844
2005-04-28 12:30 jesterpm Note Added: 0009846
2005-04-28 15:50 syzop Note Added: 0009848
2005-04-28 16:19 syzop Note Added: 0009849
2005-04-28 16:47 jesterpm Note Added: 0009851
2005-04-28 18:26 syzop Note Added: 0009854
2005-05-02 12:35 jesterpm Note Added: 0009878
2005-05-02 12:41 syzop Status new => closed
2005-05-02 12:41 syzop Note Added: 0009880