View Issue Details

IDProjectCategoryView StatusLast Update
0002875unrealircdpublic2006-04-20 18:43
ReporterZellAssigned Tosyzop 
PrioritynormalSeveritycrashReproducibilityalways
Status resolvedResolutionfixed 
Platformi386OSWin32OS Version5.1.2600
Product Version3.2.4 
Target VersionFixed in Version3.2.5 
Summary0002875: VStudio 2005: Crashes on link to any server, stats, or services
DescriptionGenerated at Thu Apr 06 22:15:23 2006

Windows XP Service Pack 2 (5.1.2600)
Unreal3.2.4[FinWXeOoZE] (1.1.1.1.2.22 2006/02/05 18:03:15)
-----------------
Memory Information:
    Physical: (Available:44MB/Total:238MB)
    Virtual: (Available:2007MB/Total:2047MB)
-----------------
Exception:
    Access Violation
-----------------
Backup Buffer:
    SERVER Crystal.WA.US.NothingButStyle.Net 1 :U2306-FhinXeOoZEM-20 Server1
-----------------
Registers:
    EAX=0x00000903 EBX=0x00db5d8f ECX=0x7ffffffe
    EDX=0x00498f73 ESI=0x00000000 EDI=0x00000903
    EIP=0x78141bcd EBP=0x010cf56c ESP=0x010cf4e0
-----------------
Stack Trace:
    Module: MSVCR80
        #0 (null):0: vfwprintf_p
        #1 (null):0: vfwprintf_p
        #2 (null):0: vfwprintf_p
    Module: ntdll
        #3 (null):0: RtlAllocateHeap
        #4 (null):0: RtlAllocateHeap


* Unmodified source, using SSL+Zip,
Installed stuffie:
Platform SDK - tried on diff machines/versions with same result
VC Express 2005 - tried on VC2005, VC6 with same result
TagsNo tags attached.
3rd party modules

Relationships

related to 0002704 resolvedsyzop Self-compiled ircd w/MS VStudio 8 beta: Unreal Crashes on Join 
related to 0002890 resolvedsyzop [warnings] Warnings on compile - POSIX Deprecation 

Activities

Zell

2006-04-06 17:22

reporter   ~0011480

by the way this is with the self-compiled from source, not the prebuilt one on the site.

syztest100

2006-04-06 19:31

reporter   ~0011481

yeah, I was about to ask... presumably that one works just fine? :p

I think the bug mentioned here (crash on link) is also mentioned in one of the other crash reports with those newer VStudio's.

Odd you also crash with VC6 though (same backtrace??). Or actually, even more interesting you managed to compile, since I get other reports from people VC6 doesn't even compile current unreal anymore.. then again that could be their fault ;).

I'm not so sure why this crash stuff is happening with MSVC 2005... Most of the backtraces and debug reports from people I saw didn't have anything bad and/or were illogical. In all cases, the crashes were reproducable (by these persons) though.

Personally I'm using MSVC 7.0, and as far as I know MSVC 7.1 works well too. So if you can get your hands on such a version, I suggest you to try those.

I must confess I'm not giving this top priority, simply because "it works for me"... To be able to fix this I guess I would need to install a computer or vmware with this newest vstudio version (free version), and then start debugging. Somehow I suspects it's not going to be an easy thing, probably something obscure... The first thinig I would (suggest to) look at though, is the libs... I kinda suspect that there's an incompatibility between c-ares built on MSVC7 and then the rest of the ircd in MSVC6 or MSVC8... the c-ares source is publicly available so one could build it themself on MSVC8 (unfortunately the c-ares source is generally broken on windows so you need to hack things in to make it work..).
If libs are not the problem, then it would be even more fun.

Anyway, if I think of it, I'll put online my fixed c-ares source I used for 3.2.4 (and still current CVS) so you can give that a try.
Though, don't expect much other action from my side... so some help (testing etc) would be appreciated ;p

Zell

2006-04-06 19:40

reporter   ~0011482

I'm more than willing to help out with Windows development, as I've spent hours myself debugging crap on here.

Yeah, that C-Ares lib sucks on windows. Ive made it compile, but the resulting lib was like 400KB versus the 72KB one included with Unreal o.O

Yeah, VC6 compiled. It was a PITA to get, not sure about the backtrace I erased them already, and i think i ended up building it off of the 2003 SDK. I have more trouble with the April 2005 SDK than the 2003 SDK. (I have a 2003, 2004, and 2005 SDK on cd, ordered them from microsoft to save download time lmfao). Note that in windows, with VC2005 and the latest SDK, the "Open SDK Command Prompt" shortcut is completely broken (forcing the user to set environment variables manually.) (Apparently, it requires an older version of Visual Studio (7.0, maybe?).

Can you post what exactly you used to make the one that is posted on the site? (That one does work.) If you can, i'd like the dotted version number of your Visual C software, which release of the PSDK you have, and which OS (or VMWare'd OS) you are using. I have every release of windows (except Vista), so I'd be able to easily replicate your environment (once obtaining 7.0, of course.)

syzop

2006-04-07 05:56

administrator   ~0011484

Last edited: 2006-04-07 06:01

Good ;)

I've uploaded my (compiled!) c-ares directory zipped to:
http://www.vulnscan.org/tmp/c-ares-1.3.0-unreal.zip
(side note: the zip does not create a c-ares directory!)

To compile it, open vc\areslib\areslib.vcproj in vstudio (not .dsp or the other ones, they are still unconverted / old vc6 stuff).
Then:
- build -> clean solution
- build -> build areslib
That should be all.
It's easy once the source is fixed ;)

The resulting .lib will be in vc\areslib\Release\areslib.lib

Thanks :P

EDIT: hm, I hit the add button too early, upload finished now ;p

Zell

2006-04-09 23:48

reporter   ~0011504

Successfully built the areslib with 0 errors.

Added to unreal, compile etc.

Result: CRASH!!!, but differently.

Before, on linking OUT to a server, it would have 3 entries about vfwprintf_p
Before, on linking IN to the Win32 server, it would only have 3 entries about vfwprintf_p and two entries under ntdll

NOW, on linking OUT to a server, it has only one entry about vfwprintf_p
NOW, on linking IN to the Win32 server, it still has 3 entries about vfwprintf_p, but no entries about ntdll.

Tested 3 times each for both scenarios (total of 6 link attempts and 6 crashes)

syzop

2006-04-10 06:50

administrator   ~0011508

Hm.. what a pitty...
I had hoped the solution was as easy as this ;). Guess not...

Zell

2006-04-10 13:06

reporter   ~0011518

Its possible that the areslib was part of the problem, but not the entire problem? Since updating it there are *less* errors on the backtrace... lol

I've been able to bypass the crash by commenting a few lines / making the m_server function return early. Perhaps stepping through it line by line is the only way?

syzop

2006-04-10 13:09

administrator   ~0011519

Ic.

Could you compile latest CVS? This contains minidump functionality on-crash, which means it will dump some semi-useful info to a .mdmp file.
If you could then mail this .mdmp file to me along with your zipped unreal directory, hmm...
wait..
nevermind
then I also need VC2005 since older versions can't debug newer versions
so that won't work.... CRAP :P


Yes, if you could somehow step line by line or find out where it is, it would definately help...

Zell

2006-04-10 13:54

reporter   ~0011522

Syzop, i took it apart with a disassembler/debugger and the error definitely occurs in the C Runtime due to a bad call from Unreal.

I dunno how much you know about registers and blablabla but heres where the fault is:

MSVCR80: 0x78141BCD: CMP BYTE PTR DS:[EAX],0
where [EAX] is 0x00000903
0x00000903 is invalid, causing a 0xC0000005 exception (Access Violation)

Which means theres probably something that isnt working right in windows version of the code since this obviously works in *nix.

I'll start hunting down the exact troublesome line now.

Yamake

2006-04-10 20:42

reporter   ~0011529

This is an interesting bug. Zell, are you getting *any* crashes at all when you join channels?

Zell

2006-04-10 22:00

reporter   ~0011530

No, nothing like that. The only issue I have so far is with linking. Please do note, Yamake, that my Operating system is Windows XP Professional, Service Release 2.

Zell

2006-04-13 16:31

reporter   ~0011541

a bunch of line-by-line hunting heeded the result:

m_server() -- a good function

the server crashes when m_server_synch() is called, i'm going to hunt that now.

2006-04-13 17:42

 

m_server.zip (8,282 bytes)

syzop

2006-04-13 17:46

administrator   ~0011542

Last edited: 2006-04-13 17:47

Good job ;P

I've attached a m_server.zip which contains m_server.c with a lot of debug crap added. If you could replace your m_server.c with this one and see what you get, that should help. If you could then past the last, say, 10 lines of ircd.log output here.. :)

PS: I hope I didn't break any of the actual code when adding these tons of lines... but we'll see about that when we get the results ;).

PPS: This is an editted m_server.c from cvs, but I *presume* that should be ok. (hint: don't upgrade to latest CVS, since I have changed the c-ares library, so it won't work).

Zell

2006-04-13 18:14

reporter   ~0011543

I havent looked at your file yet as I have discovered an atrocity with the Server module. The IRCd actually doesnt output anything to the user till the 'return' is called. (I had a debug sendto_realops("1"); after every line and it crashed without sending "1" to all opers even once. I had to change it to

sendto_realops("1");
return 0;

and just move it around to find the spot.

Zell

2006-04-13 18:16

reporter   ~0011544

* Bug found *

m_server() -> m_server_synch() -> tkl_synch()

tkl_synch() causes segfault IF AND ONLY IF there are any timed global lines passed to the linking server, else it continues... (and crashes somewhere else, im about to paste that in a sec)

Zell

2006-04-13 18:20

reporter   ~0011545

The CRASHBUG Found.

sendto_one(cptr, "%s %i %li %i %s 0 0 0 :%s",
        (IsToken(cptr) ? TOK_NETINFO : MSG_NETINFO),
        IRCstats.global_max, TStime(), UnrealProtocol,
        CLOAK_KEYCRC,
        ircnetwork);

Causes the segfault.

Zell

2006-04-13 18:22

reporter   ~0011546

Idea: i'll take it apart and send it one parameter at a time till it breaks.
We can deal with the tkl_synch() crashbug after this is resolved, as the tkl sh*t is in another file. blah.

I'm using a mIRC script that fakes a server so I can see all raw output. This makes offline debugging amazingly simple.

Zell

2006-04-13 18:38

reporter   ~0011547

i think that might be an issue with the cptr pointer in parameter 0 of sendto_one, because when i take apart the statement:

sendto_realops("%s", IsToken(cptr) ? TOK_NETINFO : MSG_NETINFO);
sendto_realops("%s", MSG_NETINFO);
sendto_realops("%i", IRCstats.global_max);
sendto_realops("%li", TStime());
sendto_realops("%i", UnrealProtocol);
sendto_realops("%s", CLOAK_KEYCRC);
sendto_realops("%s", ircnetwork);

It works fine.
But when its in the sendto_one, it f*cks up. Ideas?

Zell

2006-04-13 19:30

reporter   ~0011550

The offending line that makes tkl_synch() crash also is a sendto_one() problem... m_tkl.c, line 1729, 1738.

i'm at a loss for what is causing this problem. with the cptr in sendto_one.

Zell

2006-04-13 21:47

reporter   ~0011551

Last edited: 2006-04-13 22:23

Another crashbug. (I "bypassed" the tkl_sync(), NETINFO functions that cause the crashbug that prompted this report)

Got them to link, but it crashes on something else (token '&') as well. I'm beginning to see a pattern here... Microsoft sucks.

I could post the backtrace, but it's the same as the one at the top (with a different fault address, but the same MSVCR80->vfwprintf_p)

syzop

2006-04-14 07:16

administrator   ~0011554

Hmmm.. strangestrange :/

I hoped it was some kind of bug in our code that was getting triggered by this new compiler (this sometimes happens), but guess not...

Hmz.. well I guess I need to install a vmware with all that stuff myself then... But I don't have much time for this right now.

I see you once posted a comment requesting what my build environment was (so to know which environment actually works).
Well, I'm using MSVC 7.0 (but I know MSVC 7.1 works as well). As for the PSDK.. I think that's included in MSVC 7.x.

Zell

2006-04-14 13:21

reporter   ~0011555

Well... the exact error when you dig deep into the assembly using debuggers and stuff, the fault occurs when the program references an invalid address (yes, we all know what an access violation is..).

The thing I cant figure out is where it's calling the invalid addy from in the source code. I suppose with my free time I can see if I can get VC2005 to attach the source and the running process and step it line by line till it fails. May take a few days, but I'll work on it.

Yamake

2006-04-15 12:01

reporter   ~0011557

Last edited: 2006-04-15 12:01

Actually, in reference to ID 0002704 -- Unreal will crash on join ***AND*** all that you have mentioned, and I am using Visual Studio 8 on XP Pro SP 2. I've stepped through the code, however, and found absolutely nothing (odd o.o). I'm going to try compiling 3.2.4 and see what happens there, but I'm not sure what exactly will happen. Should I post any further backtraces here, or in 0002704?

-Yamake

syzop

2006-04-15 12:41

administrator   ~0011558

Last edited: 2006-04-15 12:43

any backtrace which includes some unreal files (so not only vfwprintf in MSVCR80 and stuff in ntdll etc, but something useful like xxxxx.c:xxx) would be welcome if posted here.
Then again, I'm not sure if it still looks doable to debug this remotely...

This whole issue might as well be something weird DLL memory mapping/accesspermission stuff related... seeing I get claims of cptr being NULL (here) and chptr(->xxx) being NULL (other bugid)

2006-04-18 21:12

 

vc2005-bug2875-fix.patch (2,575 bytes)

2006-04-18 21:13

 

vc2005-vsprintf-fix.patch (1,007 bytes)

2006-04-18 21:27

 

vc2005-vsnprintf-fix.patch (1,060 bytes)

Xuefer

2006-04-18 21:27

reporter   ~0011574

FYI, MSDN said: If _USE_32BIT_TIME_T is defined, time_t is a long integer. If not defined, it is a 64-bit integer.

updated, and _renamed_ vc2005-vsprintf-fix.patch vc2005-vsnprintf-fix.patch
vc2005-vsnprintf-fix.patch is not required by this bug, but kill a lot of warnings about vsnprintf. they have just added vsnprintf .... but snprintf is introduced as _snprintf, thus:
+# define HAVE_SNPRINTF
+# define snprintf _snprintf
is not required to kill the warnings.

syzop

2006-04-19 07:46

administrator   ~0011579

Last edited: 2006-04-19 07:47

Thanks Xuefer :). Added in CVS [.491].

/G5 is not supported, maybe i have some component not installed
DONE, REMOVED

/debugtype:BOTH was not supported, and now not recognized
DONE, REMOVED

/Od to disable optimization, kill the pain debuging
DONE, HALF-ADDED (comment on that later)

/MDd ... u know
DONE, HALF-ADDED (comment on that later)

/D _CRT_SECURE_NO_DEPRECATE kill warning about strcat etc is insecure, untill unreal decide to fix all those calls.
ADDED. CLARIFICATION: btw, there's nothing wrong or insecure about strcat and strcpy IF you use them right, unfortunately some people try to pretend otherwise (MS, and some ""security people""). In our case strcpy/strcat is even better due to superior performance ;). If you are bored, check out the 'strlcpy' discussion on the glibc mailinglist to see why they never added it (note that I don't agree with exactly that, I would like to be able to use BOTH strcpy/strcat AND strlcpy/strlcat/strlncat, depending on the situation...). Anyway, enough off-topic talk ;)

/D _USE_32BIT_TIME_T fix bug 0002875
ADDED

Regarding the half-added things: we always compile(d) releases with DEBUG=1 on, so we can get a lot of debuginfo. I've renamed it to DEBUGEXTRA and added your flags there (/MD->/MDd and added /Od) with some comments.. see the new makefile.win32 for more info ;p.


Anyway, thanks for your work. If you, or actually even better: others, can verify that the fixes in CVS now appear to be working fine, then I'll close this bug.

EDIT: Forgot to mention it was added in cvs / releaseid.

Zell

2006-04-19 11:17

reporter   ~0011581

syzop can you pull a tarball off the latest CVS with your changes and I'll download it and test it on mine?

In the meantime, i'll add in the additions manually.
Yamake: yeah, I had already implemented a solution for the vsnprintf warnings, with something similar.

Hopefully this fixes the issue.

Yamake

2006-04-19 15:16

reporter   ~0011584

Last edited: 2006-04-19 15:17

I'm going to see if it will run under the new define flag.

syzop

2006-04-20 07:50

administrator   ~0011587

The fixes are already in CVS, so you can grab that...

If it's a problem getting CVS, you can grab my daily .tar.gz's of CVS from http://www.vulnscan.org/cvs (they are now generated again) :)

Zell

2006-04-20 17:26

reporter   ~0011591

[fixed]
The CVS Release as of file Unr3.2-20060420.tar.gz:

*** BUG FIXED ***
(sync) Link Crystal.WA.US.NothingButStyle.Net -> Garnet.NC.US.NothingButStyle.Net is now synced [secs: 2 recv: 3.737 sent: 0.549]

=) (somehow i knew this bug was microsoft's fault...)

syzop

2006-04-20 18:43

administrator   ~0011593

Thanks for the confirm

Issue History

Date Modified Username Field Change
2006-04-06 17:22 Zell New Issue
2006-04-06 17:22 Zell Note Added: 0011480
2006-04-06 19:31 syztest100 Note Added: 0011481
2006-04-06 19:40 Zell Note Added: 0011482
2006-04-07 05:56 syzop Note Added: 0011484
2006-04-07 05:58 syzop Note Edited: 0011484
2006-04-07 06:01 syzop Note Edited: 0011484
2006-04-09 23:48 Zell Note Added: 0011504
2006-04-10 06:50 syzop Note Added: 0011508
2006-04-10 13:06 Zell Note Added: 0011518
2006-04-10 13:09 syzop Note Added: 0011519
2006-04-10 13:20 syzop Summary [link] Crashes on link to any server, stats, or services => VStudio 2005: Crashes on link to any server, stats, or services
2006-04-10 13:21 syzop Status new => acknowledged
2006-04-10 13:25 syzop Relationship added related to 0002704
2006-04-10 13:54 Zell Note Added: 0011522
2006-04-10 20:42 Yamake Note Added: 0011529
2006-04-10 22:00 Zell Note Added: 0011530
2006-04-13 16:31 Zell Note Added: 0011541
2006-04-13 17:42 syzop File Added: m_server.zip
2006-04-13 17:46 syzop Note Added: 0011542
2006-04-13 17:47 syzop Note Edited: 0011542
2006-04-13 18:14 Zell Note Added: 0011543
2006-04-13 18:16 Zell Note Added: 0011544
2006-04-13 18:20 Zell Note Added: 0011545
2006-04-13 18:22 Zell Note Added: 0011546
2006-04-13 18:38 Zell Note Added: 0011547
2006-04-13 19:30 Zell Note Added: 0011550
2006-04-13 21:47 Zell Note Added: 0011551
2006-04-13 22:23 Zell Note Edited: 0011551
2006-04-14 07:16 syzop Note Added: 0011554
2006-04-14 13:21 Zell Note Added: 0011555
2006-04-15 12:01 Yamake Note Added: 0011557
2006-04-15 12:01 Yamake Note Edited: 0011557
2006-04-15 12:41 syzop Note Added: 0011558
2006-04-15 12:43 syzop Note Edited: 0011558
2006-04-18 21:12 Xuefer File Added: vc2005-bug2875-fix.patch
2006-04-18 21:13 Xuefer File Added: vc2005-vsprintf-fix.patch
2006-04-18 21:27 Xuefer File Added: vc2005-vsnprintf-fix.patch
2006-04-18 21:27 Xuefer Note Added: 0011574
2006-04-19 07:46 syzop Note Added: 0011579
2006-04-19 07:47 syzop Note Edited: 0011579
2006-04-19 11:17 Zell Note Added: 0011581
2006-04-19 15:16 Yamake Note Added: 0011584
2006-04-19 15:17 Yamake Note Edited: 0011584
2006-04-20 07:50 syzop Note Added: 0011587
2006-04-20 17:26 Zell Note Added: 0011591
2006-04-20 18:20 syzop Relationship added related to 0002890
2006-04-20 18:43 syzop Status acknowledged => resolved
2006-04-20 18:43 syzop Fixed in Version => 3.2.5
2006-04-20 18:43 syzop Resolution open => fixed
2006-04-20 18:43 syzop Assigned To => syzop
2006-04-20 18:43 syzop Note Added: 0011593