View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0004046unrealircdpublic2011-10-30 22:202015-05-23 19:06
Reporterseraph Assigned To 
PriorityimmediateSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
PlatformallOSallOS Versionall
Product Version3.2.9-RC2 
Fixed in Version3.4-alpha3 
Summary0004046: SSL Renegotiation Flood
DescriptionTHC released a easy-to-use-tool for flooding SSL based server applications causing high load due to permanent renegotiation of the SSL key.

Tested it using 3 parallel connections and caused 21% CPU one one core of my atom D525/debian lenny.

Tool: http://www.thc.org/thc-ssl-dos/

My recommendations:
Either set an Option to disable renegotiation caused by the client completely
or enforce the options set at set::ssl as strict rule not allowing any other happenings.

Due to the fact that this tool is easy usable it should be urgently fixed at least in .9 stable release.
Steps To Reproducestart the tool and attack a SSL port of the ircd
TagsNo tags attached.
3rd party modules

Activities

syzop

2011-10-31 09:37

administrator   ~0016766

Believe it or not, I was about to release 3.2.9 today. I already wrote the release announcement ;)

Totally disabling renegotiation is no option. It would break things.

I don't see an OpenSSL option to limit the # of renegotiations, there's no max. limit and not a #/minute limit.

I found BIO_get_num_renegotiates, which supposedly returns the # of renegotiations. I presume this counts both renegotiations initiated by the server and the client. If this is true, then we can handle this ourselves.

seraph

2011-10-31 09:54

reporter   ~0016767

ok will it be fixed in .9 already?
I feel kind of ashamed for delaying the release ^^

syzop

2011-10-31 10:38

administrator   ~0016768

Well 1 or a few days doesn't matter now anymore ;)

Okay, BIO_get_num_renegotiates is of no use to us, it only returns the # of renegotiations initiated by ourselves (caused by the max bytes / max time limits).

That said, I'm having difficulty with this THC proof-of-concept DoS... it seems instead of renegotiating it rather disconnects due to ssl errors.

syzop

2011-10-31 10:41

administrator   ~0016769

If you use this to connect to an SSL server:
openssl s_client -connect IP:PORT

and then type R + Enter (so 'R' on it's own line), then it will renegotiate.
However if you do this a 2nd time, then the server disconnects.

That looks good, but I don't know why it does that ;)

The same happens if you renegotiate, then do 'user x x x x' and 'nick xyz', and then renegotiate again... connection is dropped.

Since multiple renegotiations are normal (eg: bytes/time exceeded) this would be odd as it would fail existing servers (which AFAICT is not happening), unless OpenSSL would have some limit of # renegotiations per 'something', which it does not, AFAIK.

syzop

2011-10-31 11:42

administrator   ~0016770

Last edited: 2011-10-31 11:44

 Not sure how to proceed.

I also tried to force renogitation in the native UnrealIRCd code, to see if I can make that happen at all, I did so by this insane block:
set {
        ssl {
                renegotiate-timeout 10s;
                renegotiate-bytes 1k;
        };
};

However I can't see any renegotiations in the wireshark output

Also I added this in exit_client:
        if (MyConnect(sptr) && IsSecure(sptr))
        {
                long r, r2;
                r = BIO_ctrl(SSL_get_rbio(sptr->ssl),BIO_C_GET_SSL_NUM_RENEGOTIATES,0,NULL);
                r2 = BIO_ctrl(SSL_get_wbio(sptr->ssl),BIO_C_GET_SSL_NUM_RENEGOTIATES,0,NULL);

                ircd_log(LOG_ERROR, "%s: %ld/%ld handshakes for this session", get_client_name(sptr, FALSE), r,r2);

        }

However in all cases it returns 0/0.
'All cases' being:
1) The server-server test with low bytes/time for reneg.
2) mIRC
3) THC DoS

Like I said, not sure how to proceed from here.

EDIT: I should note that I read the OpenSSL source and can't find an explanation why '1' does not appear to work. It does a if t<60 then t=5, and it does require >=512 bytes, but that's all.. it should 'just work' AFAICT.

syzop

2011-10-31 12:30

administrator   ~0016771

Ok, setting asside the renegotiate-timeout&-bytes which don't seem to be working for me (bug?)....

I added a command to renegotiate, and when f.e. a server is connected I can run that XX times a second, and then indeed the CPU usage goes up.

Which brings us back to detecting & preventing this, at this point it seems this is not possible as OpenSSL does not give us this information (as said, BIO_get_num_renegotiates is of no use).

Disabling renegotiations would prevent client certificates from working if I understand it correctly.
It also would prevent the renegotiation after an hour or so (and the other timeout&-bytes setting) from working, defeating some security.

So, seems at this point, there's nothing we can do.

seraph

2011-10-31 12:47

reporter   ~0016772

I hope there will be a solution given with the next release of openssl, I think due to this tool release they are forced to do something against this.

syzop

2011-10-31 12:52

administrator   ~0016773

ha.. never give up ;)

SSL_num_renegotiations()

However, next time I can work on Unreal is later this week (or weekend), I'll see if I can come up with a patch.
Or, if binki or someone else is interested in doing this:
my idea is this:
1) check this SSL_num_renegotiations(cptr->ssl) every.. well.. read? maybe double check for overhead, but it's only returning a value, so I doubt it's noticeable.
2) add two values to the client struct (no other choice here), a counter and a timestamp, and with the help of that limit the number of renegotiations to <something> per <something>. Like a value of 3 per 10 minutes should do the job.
3) If possible, exempt servers from this (once they are authorized, of course). Because in high traffic situations they can reach the reneg-bytes/time.

seraph

2011-10-31 14:22

reporter   ~0016774

Last edited: 2011-10-31 14:23

 098765430ztydfghjklöäxyän1


//edit WHOOPS just used the vakuum cleaner, sry

syzop

2011-10-31 19:37

administrator   ~0016775

hehe ;)

ok, seems SSL_num_renegotiations() is only updated on renegotiations we initiate, that isn't useful, so we're back at the situation a few comments ago.. can't do anything about it ourselves.

I wonder if the OpenSSL team will take it seriously; there's always an inherent danger of using SSL with regards to experiencing high CPU usage. However, this particular issue is an extreme case (more CPU % than the usual encryption operations) and something which can be avoided, so IMO they should fix this.

This means I'll probably continue with my release schedule...

syzop

2012-01-22 17:39

administrator   ~0016896

I haven't had a chance to test, but I wonder if in latest OpenSSL beta's they addressed it... I couldn't really find anything about it in the NEWS / changelog so I'm pretty skeptical.

I also did find out that it's possible to detect a handshake by adding an SSL info callback.
SSL_set_info_callback(acptr->ssl, ssl_info_callback);
..
void ssl_info_callback(const SSL *ssl, int where, int ret)
{
       if (where & SSL_CB_HANDSHAKE_START)
       {
               ircd_log(LOG_ERROR, "Handshaking..."); /* DEBUG / FIXME */
       }
}

though then you still need to set some private data, so that function knows which acptr... and you still need to add counters and time to the client struct...

lot of work for fixing something that IMHO should be addressed in OpenSSL itself!

nenolod

2013-05-06 07:27

reporter   ~0017500

I will look into this as part of the hardening work in 3.4.

syzop

2015-05-23 19:06

administrator   ~0018333

Thanks seraph, this has now been fixed in 3.4-alpha3:
Add protection against SSL Renegotiation attacks (0004046).
https://github.com/unrealircd/unrealircd/commit/3623ebae05a89fb158459a1513fb87c8186caa71

Issue History

Date Modified Username Field Change
2011-10-30 22:20 seraph New Issue
2011-10-31 09:37 syzop Note Added: 0016766
2011-10-31 09:54 seraph Note Added: 0016767
2011-10-31 10:38 syzop Note Added: 0016768
2011-10-31 10:41 syzop Note Added: 0016769
2011-10-31 11:39 syzop Product Version => 3.2.9-RC2
2011-10-31 11:42 syzop Note Added: 0016770
2011-10-31 11:44 syzop Note Edited: 0016770
2011-10-31 12:30 syzop Note Added: 0016771
2011-10-31 12:47 seraph Note Added: 0016772
2011-10-31 12:52 syzop Note Added: 0016773
2011-10-31 14:22 seraph Note Added: 0016774
2011-10-31 14:23 seraph Note Edited: 0016774
2011-10-31 19:37 syzop Note Added: 0016775
2012-01-22 17:39 syzop Note Added: 0016896
2013-05-06 07:27 nenolod Note Added: 0017500
2015-05-23 19:06 syzop Note Added: 0018333
2015-05-23 19:06 syzop Status new => resolved
2015-05-23 19:06 syzop Resolution open => fixed
2015-05-23 19:06 syzop Fixed in Version => 3.4-alpha3
2015-05-23 19:06 syzop View Status private => public