View Issue Details

IDProjectCategoryView StatusLast Update
0004356unrealircdpublic2015-06-03 11:26
Reportersyzop Assigned Tosyzop  
Status resolvedResolutionfixed 
Product Version3.4-alpha3 
Target Version3.4-alpha3Fixed in Version3.4-alpha3 
Summary0004356: Move to another regex engine
DescriptionThis was discussed back in 2008 already or so: we should move away from TRE, which has long-standing unsolved bugs and still not ideal performance, to another regex engine.
As already pointed out back then, best is to move to PCRE rather than picking yet another obscure POSIX engine. Nowadays this is only more true when you see POSIX regex functions being considered deprecated in PHP for example.

So in 3.4.x we want to use PCRE, to have something maintained (=less bugs) and fast (=better performance).

Of course, POSIX and PCRE don't mix perfectly. You cannot safely convert one to the other AFAICT. When it comes to mass-killing of users with spamfilters this is quite important :D
Linking 3.2.x - 3.4.x should still work reasonably OK. I think non-functional spamfilters are a showstopper that would prevent people from deploying 3.4.x if they already have a 3.2.x network, and thus would hinder getting 3.4.x becoming popular.

So best to be compatible. How about this:
Current syntax:
/spamfilter [add|del|remove|+|-] [type] [action] [tkltime] [reason] [regex]

Proposed syntax:
/spamfilter [add|del|remove|+|-] [-regex|-posix|-simple] [type] [action] [tkltime] [reason] [regex]

So add an -xxxxx to specify the matching engine type.

Would provide 3 options:
* New PCRE engine (-regex)
* Old TRE POSIX engine (-posix)
* Simple globbing with * and ? to have something fast and understandable for everyone (-simple) (requested in bug ...)

For 3.4.x:
* Make specifying this -enginetype mandatory. Print out a helpful error if you don't specify it (easy to detect, the next field, type, will never start with a '-' anyway.
* -posix should print out a warning saying that it's deprecated (but still working)
* detect linking with 3.2.x (or rather: non-3.4.x-versions-that-support-this): then 1) don't send the PCRE regexes over the wire and 2) send out a notice to all opers when this happens, so on serverlink and on /spamfilter add...
TagsNo tags attached.
3rd party modules



2015-05-30 15:18

administrator   ~0018377

Also a good idea to add an abstract layer I think, for matching in general. So we can just throw in a struct of some type somewhere and support regular wildcards and regexes. For the occasional request to make feature XYZ support regex. Also good when changing regex engines.


2015-06-03 11:26

administrator   ~0018381

Done in 3.4-alpha3

Issue History

Date Modified Username Field Change
2015-05-30 15:14 syzop New Issue
2015-05-30 15:14 syzop Assigned To => syzop
2015-05-30 15:14 syzop Status new => assigned
2015-05-30 15:18 syzop Note Added: 0018377
2015-06-03 11:26 syzop Note Added: 0018381
2015-06-03 11:26 syzop Status assigned => resolved
2015-06-03 11:26 syzop Fixed in Version => 3.4-alpha3
2015-06-03 11:26 syzop Resolution open => fixed