View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0006345unrealircdpublic2023-10-05 19:232023-10-11 19:13
Reporterarmyn Assigned To 
PrioritynormalSeveritymajorReproducibilityalways
Status newResolutionopen 
Product Version6.1.2.1 
Summary0006345: Spamfilter multiple hits and whitelisting/exempting text
DescriptionI installed the new UnrealIRCd earlier this afternoon and just restored my entire antispam filter list (around 500 blacklisted expressions).

For example, it includes:

/spamfilter $1- c block 0 %reason_uri_block http
/spamfilter $1- c block 0 %reason_uri_block www

Now, try sending this into the channel:
"Test https://www.test.com test2"

It seems the spamfilter has caused some issues. There are exceptions that no longer work, and there's another issue that might be related. The second problem is that if you send "Test https://www.test.com test2," you'll notice the server notices twice for the spamfilter line with "http" and "www," whereas previously it only triggered once. I assume the issue with exceptions not working is why a single user-entered line with 10 filtered words results in 10 spamfilter notices. Is this a bug or a new feature of the spamfilter?
TagsNo tags attached.
3rd party modules

Activities

syzop

2023-10-05 19:28

administrator   ~0023042

Last edited: 2023-10-05 19:28

 Yeah, we no longer stop on first match but try to continue. So that would explain the multiple notices. To the user that can be a bit confusing indeed :D.

What do you mean with "exceptions not working", what sort of exceptions do you mean? Do you have an example of something that doesn't work?

armyn

2023-10-05 19:38

reporter   ~0023043

Yes users can no longer send filtered URLs, with these 2 commands at the end of the list:

    /spamfilter $1- c warn 0 - ^.*(youtu.be\/|v\/|embed\/|watch\?|youtube.com\/user\/[^#]*#([^\/]*?\/)*)\??v?=?([^#\&\?]*).*
    /spamfilter $1- c warn 0 - ^.*(media.discussionner.com\/message-audio|network.discussionner.com|discussionner.com).*

now no exceptions work

is there another solution?

syzop

2023-10-05 19:48

administrator   ~0023044

I see. We don't actually have that sort of "exceptions" so what you are using is more like a "trick" and I don't think it is a good trick.

Let me illustrate by example. Say you allow youtu.be like you just showed me, so this will be allowed:
<Syzop> go to youtu.be/example
Then the user could also send:
<Syzop> don't go to youtu.be/example but to https://www.some-dangerous-site.org/
And it would go through because the line matched the youtu.be exception... ;)
Unless I am missing something...

To really do what you want you would need to have like URL-like extraction and then have whitelists/blacklists for each URL extracted. That's not something spamfilter provides. I have once made a module along those lines, long ago, for 3.2.x or something, and with an old TLD list.

Anyway, if you insist with this not really secure trick, then you can do so via spamfilter { } blocks in the config, because there we have a "stop" action:
spamfilter {
[..]
    action stop;
}

And you can also do several actions, to do that "warn" action plus stopping:

spamfilter {
[..]
    action { warn; stop; }
}

The "stop" action and the multi-actions are not possible via the SPAMFILTER command currently (you will get a strange error).

armyn

2023-10-06 08:30

reporter   ~0023047

Hi,

It's very frustrating that it's no longer possible to make exceptions with the method I was using.

As you can see, I've added three blacklists: one for TLDs like .com, .fr, etc., one for "www," and also one for "http," which also includes https. As a result, all links are blocked in the channels.

I've added two exceptions, as I showed you: one for YouTube links and another for the site "discussionner." This allows users to send "discussionner" links and even upload images and voice messages in the channels. People have been complaining non-stop since yesterday because they can't send links anymore.

The Unrealircd 6.1.2 server is working fine, but if I can't find a solution for links to pass through the channels by tonight, I'll revert to the old version, 6.1.1.1, which is still available. I just need to shut down the new Unrealircd and switch back to the old one.

I tried adding this to my spamfilter.conf file, but it didn't work:

spamfilter {
    match-type regex;
    match "^.*(youtu.be\/|v\/|embed\/|watch\?|youtube.com\/user\/[^#]*#([^\/]*?\/)*)\??v?=?([^#\&\?]*).*";
    target { channel; }
        action { warn; stop; }
}

It was added to the end of the spamfilter.conf file. I'm not sure how to use it correctly.

In summary, the spamfilter blacklists include some major TLDs like ".com," ".net," ".fr," and "http" and "www," that's it.

I should be able to block the sending of all URLs in the channels except those defined in a list or a regex.

syzop

2023-10-06 10:04

administrator   ~0023048

Yeah it sounds to me like you better downgrade to 6.1.1.1 and then think of a new plan for the future.

As I explained with my examples, what you did was not secure either. Everyone could bypass such a restriction.

armyn

2023-10-06 12:16

reporter   ~0023049

Ah, that's good, it's working.

Right after configuring in the spamfilter.conf file, don't forget to delete all the /spamfilter entries with "del" and then re-add them, or else it won't work.

Here is my spamfilter.conf:

spamfilter {
    match-type regex;
    match "^.*(youtu.be\/|v\/|embed\/|watch\?|youtube.com\/user\/[^#]*#([^\/]*?\/)*)\??v?=?([^#\&\?]*).*";
    target { channel; }
    action { warn; stop; }
}

spamfilter {
    match-type regex;
    match "^.*(media.discussionner.com\/message-audio|network.discussionner.com|discussionner.com).*";
    target { channel; }
    action { warn; stop; }
}


Then, remove all the /spamfilter add commands and re-add them

Yes it's not secure, but it suits me, it's better than nothing

armyn

2023-10-08 14:11

reporter   ~0023053

Hi, It was a false alarm. It doesn't work. It does work, but only from a fixed IP that is added as an exception in unrealircd.conf in various places. It's strange because for the test user, when I remove the exceptions, it no longer works, and I can't send the link in the channel. When I add the exception back, it works, and I can send YouTube links in the channel, but only from my fixed IP, not with others or other users. I will revert to the old version of UnrealIRCd this afternoon.

Issue History

Date Modified Username Field Change
2023-10-05 19:23 armyn New Issue
2023-10-05 19:28 syzop Note Added: 0023042
2023-10-05 19:28 syzop Note Edited: 0023042
2023-10-05 19:38 armyn Note Added: 0023043
2023-10-05 19:48 syzop Note Added: 0023044
2023-10-06 08:30 armyn Note Added: 0023047
2023-10-06 10:04 syzop Note Added: 0023048
2023-10-06 11:04 syzop Summary Problem in the spam filter with or without utf8 => Spamfilter multiple hits and whitelisting/exempting text
2023-10-06 12:16 armyn Note Added: 0023049
2023-10-08 14:11 armyn Note Added: 0023053