View Issue Details

IDProjectCategoryView StatusLast Update
0005287unrealircdpublic2019-08-18 15:17
ReportersyzopAssigned Tosyzop 
PrioritynormalSeverityfeatureReproducibilityN/A
Status resolvedResolutionfixed 
Product Version 
Target VersionFixed in Version5.0.0-alpha1 
Summary0005287: U5: Channel history
DescriptionSo... we want channel history recording and playback in UnrealIRCd. I have been doing this for at least 10 years with my own custom mods and I think it'd be nice for in official UnrealIRCd as well.

Channel mode:
Should it be opt-in or opt-out? I think opt-in for sure, so it's always clear that history is being recorded.
Now, as for InspIRCd, it has "+H count:period" to limit the channel history
But... I think there is more to it than that.

We have two things:
1) The history to playback on-join (limited in number of lines / time)
2) The total history to keep

Especially now that IRCv3 has CHATHISTORY (and I've had something like this in the past as well)

So... that means we have 2 settings (or 4 actually):
1. Maximum number of lines to send on-join
2. Maximum number of time period to send on-join
3. Maximum number of lines to keep (for CHATHISTORY command)
4. Maximum number of time period for logs to keep (for CHATHISTORY command)

And naturally the limits of 1 & 2 must always be less or equal than 3 & 4.

I'm thinking 3 & 4 are really a channel policy decission that chanops/owners/whatever should take.
But 1 & 2... I don't think this should be up to chanops. It should be a global decision (ircd) or even a personal decission (by user)
TagsNo tags attached.
3rd party modules

Relationships

related to 0005290 acknowledged U5: extban to prevent channel history 
child of 0005279 acknowledged UnrealIRCd 5 master tracking issue 

Activities

syzop

2019-05-18 18:30

administrator   ~0020683

set {
        chanlog {
                on-join {
                        maxlines 10;
                        time 1d;
                }
                maximum {
                        maxlines 500;
                        time 7d;
                }
        }
}

As for rules:
1) +H setting...
2) ...with a maximum of set::chanlog::maximum
3) If this +H setting is lower than set::chanlog::on-join then of course the lowest ones of them wins (eg in case of +H 5:3600)

syzop

2019-06-09 19:13

administrator   ~0020726

Last edited: 2019-06-09 20:30

View 5 revisions

I'm currently writing the module. I think it will be like this in the configuration file:
set {
        history {
                channel {
                        /* How many lines to playback on join? */
                        playback-on-join {
                                lines 10;
                                time 1d;
                        }
                        /* How much history to keep. These are the
                         * upper maximums for channel mode +H lines:time
                         */
                        max-storage-per-channel {
                                lines 5000;
                                time 1w;
                        }
                }
        }
}


Means it's a bit more future-proof. And I think the "playback-on-join" and "max-storage-per-channel" should be easy to understand for users.

Possibly there should be a total maximum somewhere too, so the IRCd won't use 8GB of memory for channel logs :D. Would have to think about what it should do in such a case then... ah well.... for later ;)

syzop

2019-06-12 19:43

administrator   ~0020732

Current code committed.

TODO:
* hashing algorithm (but possibly wait until generic algo is committed, for nick and channels too)
* BATCH for history event (and crosscheck with labeled-response, which should result in nested batches)
* Some S2S command to make sure that channel msg events are broadcasted by all servers instead of multicasted like now. Probably not a PROTOCTL but something else. It should enable the broadcasting behavior network-wide, if a single server requests it.
* There should be an option not to store history at all, like on a hub. You would still need the channel mode but not the _mem storage. Maybe a _null storage module?

syzop

2019-07-13 08:33

administrator   ~0020775

as for the broadcast stuff:

commit 0cda60301dac17b4e14616a4bd587315ba681a82
Author: Bram Matthys <syzop@vulnscan.org>
Date: Sat Jul 13 07:59:12 2019 +0200

    Add set::broadcast-channel-messages [auto|always|never]:
    This determines when UnrealIRCd will use broadcast instead of multicast
    for delivering channel messages to servers.
    The default is 'auto' which uses multicast but switches to broadcast
    when channel mode +H is set. This is what people should normally use.
    If you set it to 'never' then +H will not work properly if there are
    servers with 0 users on them.

TODO: document

syzop

2019-07-13 08:34

administrator   ~0020776

only left to do are:
* BATCH for history event (and crosscheck with labeled-response, which should result in nested batches)
* There should be an option not to store history at all, like on a hub. You would still need the channel mode but not the _mem storage. Maybe a _null storage module?

syzop

2019-07-13 15:46

administrator   ~0020778

BATCH is done

Now the null history thingy

syzop

2019-07-13 16:00

administrator   ~0020779

Done as well

syzop

2019-07-13 16:03

administrator   ~0020780

Need to re-check the configuration stuff/rules, and then this can be closed I think. To my knowledge it is working OK, and it is tested as part of the test framework as well.

syzop

2019-08-10 11:30

administrator   ~0020800

Bump!!

syzop

2019-08-15 14:38

administrator   ~0020816

To my knowledge this is all working as expected.

Issue History

Date Modified Username Field Change
2019-05-18 17:56 syzop New Issue
2019-05-18 17:57 syzop Status new => acknowledged
2019-05-18 18:30 syzop Note Added: 0020683
2019-05-19 08:33 syzop Relationship added related to 0005290
2019-06-09 19:13 syzop Note Added: 0020726
2019-06-09 19:14 syzop Note Edited: 0020726 View Revisions
2019-06-09 19:23 syzop Note Edited: 0020726 View Revisions
2019-06-09 19:30 syzop Note Edited: 0020726 View Revisions
2019-06-09 20:30 syzop Note Edited: 0020726 View Revisions
2019-06-12 19:43 syzop Note Added: 0020732
2019-06-12 19:45 syzop Relationship added child of 0005279
2019-07-13 08:33 syzop Note Added: 0020775
2019-07-13 08:34 syzop Note Added: 0020776
2019-07-13 15:46 syzop Note Added: 0020778
2019-07-13 16:00 syzop Note Added: 0020779
2019-07-13 16:03 syzop Note Added: 0020780
2019-08-10 11:30 syzop Note Added: 0020800
2019-08-15 14:38 syzop Assigned To => syzop
2019-08-15 14:38 syzop Status acknowledged => resolved
2019-08-15 14:38 syzop Resolution open => fixed
2019-08-15 14:38 syzop Fixed in Version => 5.0.0-alpha1
2019-08-15 14:38 syzop Note Added: 0020816
2019-08-18 15:17 syzop View Status private => public