View Issue Details

IDProjectCategoryView StatusLast Update
0005299unrealtestspublic2019-08-18 15:17
ReportersyzopAssigned Tosyzop 
PrioritynormalSeverityfeatureReproducibilityN/A
Status acknowledgedResolutionopen 
Product Version 
Target VersionFixed in Version 
Summary0005299: U5: different test framework
DescriptionI hate our current test framework, since the beginning already. The only good thing about it is that we have SOME test framework :D

So... switch to something different.
The things I dislike most:
1) The auto-synchronization idea is missing, even though I provided this before the 1st test framework was coded :(. This results in lots of sleep() at various places. Causes unnecessary slowdown and random test failures if you miss cases. Very annoying.
2) Ruby... argh !

And... when looking at something new.... have a look at msgid and such extraction / comparison tests.

I'm thinking we probably will use some low level approach (more low level sockets) rather than the high level approach of the current test frame network (which uses an IRC library).
TagsNo tags attached.
3rd party modules

Relationships

child of 0005279 acknowledged UnrealIRCd 5 master tracking issue 

Activities

syzop

2019-05-30 16:54

administrator   ~0020713

Wishlist:
* Auto syncher
* Parallel tasks with random nicks and channels (and some flag if it cannot run in parallel, or if that is too difficult then just split everything in two: parallel and non-parallel)
* msgid/timestamp/account comparisson across servers (and always expect it in channel events.. etc...)

Python would be preferred, PHP would be the lazy option

syzop

2019-06-01 07:59

administrator   ~0020717

I worked on this yesterday and it's looking nice. Auto synchronization works, across multiple servers even. Parallelization is no problem, all random based nicks and chans etc.
Next few days I probably won't do much work due to the nice weather (:D) but basically:
1) Clean up a bit
2) The parallel main thing is currently a bash script, perhaps this should be python too so we can more easily run on Windows
3) msgid/timestamp/account comparisson accross servers -> this can be added later tho
Main thing :( :
4) Write all tests to use this new framework

syzop

2019-06-03 13:41

administrator   ~0020719

The old test framework (ruby, by Heero) has been removed in U5 and the new one has been added (python, by me).
The new test framework currently has about 25% of the tests that the old framework had. More will follow soon...
TODO: run on Windows, currently uses a bash script with xargs

syzop

2019-06-24 15:04

administrator   ~0020741

A lot more tests have been added. Some that existed in the previous test framework and some new.

Some tests from the previous test framework still need to be migrated. Off the topic of my head these are: extended bans and oper commands

syzop

2019-08-10 11:30

administrator   ~0020799

Still some tests needs to be migrated, indeed.

Issue History

Date Modified Username Field Change
2019-05-25 08:26 syzop New Issue
2019-05-25 08:27 syzop Relationship added child of 0005279
2019-05-25 08:27 syzop Assigned To => syzop
2019-05-25 08:27 syzop Status new => acknowledged
2019-05-30 16:54 syzop Note Added: 0020713
2019-06-01 07:59 syzop Note Added: 0020717
2019-06-03 13:41 syzop Note Added: 0020719
2019-06-24 15:04 syzop Note Added: 0020741
2019-08-10 11:30 syzop Note Added: 0020799
2019-08-18 15:17 syzop View Status private => public