View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0005299 | unreal | tests | public | 2019-05-25 08:26 | 2019-11-27 10:44 |
| Reporter | syzop | Assigned To | syzop | ||
| Priority | normal | Severity | feature | Reproducibility | N/A |
| Status | resolved | Resolution | fixed | ||
| Fixed in Version | 5.0.0-rc2 | ||||
| Summary | 0005299: U5: different test framework | ||||
| Description | I hate our current test framework, since the beginning already. The only good thing about it is that we have SOME test framework :D So... switch to something different. The things I dislike most: 1) The auto-synchronization idea is missing, even though I provided this before the 1st test framework was coded :(. This results in lots of sleep() at various places. Causes unnecessary slowdown and random test failures if you miss cases. Very annoying. 2) Ruby... argh ! And... when looking at something new.... have a look at msgid and such extraction / comparison tests. I'm thinking we probably will use some low level approach (more low level sockets) rather than the high level approach of the current test frame network (which uses an IRC library). | ||||
| Tags | No tags attached. | ||||
| 3rd party modules | |||||
|
|
Wishlist: * Auto syncher * Parallel tasks with random nicks and channels (and some flag if it cannot run in parallel, or if that is too difficult then just split everything in two: parallel and non-parallel) * msgid/timestamp/account comparisson across servers (and always expect it in channel events.. etc...) Python would be preferred, PHP would be the lazy option |
|
|
I worked on this yesterday and it's looking nice. Auto synchronization works, across multiple servers even. Parallelization is no problem, all random based nicks and chans etc. Next few days I probably won't do much work due to the nice weather (:D) but basically: 1) Clean up a bit 2) The parallel main thing is currently a bash script, perhaps this should be python too so we can more easily run on Windows 3) msgid/timestamp/account comparisson accross servers -> this can be added later tho Main thing :( : 4) Write all tests to use this new framework |
|
|
The old test framework (ruby, by Heero) has been removed in U5 and the new one has been added (python, by me). The new test framework currently has about 25% of the tests that the old framework had. More will follow soon... TODO: run on Windows, currently uses a bash script with xargs |
|
|
A lot more tests have been added. Some that existed in the previous test framework and some new. Test framework now run on Windows too. Some tests from the previous test framework still need to be migrated. Off the top of my head these are: extended bans and oper commands |
|
|
Still some tests needs to be migrated, indeed. |
|
|
I added another ~25 tests, from scratch actually, forgot that I could look at the old tests :D. Nearly all user commands are now tested. Oper commands still need to be done. Some other areas as well probably, but not much, mostly oper commands. Oh yeah and extbans. |
|
|
I will have a go at including services in the tests, anope and atheme. Will probably run the tests twice then but that's OK. Then at least we can test some basic functionality with services, something that is currently not done at all... and I normally don't use 1 of the 2 services brands myself which doesn't help either.. |
|
|
Lots of tests were added in rc2, including the remaining things mentioned in the last notes. This can be closed, even though it is - of course - an always ongoing effort :) |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2019-05-25 08:26 | syzop | New Issue | |
| 2019-05-25 08:27 | syzop | Relationship added | child of 0005279 |
| 2019-05-25 08:27 | syzop | Assigned To | => syzop |
| 2019-05-25 08:27 | syzop | Status | new => acknowledged |
| 2019-05-30 16:54 | syzop | Note Added: 0020713 | |
| 2019-06-01 07:59 | syzop | Note Added: 0020717 | |
| 2019-06-03 13:41 | syzop | Note Added: 0020719 | |
| 2019-06-24 15:04 | syzop | Note Added: 0020741 | |
| 2019-08-10 11:30 | syzop | Note Added: 0020799 | |
| 2019-08-18 15:17 | syzop | View Status | private => public |
| 2019-11-11 17:47 | syzop | Note Added: 0021071 | |
| 2019-11-11 17:47 | syzop | Note Edited: 0021071 | |
| 2019-11-18 08:55 | syzop | Note Added: 0021075 | |
| 2019-11-18 08:56 | syzop | Note Edited: 0020741 | |
| 2019-11-18 08:57 | syzop | Note Edited: 0020741 | |
| 2019-11-27 10:44 | syzop | Status | acknowledged => resolved |
| 2019-11-27 10:44 | syzop | Resolution | open => fixed |
| 2019-11-27 10:44 | syzop | Fixed in Version | => 5.0.0-rc2 |
| 2019-11-27 10:44 | syzop | Note Added: 0021120 |
