View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0001752 | unreal | ircd | public | 2004-04-21 01:49 | 2007-04-27 05:47 |
| Reporter | Ron2K | Assigned To | |||
| Priority | normal | Severity | feature | Reproducibility | N/A |
| Status | closed | Resolution | wont fix | ||
| Product Version | 3.2-RC2 | ||||
| Summary | 0001752: caching of remotely included files | ||||
| Description | After submitting bug 0001751, I came up with the following feature. Get the IRCd to cache all remotely included files (save them to a subdirectory /cache/ or something like that). Now, when rehashing or when restarting the IRCd, it should check the remote server first. If it cannot connect to the remote server for whatever reason, it should use the version of the file that's in the cache. If it connects to the remote server, it should compare the file on the remote server to the file in the cache and update the cache if necessary. Optionally, a configuration directive could also be included that specifies how long the IRCd should try connecting to the remote server before giving up and connecting to the cache. It could be called something like set::remote-timeout and could default to, say, 60 seconds if not specified (which could eliminate problems if this directive was somehow placed in a remotely included file). The directive would be ignored if the server is compiled without remote includes. Knowing that Unreal3.2 is now feature frozen - how about seeing this in 3.3? | ||||
| 3rd party modules | |||||
|
|
First let me say, Unreal does indeed already store the files cached. It works similar to a browser. It will ask the server "has the file changed?" If not, then it just loads the version in the cache, rather than redownloading it. As for the use cache if failed, that's already on my TODO list. However I'm planning to do it differently than you suggest. It will not use the cached version at startup. Because that could be weeks old. And it will not even use the cached by default, the user will say what to do. include "http://www.blah.com/something.conf" { on-failure use-cache; }; In that case, it would generate a warning saying it failed to d/l, and would use the cached version. If you don't have that, it would work like it currently does. As for the timeouts, yeah having the connect timeout configurable is a good idea. I'd probably make it like set::remote-includes::connect-timeout so that we can easily group all remote include options together. |
|
|
How about? include "http://www.blah.com/something.conf" { on-failure use-cache; cache-expire 2d; // File in cache is "invalid" after 2 days. }; (Or a master option like set::remote-includes::cache-expire?) (Another possibility is allowing the HTTP server to set expiration times through the HTTP headers, but how you would pull that from libcurl is beyond me ;) ). |
|
|
At which point it would generate an error again? Yeah, HTTP does have some kinda expiration header. I'll look at that. libcurl does allow us to retrieve headers, it's a pain, but it can be done... |
|
|
If your HTTP server is down for longer than 2 days, then maybe it's about time to find a new hosting company ;) . |
|
|
aquanight, self hosting... it is done... hell, ive run a temporary webserver off my dialup for when ive needed files elsewhere. so then you get hardware failure, and need to get a new component "it wont arrive for a week" <-- there you go. >2 days. |
|
|
In which case you would temporarily increase the cache time, or use an alternate server. |
|
|
Yeah, I'm not 100% sure about the "after N days, start generating an error" There should at least be a way to override it I think... |
|
|
Hmm... After thinking about it a bit more, I figured that maybe the cache timelimit would only be useful if you did remote includes 100% the "web browser" way: use the cache file until it expires (which could possibly be forced using /rehash) and only when it expires do you wip out libcurl. Looking at your first post, this comes close the way it is now. (The only difference being the explicit check for changes, instead of just expiring the cache and downloading a new one.) Of course this causes problems if a half-finished "new configuration" is posted (when using an editor, you _do_ save often, don't you ;) ?) and then the server curls it and now the config won't load, but then you just /rehash -expire (or something) when the remote conf is finished ;) . |
|
|
Please resubmit this bug if it persists on 3.2.6-7/3.3 |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2004-04-21 01:49 | Ron2K | New Issue | |
| 2004-05-04 13:38 |
|
Note Added: 0006043 | |
| 2004-05-04 17:48 | aquanight | Note Added: 0006060 | |
| 2004-05-04 18:02 |
|
Note Added: 0006062 | |
| 2004-05-04 18:18 | aquanight | Note Added: 0006065 | |
| 2004-05-04 18:46 | w00t | Note Added: 0006070 | |
| 2004-05-04 18:51 | aquanight | Note Added: 0006072 | |
| 2004-05-04 21:10 |
|
Note Added: 0006081 | |
| 2004-05-04 21:18 | aquanight | Note Added: 0006083 | |
| 2007-04-27 05:47 |
|
Status | new => closed |
| 2007-04-27 05:47 |
|
Note Added: 0013841 | |
| 2007-04-27 05:47 |
|
Resolution | open => wont fix |
