Rebol.org crash
@Graham Chiu
Just a courtesy heads-up – you’ve had a serious crash-type problem with Rebol.org over (at least) the last couple of months. As the Montreal Trust team appear not to have acted (or to have taken away my ancient-yet-still-viable credentials), I have taken ex-executive action to apply a quick work around.
BACKGROUND TO PROBLEM
Rebol.org caches the results of searches, if they return more than one page worth’s of results. Expired cache entries used to be deleted regularly. But something has killed that background process.
That resulted in there being some 500,000 old cached search results, taking up several gig of server memory. That was well over the disk quota set by the ISP.
Under the version of Linux that Rebol.org runs under, if we try to write a file that exceeds the disk quote you end up with a ZERO BYTE long file, and no error indication. That simply means any file you write is silently corrupted.
VISIBLE PROBLEM
All searches were failing with a HTTP status code of 500. Probably other things going wrong too. Possibly some important index files corrupted (which may explain why the Articles section is broken).
Google (for one) started reporting the site was broken, and it would soon deprioritise it in SERPS.
SHORT-TERM FIX
I’ve implemented a crude search cache flush script:
http://rebol.org/flush-caches.r
Graham -- If you can CRON that to run every few days, the disk quote limits should never be exceeded.
(When Rebol.org is being rampaged by bots, there can be 100,000 search URLs per day, so the disk quota can be exceeded very quickly).
(There is no real downside to flushing the cache more often, other than a small performance hit on any active searches).
MEDIUM-TERM FIX
The Montreal Team - who are still formally the people who have assumed management of the site – should investigate why the cache-flush is broken. Maybe they could fix Articles at the same time.
LONGER-TERM AMELIORATION
Any website that used Rebol’s WRITE should patch WRITE to double check that the data has been correctly written. That should at least catch disk quota problems – though it may not always be possible to repair or recover (not obvious what to do if WRITE fails during a transaction rollback triggered by a WRITE failure – but that’s a fun case for someone to design for).
(Rebol.org already has a load of “verify op sys did the right thing because it often doesn’t in high contention and/or low resource conditions” code – which contributes to its low failure rate despite four years of no maintenance, and severe hammerings by bots. Sadly, we missed adding corruption detection to WRITE – please learn from our mistakes :)
posted by: Sunanda 15-Jun-2017/8:14:36-7:00
Hi Sunanda,
I don't believe there was ever a Trust team set up (at Montreal). It was as I see it a concept to have a body of devs to overcome the dependency of an ever so absent owner of the domain.
"But something has killed that background process." Yes, the outage of the site mainly caused by this dependency and not resetting it properly?
I don't even think the Trust was meant to consist solely of those who were present at the conference.
As far as I know no follow up has been made to establish such a Trust and deciding which community members were volunteering and accepted as capable to do the job.
My take on this is that if you want to be in on the 'Trust' that will be no problem (with me). You have shown with this action that you still care for the good cause.
But you cannot expect people to get in contact with community members that are not hanging around anymore ;-)
posted by: Arnold 15-Jun-2017/8:55:21-7:00
There was a plan for the rebol .net/.org domains proposed by Brian within the 'E Rebolus Unum' talk and for my part I sketched out a couple of ways that the current functionality of rebol.org could work in alignment with that (with particular regard to handling legacy urls). However, as no trust was established, no tasks assigned, no keys handed over, and since then the very idea of 'Unum' has vacated the 'Rebolus' world, I don't believe it's on anyone's immediate agenda.
I'm not saying that I'm personally not interested--I have a very strong interest in seeing the slate of Rebol-related web sites look and work better and am willing as ever to contribute time to this end.
posted by: Chris 15-Jun-2017/12:22:33-7:00
@Arnold
> Yes, the outage of the site mainly caused by this dependency and not resetting it properly?
I have no idea what stopped the background process. But once it had gone, and the disk quota got exceeded, we were lucky not to sustain major damage with vital control files overwritten with zero byte length files.
I am still willing to act in a emergency (such as this one, and the lapsed domain name of a couple of months ago).
But there is no point in my doing any development while there are other aspiring project teams whose published dev plans involve a clean start with a new code base. I'd just be writing short-term stuff that others already plan to throw away. Better not to get in their way.
posted by: Sunanda 16-Jun-2017/3:47:16-7:00
One potential solution is to make the existing content statically available. That's a one-time job which I'd be happy to help with. The most important value of the site at this point is to maintain the archive of publicly generated content. Google can provide search for the content.
posted by: Nick 16-Jun-2017/5:09:11-7:00
@Nick I'd be happy to review this too. I actually have a copy of the codebase but no content and am unable to create a working mirror on my local system. As I said, I'm very much interested in getting this working and indeed providing maintenance as well with a view to publishing the code on a public repository (subject to support of the contributors to the current codebase).
@Sunanda There is no 'they'.
Even if in the future the domain is reapplied per, for example, the aforementioned proposal--something that is still very much in the hands of RT--the site should still be functional in some capacity.
posted by: Chris 16-Jun-2017/13:09:27-7:00
In the spirit of "Cool URIs don't change"...
https://www.w3.org/Provider/Style/URI
Statically serving the links that were previously available, and delegating search/etc. to Google or whoever, and removing any further uploading to the archival linkbase seems fine. It seems a sound strategy which is not incompatible with any future plan.
Obviously nothing happens terribly fast in Rebolverse. Someone once said that its development schedule was "synchronized with the Clock of the Long Now".
http://longnow.org/clock/
But it feels to me like, if people are willing to *allow* for progress, it will happen.
posted by: Fork 17-Jun-2017/13:11:09-7:00
|