[okfn-help] Last on Caching for this evening...

Sun Sep 5 04:52:53 BST 2010

 eu3 keeps falling over. the problems are (were)
compounded by running the cache on the same
host!

So I've made a new entry in the DNS, cache.ckan.net
which points to eu6.

eu6 is running squid -- this by no means precludes
using varnish in the future, but the varnish config
is quite fiddly - a typeo in a regular expression took
down all of the other ckans earlier.

the main ckan site and api are running on eu7. the
others are still running on eu3. the frei universitat
is crawling the site every half hour or so (maybe it
is 15 minutes, we'll see)

the package api controllers have been factored out
and do some caching. see other mail.

the front page and the heavy html pages use the
simple proxy_cache decorator which just sets the
headers.

so far so good. i can crawl the api in 1 minute 20
seconds when the cache is full.

Despite the fact that all seems well at the moment,
I think we should spin up a new host and migrate
the sites currently on eu3 to it, or perhaps to both
it and eu7 and load balance. In either case, ditch
eu3 just for prudence's sake

Hopefully there will be no further problems until
the lod folks get through drawing their cloud
pictures...

-w

-- 
William Waites           <william.waites at okfn.org>
Mob: +44 789 798 9965    Open Knowledge Foundation
Fax: +44 131 464 4948                Edinburgh, UK

RDF Indexing, Clustering and Inferencing in Python
		http://ordf.org/