[ckan-dev] [sysadmin-coord] Reduce number of files in VM

Wed Jun 27 11:25:58 UTC 2012

On 2012-06-26 18:07, James Casbon wrote:
> On 26 June 2012 16:54, Nils Toedtmann <nils.toedtmann at okfn.org> wrote:
>> Hi *
>>
>> one of the causes of the increased load we put on the Fry hosting
>> infrastructure is that some of our VMs have way too many (small) files
>> in them. [*] In particular CKAN session dirs collect >10 million files
>> because CKAN doesn't clean up after itself.
> 
> Thanks Nils,
> 
> CKAN devs:  how much effort would it be to get CKAN to run from a
> read-only filesystem?

I'm not going to comment on how difficult this would be, not knowing
CKAN intimately. (Although I will hazard a guess at "not too hard" --
you'd need to swap out the Beaker filesystem session storage with a
Memcached store).

I will however note that the Beaker documentation clearly states that
Beaker doesn't clean up old session files on any backend[1], so I would
strongly suggest that the CKAN team look into scheduling a job *as part
of CKAN* (not a separate step in the INSTALL documentation which gets
routinely overlooked) which cleans up old session files.

[1]:
http://beaker.readthedocs.org/en/latest/sessions.html#removing-expired-old-sessions

Best wishes,
Nick

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20120627/4c328995/attachment.sig>