[ckan-dev] Improve static files management (getting rid of fanstatic)

Tue Nov 12 10:32:24 UTC 2013

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all,
we're about to deploy in production a new installation of Ckan, so I
was looking at a proper way to serve the static files in "production
mode".

Sadly enough, it looks like it's impossible to do it properly (and for
"properly" I mean something like Django's staticfiles application: all
the static files get collected in a directory that can be served
directly by the webserver).

Even worse, URLs generated by fanstatic are way too long and contain
some "bad" characters, (eg. colons) which our sysadmins reported
causing troubles with caching proxies.

So I dug a little more into how Fanstatic works, and I found an
incredible amount of wrongness...

* The principle that "serving static files from the wsgi app is better
for larger applications" is completely wrong. While it might be OK for
very small applications, the moment you grow a bit, the useless waste
of resources will bite you.

* The cache management is just stupid: why the hell reinventing the
wheel (infinite cache timeout + urls with datestamps) when you have
proper ways to do this (eg. etags) and, even better, your web server
knows how to handle this?

* While they supposedly should, javascript and css files **don't** get
aggregated and minified. So this thing has no purpose at all.

I was talking with some other guys on IRC, and they're experiencing
the same issues, so I decided to open an issue and start working out a
way to get rid of fanstatic, before preparing a pull request.

https://github.com/okfn/ckan/issues/1311

I'd like to have some feedback on the topic before even starting to
write code:

* What do you think about this? Any valid reason to keep fanstatic?
* Do you know a valid replacement for this? (At worse, I guess we
could write something minimal that replicates Django behavior, while
keeping compatibility with fanstatic tags to prevent breaking things)
* Other ideas / proposals / use cases / ..?

Cheers

- -- 
Samuele Santi

Trento Rise
http://www.trentorise.eu/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJSggO3AAoJEGgM+6dff2m7ygkP/iTDx20a2IqLd3gQCNoSkY9J
jh7eYt4sMKhRvfDhdZlkd6Ev21Khbv5XMu9VBn6ZAXNqPCwjYANBm4DRL4eYuOhA
sT89pyEh9xCYRhkBqr7PbR0vUl4ASVXb9BbQGpssj2nLRT0nfteYTOVqteVwcWb7
uNG6ztThXLEJHD7arMHo/rXKfUfmGhvIO5V5q+TXKOHmPj8R/mqi9j3HAQUx4EwC
J6D3eZODv/d5YlQ+NsX/xGIoWHForzLitkZnbYZu4UJN1mg+MED57tIRGQ2miNfK
05h7NhVSDCpZTAvbUzrvVLz9Z4MCgE7Hfz/iw1tWfRkW9G3c39DqXirvozXGMNBr
jXGxeq0w3Ml7c1OPgPrbfgoEbw9OijsEX2IXtH2vehuI3qjHY2bOobPKB8dc3ui7
St2Jj+paiNuOf/h0n6IdRq64M9SxpeqzfH9tBiSHs504YzL88fOPqxuZXav1oJm0
DAudygcezbBNvjJF1cLWQSXWP5j8ype9bI6Ss6U26GLBMD3f3cV0DT97zSQj+rTL
Z05hbypuNtiE0JVsblPN50ToW8HYP1Wx5o+nTa8WcfmOWurZhdbsohs7y3BXpvaB
j4z+GutAnQ9QVFJ2tQKmgz2dhi/hJwq+9O1WTqxXFy5kwVruEvFCRv47fiP6toQC
LkJx0FQp10G3jl161vum
=8eoD
-----END PGP SIGNATURE-----