[openspending-dev] A little hack around OSEP 2.

Friedrich Lindenberg friedrich at pudo.org
Thu Dec 25 20:07:42 UTC 2014


Hey people,

I've been thinking a bit about the question of the split-up of the ETL
process into various pieces, and having each stage stored to S3 for future
generations (will they thank us?). To get a sense of what this could look
like, I decided to hack up a straw man implementation of a storage-based
ETL pipeline today. Would be grateful for any comments and input!

* https://github.com/pudo/loadkit

Example:

* https://github.com/pudo/loadkit/blob/master/demo.py

Cheers,

- Friedrich
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/openspending-dev/attachments/20141225/44be0163/attachment.html>


More information about the openspending-dev mailing list