[wdmmg-discuss] [wdmmg-dev] Blog post on WDMMG backend tech

Tue Feb 15 09:21:54 UTC 2011

Hi David,

On Tue, Feb 15, 2011 at 10:01 AM, David Cabo <david.cabo at gmail.com> wrote:
> One question: I assume all the loader scripts need to be modified,
> but is this going to impact the API too?

Indeed, the loader script needs to be changed a bit to accomodate the
change, although the nasty part (i.e. getting the data read in) is of
course the same. From what I can tell, some work has already been done
to ensure that the spain loader is still functional, but it would
certainly be worth another look to see if it can be improved.

While you could just look at one of the loaders [1] to see how this
works now, I'd be happy to help you through IRC (irc.oftc.net, #wdmmg)
or Skype (friedrich.lindenberg). If you can also collect and relevant
questions (e.g. under ), we can make sure to document this better.

As for the API, we've done our best to make sure that it remains
mostly the same, even though some parts have now been moved to an
extension called wdmmg-legacy [3] (which is only needed when you want
to run the flash against the data store).

>  Regarding adding data from other countries, I've parsed the spanish
> budget for the last few years, and each expense programme is broken
> down into quite a lot of categories and sub-categories. (The
> visualization [1] shows the top three levels, but there're 3-4 more
> below.) Just let me know if you're interested in loading this into the
> back-end. I guarantee you there are some tricky bits. :)

I would, in fact, very much like to understand this. Maybe you can
walk me through it? As for depth, this seems almost as bad as the EU
budget - but I'm quite confident we can get some nice tools in place
to deal with the complexity :-) The current data store already allows
for exploring arbitrary depth, even though only one level is explored
at any time at the moment. See [4]

Cheers,

 Friedrich

[1] https://bitbucket.org/okfn/wdmmg-ext/src/1c17aff5fe4a/wdmmgext/load/eu.py
[2] http://wiki.openspending.org/Technical_Format
[3] https://bitbucket.org/okfn/wdmmg-legacy
[4] http://data.wheredoesmymoneygo.org/entity/EU27