[wdmmg-dev] Modelling data for OpenSpending

Wed Aug 17 13:59:24 UTC 2011

Hello,

Over the last several months we've had lots of different stories about how
to get data into OpenSpending; pretty much all the new ideas have involved a
CSV file of data and some sort of representation of the model inside the
database. We've successfully abolished the non-CSV means of loading data in
the future, which leaves us with the problem of how to map CSV columns to
database dimensions. All the solutions to that have involved a JSON
representation of the model, now documented Ok-ishly on the wiki.

How do we generate this JSON blob? So far, we've tried:

1) a web UI for modelling, saved into the DB
2) a Google-docs CSV mapping format
3) handcoding the thing
4) heuristically generating it based on analysing the CSV
5) a new web UI for generating it

The proposed new web UI (by Nick Stenning) is at:
http://nickstenning.github.com/openspending.modeleditor/

Ideally, it would be useful to be able to:

* edit the model iteratively (rather than generate-once)
* share models between datasets (many datasets will have identical models,
though maybe they shouldn't be separate datasets)
* have UI for generating views etc, such that datawranglers get good
visualisations out of the box

Any thoughts, comments?

Mk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/openspending-dev/attachments/20110817/8bff10b8/attachment.html>