[wdmmg-dev] Free the pickles! - How to re-generate test data

Wed Oct 5 12:18:08 UTC 2011

Hi Nick,

thanks for the help.

On Wed, Oct 5, 2011 at 1:47 PM, Nick Stenning <nick.stenning at okfn.org> wrote:
> See, for instance commit:
>
> https://github.com/okfn/openspending/commit/f8c70b7bb7493d87d013ef90b63a7f1a8fa38fe2
>
> The data comes from these files:
>
> https://bitbucket.org/okfn/wdmmg/src/tip/wdmmg/tests/fixtures/cra
>
> Your confusion comes, I think, from the fact that each line in the
> original CSV creates multiple entries -- in your custom loader.

Yes and it seems non-trivial to convert one into the other: if I
transpose the sample fixture to put each year into its own row, I get
69 rows. I'm not sure if I will try to manually find which of the rows
are supposed to be part of this set.

> If you want to recreate a useful CSV, I suggest you do:
>
>  ostool development.ini db loadexample cra
>
> And export via the API.

Then I need to either create a program to convert this into its source
CSV form or write a new model handling the CSV generated by
OpenSpending, right? Might be the easiest route at this stage.

> As for the *current* method of loading CRA, and an OS model, see:
>
>  https://github.com/okfn/dpkg-cra

This is beautifully done, especially the documentation on COFOG fixes!

- Friedrich

> P.S. CCing the list for future reference.
>
> On Wed, Oct 5, 2011 at 12:21, Nick Stenning <nick.stenning at okfn.org> wrote:
>> In lectures, will do asap.
>>
>> On Wed, Oct 5, 2011 at 10:05, Friedrich Lindenberg
>> <friedrich.lindenberg at okfn.org> wrote:
>>> Nick,
>>>
>>> if you have a moment, could you send me some info on this? I'm
>>> effectively blocked atm.
>>>
>>> Thanks,
>>>
>>>  Friedrich
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Friedrich Lindenberg <friedrich.lindenberg at okfn.org>
>>> Date: Tue, Oct 4, 2011 at 2:49 PM
>>> Subject: Free the pickles! - How to re-generate test data
>>> To: wdmmg-dev at lists.okfn.org, Nick Stenning <nick at whiteink.com>
>>>
>>>
>>> Hi Nick,
>>>
>>> I'm having a hard time re-generating the pickled database images:
>>>
>>> 1) All the CRA sample sheets I can find have 12 rows, the pickled copy
>>> has 36. What is the source data for this?
>>> 2) Is there a model for CRA? Where?
>>>
>>> Thanks,
>>>
>>> - Friedrich
>>>
>>> --
>>> Open Knowledge Foundation
>>> Promoting Open Knowledge in a Digital Age
>>> http://www.okfn.org/ - http://blog.okfn.org/
>>>
>>> http://twitter.com/pudo
>>> http://pudo.org
>>>
>>>
>>>
>>> --
>>> Open Knowledge Foundation
>>> Promoting Open Knowledge in a Digital Age
>>> http://www.okfn.org/ - http://blog.okfn.org/
>>>
>>> http://twitter.com/pudo
>>> http://pudo.org
>>>
>>
>

-- 
Open Knowledge Foundation
Promoting Open Knowledge in a Digital Age
http://www.okfn.org/ - http://blog.okfn.org/

http://twitter.com/pudo
http://pudo.org