[wdmmg-discuss] Amounts paid to each supplier by Government bodies

[Rufus Pollock]

On 22 June 2010 21:16, Michael Friis <friism at gmail.com> wrote:
>> Would you be able to update this CKAN package with links to your bulk
>> scraped data and perhaps some information about it (perhaps a link to
>> a README outlining structure or XML and main fields):
> I've updated the page to let people know I have the date. I'll try and
> build the API as soon as possible.

That's great Michael. I note that, in general, we'd put stuff like
videos as a link in the Notes field rather than linking them from
Resources which are supposed to be be for the data itself :) --
perhaps something we should note in our instructions on the edit page
...

Would it be possible to get download links for the bulk versions of
the dataset and to enter them into the Resources? You could always
upload the data to somewhere like archive.org if you are worried about
size/bandwidth :)

(As an aside: while an API would be great, getting the bulk data out
there would be very useful too and probably a simpler first step ...)

Rufus

>> I also wanted to ask whether (and how) you are keeping the data up to
>> date? Are you regularly re-scraping the entire thing or able to get
>> just the updates ...
> I've just run the scraper at some point each spring getting the entire
> previous year. Making the scraper incremental would not be a big deal
> since the EU assigns each document a sequential id (starting from 1 on
> January 1.).
>
> Michael
>
> --
> http://friism.com
> (+45) 27122799
> Sapere aude
>



-- 
Open Knowledge Foundation
Promoting Open Knowledge in a Digital Age
http://www.okfn.org/ - http://blog.okfn.org/