[okfn-labs] csv/xls files for ONS/weather/etc data

[Andy Twigg]

Hi Rufus,

Thanks. The API calls I discussed with ckanclient are still failing -
so maybe that is because the ckanclient is indeed deprecated?
(although that isn't clear from the documentation...) I think I will
just use python-requests to post to the URLs you mention.

A couple of questions:
1. Is datahub.io intented to be superset of data.gov.uk ? What is the
goal of datahub.io?
2. I'm working on a large-scale machine learning project that involves
using lots of public data, eg ONS/police/weather/crime/events. I would
love to get as much public data as possible, reasonably well-formatted
and cleaned, ideally CSV/XLS. Would this be something that you guys
would aim to provide in the form of dahub.io/data.gov.uk? For example,
I notice at the moment that weather, police, and some ONS data in
data.gov.uk are simply links to 'DIY' pages on the respective websites
(e.g. one has to sign up separately for the metoffice service). Is
pulling all this data into one place a goal of the OKFN?
3. (possibly a naive question) What is the difference between OData
producers such as Azure Marketplace/infochimps and the CKAN stores of
data.gov.uk/datahub.io ? Will data.gov.uk/datahub.io become an OData
producer?

Cheers,

Andy


On 18 February 2013 09:42, Rufus Pollock <rufus.pollock at okfn.org> wrote:
> Hi Andy,
>
> First off, a warm welcome!
>
> As Stefan pointed out the API does seem to be working, did you have
> better luck after you sent your mail? I note that this query can get
> you all the CSVs in one go (though not by tag obviously):
>
> http://data.gov.uk/api/search/dataset?res_format=CSV
>
> Here's all the XLS: http://data.gov.uk/api/search/dataset?res_format=XLS
>
> If you want to get all the data as part of the search results just add
> all_fields=1 e.g.
>
> http://data.gov.uk/api/search/dataset?res_format=CSV&all_fields=1
>
> Rufus



--
Dr Andy Twigg
Junior Research Fellow, St Johns College, Oxford
Room 351, Department of Computer Science
http://www.cs.ox.ac.uk/people/andy.twigg/
andy.twigg at cs.ox.ac.uk | +447799647538