[okfn-labs] Github webhooks and automating creating Data APIs for data packages in Github

Wed Mar 19 12:35:08 UTC 2014

Hi All,

I have a User Story as follows:

 As a Data Wrangler (Rufus!) I want to push my tabular data package to
github and have it automatically have a Data API for that data package
created/updated so that I have a nice API I can use in my app (or JS
visualization or ...)

*Context*

I'm currently really liking the work flow of pushing Data Packaged Datasets
to Github (see growing set of "Core" datasets at
https://github.com/datasets plus
personal ones like hard drive
prices<https://github.com/rgrp/hard-drive-prices>).
However, it would be really nice to get a proper Data API. CKAN, and hence
the DataHub <http://datahub.io/>, now has a nice DataStore with a rich Data
API <http://docs.ckan.org/en/latest/maintaining/datastore.html> which we
could use. There is even a nice standalone webapp for automatically pulling
data into the DataStore - https://github.com/ckan/datapusher

(Note: this would also address the Data API issue in the Frictionless Data
roadmap <http://data.okfn.org/roadmap>)

*Plan*

Rather than create a new Data API webapp I thought the best thing would be
create a small webapp that a github webhook can connect to. The app will
take care of then pushing the updated data to the DataStore.

*Question*

*A. *Does this sound sensible

*B. *Do folks have any thoughs and experience on build github webhook apps
(especially in nodejs or python)

All the best,

Rufus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/okfn-labs/attachments/20140319/9fa56ffe/attachment-0003.html>