[okfn-labs] Github webhooks and automating creating Data APIs for data packages in Github

Thu Apr 10 05:37:06 UTC 2014

I think the idea is very good, but would not be better to just create a
view method in CKAN or Datapusher to answer the Github Webhook?

After this will be a good idea to create a service for CKAN/Datapusher in
the repository https://github.com/github/github-services, but this is not
obligatory, is just good to be an official webhook service and be
integrated with the Github Webhook configuration page if you want to add
parameters like API keys, URL of your CKAN backend, etc.

An example of how to use Github Webhook API: you can create manually a
webhook in one of your repositories settings at "Webhooks & Services" (or
using the REST API) and configure "Payload URL" to an address where your
webapp is running (e.g. http://ckan.org/datapusher) and the kind of event
('push' is the default, but there is others
https://developer.github.com/v3/activity/events/) that will trigger a
message from Github to your webapp.

In your webapp something simple as http://pastebin.com/raw.php?i=wrdMCnr1
will be enough to receive a Github json message  with a lot of information
about the push. This can trigger a push from the Github repository to a
CKAN instance using DataStore.

E.

> Hi All,
>
> I have a User Story as follows:
>
>  As a Data Wrangler (Rufus!) I want to push my tabular data package to
> github and have it automatically have a Data API for that data package
> created/updated so that I have a nice API I can use in my app (or JS
> visualization or ...)
>
> *Context*
>
> I'm currently really liking the work flow of pushing Data Packaged Datasets
> to Github (see growing set of "Core" datasets athttps://github.com/datasets plus
> personal ones like hard drive
> prices<https://github.com/rgrp/hard-drive-prices>).
> However, it would be really nice to get a proper Data API. CKAN, and hence
> the DataHub <http://datahub.io/>, now has a nice DataStore with a rich Data
> API <http://docs.ckan.org/en/latest/maintaining/datastore.html> which we
> could use. There is even a nice standalone webapp for automatically pulling
> data into the DataStore - https://github.com/ckan/datapusher
>
> (Note: this would also address the Data API issue in the Frictionless Data
> roadmap <http://data.okfn.org/roadmap>)
>
> *Plan*
>
> Rather than create a new Data API webapp I thought the best thing would be
> create a small webapp that a github webhook can connect to. The app will
> take care of then pushing the updated data to the DataStore.
>
> *Question*
>
> *A. *Does this sound sensible
>
> *B. *Do folks have any thoughs and experience on build github webhook apps
> (especially in nodejs or python)
>
> All the best,
> Rufus
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/okfn-labs/attachments/20140410/c935eff1/attachment-0003.html>