[ckan-discuss] Data services on top of CKAN

[David Read]

On 2 September 2011 11:11, Pol, Koos <koos.pol at logica.com> wrote:
> Hi,
>
>
>
> We are working on a concept for an FOSS data _services_ catalog.  During my
> research I came across CKAN. As it seems, CKAN provides a snug fit for a lot
> of functionality we require.

CKAN is designed to be a catalogue for open data, in the form of both
downloaded files (e.g. CSV) and services like APIs. For example, we
have over 300 SPARQL end-points listed at thedatahub.org.

> So we are investigating the possibility to
> borrow from or tag along on the CKAN platform.

Great to hear it - we welcome contributions to CKAN's Open Source code.

> Some of the additional functionality we seek is:
>
> -          A more dynamic implementation than static data serving

CKAN is a catalog (i.e. a database of metadata) and we have an
extension "ckanext-storage" that stores data (i.e. uploaded files). I
understand we have some code that provides API access by
row/column/cell for spreadsheets (i.e. tabular) data. I'd be
interested to hear how CKAN might provide some sort of service for
dynamic data.

> -          Timeliness of the data. How old is the data? What are the periods
> of updates? Are dynamic updates possible? If this info is present in the
> catalog, an app designer can benefit from that and take measures.

Absolutely - these are all metadata fields that we have added into the
form for numerous installs of CKANs using the form customisation
features. We have left thedatahub.org flexible at the moment, so
people can add these fields if they wish.

> -          What are the parameters for the service? (E.g. you don’t want to
> download the entire Amsterdam public transport table, if you only need the
> table time for tram line 5 at the Okura Hotel)

Yes this would be useful metadata to add. I wonder if the basic list
of key-value pairs would be sufficient for this case, or whether a
more complex data structure would be needed.

> -          Designing a smart GUI *provider* client for data services
> providers. For instance, a Coast Guard employee drag-n-drops the high water
> marks on the canvas, and the client makes a data service for it.

I agree, this seems separate to CKAN. I think there may be few
benefits to making it a CKAN extension, since the app can provide CKAN
with any new/changed metadata with a simple CKAN API call.

David

> (This is a
> silly example. But I’m a techie, not a thats-a-good-idea-for-an-app
> generator ;-)
>
>
>
> How would the CKAN community welcome the idea of extending the CKAN platform
> with a services layer? The services layer could take the shape of a separate
> component, leaving CKAN in itself untouched. Perhaps it can be downloaded as
> a plugin.
>
>
>
> I’d very much welcome your feedback.
>
> Thanks,
>
> Koos
>
>
>
> Koos Pol | software architect Technical Software Engineering
> Laan van Kronenburg 2, 1183 AS  Amstelveen | Nederland
> T: +31 (0) 88 5646 064 | M: +31 (0) 6 54 987 082
> koos.pol at logica.com | www.logica.com
>
>
>
> Think green - keep it on the screen. This e-mail and any attachment is for
> authorised use by the intended recipient(s) only. It may contain proprietary
> material, confidential information and/or be subject to legal privilege. It
> should not be copied, disclosed to, retained or used by, any other party. If
> you are not an intended recipient then please promptly delete this e-mail
> and any attachment and all copies and inform the sender. Thank you.
> _______________________________________________
> ckan-discuss mailing list
> ckan-discuss at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-discuss
>
>