[ckan-discuss] Data services on top of CKAN

Pol, Koos koos.pol at logica.com
Mon Sep 5 10:43:12 BST 2011


[Sorry for top-posting. Bound to using Outlook 2007]

The SPARQL endpoints provide a great resource for digging!
Back to your question on how CKAN might provide/host data services:
Suppose there is a image comparison service based upon image finger prints. The fingerprints might be available from some (perhaps unknown) data hub. But if one were to offer such a comparison service, how could this service be published by CKAN? Or suppose there is some sort of specialized search function (meta search on political or geographic data. How could this service be published with CKAN? Or my original example, suppose one offers a drag and drop service for static CSV/KML/xls data to be changed instantly as an open data set?

The idea is to use CKAN not only as a data catalog, but also as a services catalog.
The point is that data is dead until someone makes information from it. Usually one creates the information himself. A journalist for instance makes his own info by digging and sifting through the open government data hubs.

I can imagine that some smart guy wants to make a penny for each time his financial service is being used (payment service!) Similarly, collaborative creativity might get a big boost, by combining services and/or data sets. Thinking about that, perhaps that would be the biggest benefit: multiple creative applications for open data, some open, some paid, some restricted, some graphical, some text based, some...

Thanks,
Koos



-----Original Message-----
From: d.t.read at gmail.com [mailto:d.t.read at gmail.com] On Behalf Of David Read
Sent: vrijdag 2 september 2011 13:49
To: Pol, Koos
Cc: ckan-discuss at lists.okfn.org
Subject: Re: [ckan-discuss] Data services on top of CKAN

On 2 September 2011 11:11, Pol, Koos <koos.pol at logica.com> wrote:
> Hi,
>
>
>
> We are working on a concept for an FOSS data _services_ catalog.  During my
> research I came across CKAN. As it seems, CKAN provides a snug fit for a lot
> of functionality we require.

CKAN is designed to be a catalogue for open data, in the form of both
downloaded files (e.g. CSV) and services like APIs. For example, we
have over 300 SPARQL end-points listed at thedatahub.org.

> So we are investigating the possibility to
> borrow from or tag along on the CKAN platform.

Great to hear it - we welcome contributions to CKAN's Open Source code.

> Some of the additional functionality we seek is:
>
> -          A more dynamic implementation than static data serving

CKAN is a catalog (i.e. a database of metadata) and we have an
extension "ckanext-storage" that stores data (i.e. uploaded files). I
understand we have some code that provides API access by
row/column/cell for spreadsheets (i.e. tabular) data. I'd be
interested to hear how CKAN might provide some sort of service for
dynamic data.

> -          Timeliness of the data. How old is the data? What are the periods
> of updates? Are dynamic updates possible? If this info is present in the
> catalog, an app designer can benefit from that and take measures.

Absolutely - these are all metadata fields that we have added into the
form for numerous installs of CKANs using the form customisation
features. We have left thedatahub.org flexible at the moment, so
people can add these fields if they wish.

> -          What are the parameters for the service? (E.g. you don't want to
> download the entire Amsterdam public transport table, if you only need the
> table time for tram line 5 at the Okura Hotel)

Yes this would be useful metadata to add. I wonder if the basic list
of key-value pairs would be sufficient for this case, or whether a
more complex data structure would be needed.

> -          Designing a smart GUI *provider* client for data services
> providers. For instance, a Coast Guard employee drag-n-drops the high water
> marks on the canvas, and the client makes a data service for it.

I agree, this seems separate to CKAN. I think there may be few
benefits to making it a CKAN extension, since the app can provide CKAN
with any new/changed metadata with a simple CKAN API call.

David

> (This is a
> silly example. But I'm a techie, not a thats-a-good-idea-for-an-app
> generator ;-)
>
>
>
> How would the CKAN community welcome the idea of extending the CKAN platform
> with a services layer? The services layer could take the shape of a separate
> component, leaving CKAN in itself untouched. Perhaps it can be downloaded as
> a plugin.
>
>
>
> I'd very much welcome your feedback.
>
> Thanks,
>
> Koos
>
>
>
> Koos Pol | software architect Technical Software Engineering
> Laan van Kronenburg 2, 1183 AS  Amstelveen | Nederland
> T: +31 (0) 88 5646 064 | M: +31 (0) 6 54 987 082
> koos.pol at logica.com | www.logica.com


Think green - keep it on the screen.

This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.





More information about the ckan-discuss mailing list