[ckan-discuss] Data services on top of CKAN

Mon Sep 5 11:54:29 BST 2011

On 5 September 2011 10:43, Pol, Koos <koos.pol at logica.com> wrote:
> [Sorry for top-posting. Bound to using Outlook 2007]
>
> The SPARQL endpoints provide a great resource for digging!
> Back to your question on how CKAN might provide/host data services:
> Suppose there is a image comparison service based upon image finger prints. The fingerprints might be available from some (perhaps unknown) data hub. But if one were to offer such a comparison service, how could this service be published by CKAN?

Koos, these sound like nice apps, but I'm struggling to see the
benefits of combining them with the catalogue.

For your example fingerprint service: it has a URL, which I'd expect
would be listed in CKAN's catalogue. A user might search in CKAN for
"fingerprints" and find both the site with the data and the site with
the service based on the data. Both seem useful, and I can't see the
problem with the user then say clicking on the comparison service link
and doing a comparison from the site. I'm sure you've got some point
here which I'm totally missing...

> Or suppose there is some sort of specialized search function (meta search on political or geographic data. How could this service be published with CKAN?

Ok, here's a concrete example of geographic data that has been
provided as services - Google Maps overlay and an API, the UK police
maps. I've put it into CKAN: http://ckan.net/package/police-uk What
more might CKAN do?

David

> Or my original example, suppose one offers a drag and drop service for static CSV/KML/xls data to be changed instantly as an open data set?
>
> The idea is to use CKAN not only as a data catalog, but also as a services catalog.
> The point is that data is dead until someone makes information from it.
> Usually one creates the information himself. A journalist for instance makes his own info by digging and sifting through the open government data hubs.
> I can imagine that some smart guy wants to make a penny for each time his financial service is being used (payment service!) Similarly, collaborative creativity might get a big boost, by combining services and/or data sets. Thinking about that, perhaps that would be the biggest benefit: multiple creative applications for open data, some open, some paid, some restricted, some graphical, some text based, some...
>
> Thanks,
> Koos
>
>
>
> -----Original Message-----
> From: d.t.read at gmail.com [mailto:d.t.read at gmail.com] On Behalf Of David Read
> Sent: vrijdag 2 september 2011 13:49
> To: Pol, Koos
> Cc: ckan-discuss at lists.okfn.org
> Subject: Re: [ckan-discuss] Data services on top of CKAN
>
> On 2 September 2011 11:11, Pol, Koos <koos.pol at logica.com> wrote:
>> Hi,
>>
>>
>>
>> We are working on a concept for an FOSS data _services_ catalog.  During my
>> research I came across CKAN. As it seems, CKAN provides a snug fit for a lot
>> of functionality we require.
>
> CKAN is designed to be a catalogue for open data, in the form of both
> downloaded files (e.g. CSV) and services like APIs. For example, we
> have over 300 SPARQL end-points listed at thedatahub.org.
>
>> So we are investigating the possibility to
>> borrow from or tag along on the CKAN platform.
>
> Great to hear it - we welcome contributions to CKAN's Open Source code.
>
>> Some of the additional functionality we seek is:
>>
>> -          A more dynamic implementation than static data serving
>
> CKAN is a catalog (i.e. a database of metadata) and we have an
> extension "ckanext-storage" that stores data (i.e. uploaded files). I
> understand we have some code that provides API access by
> row/column/cell for spreadsheets (i.e. tabular) data. I'd be
> interested to hear how CKAN might provide some sort of service for
> dynamic data.
>
>> -          Timeliness of the data. How old is the data? What are the periods
>> of updates? Are dynamic updates possible? If this info is present in the
>> catalog, an app designer can benefit from that and take measures.
>
> Absolutely - these are all metadata fields that we have added into the
> form for numerous installs of CKANs using the form customisation
> features. We have left thedatahub.org flexible at the moment, so
> people can add these fields if they wish.
>
>> -          What are the parameters for the service? (E.g. you don't want to
>> download the entire Amsterdam public transport table, if you only need the
>> table time for tram line 5 at the Okura Hotel)
>
> Yes this would be useful metadata to add. I wonder if the basic list
> of key-value pairs would be sufficient for this case, or whether a
> more complex data structure would be needed.
>
>> -          Designing a smart GUI *provider* client for data services
>> providers. For instance, a Coast Guard employee drag-n-drops the high water
>> marks on the canvas, and the client makes a data service for it.
>
> I agree, this seems separate to CKAN. I think there may be few
> benefits to making it a CKAN extension, since the app can provide CKAN
> with any new/changed metadata with a simple CKAN API call.
>
> David
>
>> (This is a
>> silly example. But I'm a techie, not a thats-a-good-idea-for-an-app
>> generator ;-)
>>
>>
>>
>> How would the CKAN community welcome the idea of extending the CKAN platform
>> with a services layer? The services layer could take the shape of a separate
>> component, leaving CKAN in itself untouched. Perhaps it can be downloaded as
>> a plugin.
>>
>>
>>
>> I'd very much welcome your feedback.
>>
>> Thanks,
>>
>> Koos
>>
>>
>>
>> Koos Pol | software architect Technical Software Engineering
>> Laan van Kronenburg 2, 1183 AS  Amstelveen | Nederland
>> T: +31 (0) 88 5646 064 | M: +31 (0) 6 54 987 082
>> koos.pol at logica.com | www.logica.com
>
>
> Think green - keep it on the screen.
>
> This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
>
>
>
> _______________________________________________
> ckan-discuss mailing list
> ckan-discuss at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-discuss
>