[ckan-dev] CKAN and RDF/SPARQL

Wed May 25 09:43:03 UTC 2011

On 25 May 2011 09:51, William Waites <ww at styx.org> wrote:
> Hallo Ed,
>
> CKAN does not directly support RDF as it uses a relational database to
> store the metadata.
>
> There are two strategies you might use to accomplish what you
> want. You could work up a mapping with D2R. The advantage of this is
> it would make the data directly queriable with SPARQL, the
> disadvantage is that the mapping could get quite complicated because
> the "extra" fields that are used are just stored as string key-value
> pairs, in some cases with the value being a JSON string.

Will, do say if I've not covered all the bases, but I don't think the
way CKAN stores its data internally is an issue for dealing in RDF to
the outside world. We can happily store any RDF properties of a
package in the extra fields, using JSON structures if necessary.

I anticipate Ed wanting customised html for viewing and editing a
package anyway, so that can deal with the extras being stored as lists
etc.. The only extension necessary would be for adding the RDF package
in/out to the API (and D2R sounds useful for this, as well as
providing an SPARQL endpoint). And SOLR's schema would need updating
to cope with these non-textual extra fields.

> The other way is to have the RDF cache done via the queue. A dataset
> gets saved, its JSON representation is sent to the queue and is
> processed by some worker processes. This is how the Solr index gets
> made, for example, and you would just consider the triplestore as
> another kind of index. The advantage is that creating the RDF is
> easier, but the disadvantage is that there is still a delay, albeit
> much shorter, before the RDF would be available.

I'm thinking that this would have the benefit of providing all of the
features of navigating the linked data that Will's provided.

One potential issue is that this is a read-only service and so writing
packages would still be via CKAN's JSON API. But perhaps D2R will be
useful here too.

David

>
> Viele gruesse,
> -w
>
> * [2011-05-24 14:30:54 -1000] Edzard Höfig <edzard.hoefig at fokus.fraunhofer.de> écrit:
>
> ] Hi List,
> ]
> ] we are currently planing to employ CKAN for an open data platform for the city of Berlin, as well as for other cities in the context of the European "Open Cities" project. One of the technical issues that we are still unclear about concerns CKAN's support for RDF and SPARQL. More concretely, we wonder if a single instance of the CKAN software can provide the opportunity to access the meta data of datasets (source, author, etc.) in the RDF format; and furthermore, if it could be configured to provide a SPARQL endpoint to do this.
> ]
> ] We know about the "semantic.ckan.net" website and the CKANRDF extension, but these seem to involve intermediary steps (e.g. polling the data at regular intervals and caching it at an independent location). We would like to be able to directly query a CKAN instance and get results back that are based on the current information stored in that instance (being able to query the "live" data, so to speak).
> ]
> ] Is anyone else interested in such a system ability?
> ] Do you know of any people that are already working on such a feature?
> ]
> ] Cheers,
> ] Ed
> ] --------------------------------------------------------------------
> ] Dr.-Ing. Edzard Höfig
> ] Senior Research Engineer
> ] -------
> ] Fraunhofer FOKUS
> ] Kaiserin-Augusta-Allee 31
> ] D-10589 Berlin
> ] phone: ++49-30-3463-7243 (fax: -8243)
> ] web: www.fokus.fraunhofer.de/go/motion
> ]
> ]
> ] _______________________________________________
> ] ckan-dev mailing list
> ] ckan-dev at lists.okfn.org
> ] http://lists.okfn.org/mailman/listinfo/ckan-dev
>
> --
> William Waites                <mailto:ww at styx.org>
> http://river.styx.org/ww/        <sip:ww at styx.org>
> F4B3 39BF E775 CF42 0BAB  3DF0 BE40 A6DF B06F FD45
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
>