[ckan-discuss] Questions for "RDF/XML" of CKAN page

[David Read]

On 15 July 2011 15:39, Pablo Mendes <pablomendes at gmail.com> wrote:
> David,
> very helpful message for all of us! I have a similar need. I want to pull
> all of the metadata in data.gov.uk about datasets that contain RDF into
> CKAN.net. There are about 160 of them.
> http://data.gov.uk/search/apachesolr_search?filters=sm_resource_formats%3ARDF
> Would the same advice apply?

The data.gov.uk datasets are in CKAN format already. e.g.
http://catalogue.data.gov.uk/api/rest/package/ni-115-substance-misuse-by-young-people
so it would be relatively simple to pull data.gov.uk datasets into
another CKAN, such as thedatahub.org. We've been considering this, but
not sure if the 7000 of those would flood thedatahub.org somewhat. And
since you want to stay in step, how do you deal with edits at both
ends?

Our current thinking is to aim at:
a) allowing the public to edit/correction metadata on data.gov.uk
(with moderation)
b) use federated search across all CKAN instances

I'd be interested to hear why you suggested pulling the RDF datasets
from data.gov.uk into thedatahub.org/ckan.net - do let us know your
thoughts.

David

> Cheers,
> Pablo
> On Fri, Jul 15, 2011 at 4:16 PM, David Read <david.read at okfn.org> wrote:
>>
>> On 15 July 2011 12:18, Tetsuro TOYODA <toyoda at base.riken.jp> wrote:
>> > Dear David,
>> >
>> > Thank you for giving us your quick reply.
>> >
>> > http://biolod.org is the download site of biological databases funded by
>> > the National Database Integration Project of Japan.
>> >
>> > RDF files are already generated and supplied from the site.
>> >
>> > Could you advise us how to register the data files including RDFs into
>> > CKAN.
>>
>> Tetsuro
>> Basically we need to write a little code that transforms the biolod
>> RDF metadata to the CKAN JSON format.
>>
>> To do a one-off import then you should use the CKAN API [1]. You could
>> do it in Python and use the ckanclient library [2].
>>
>> Or to do a daily automatic import you would write a harvester. Derive
>> from the base class [3]
>> and write a custom "_create_or_update_package" method.
>>
>> This way has the advantage that the original RDF is stored alongside
>> the CKAN version. We'd be happy to install this harvester in
>> thedatahub.org or another CKAN instance.
>>
>> David
>>
>> [1] http://packages.python.org/ckan/api.html
>> [2] https://bitbucket.org/okfn/ckanclient
>> [3]
>> https://bitbucket.org/okfn/ckanext-harvest/src/61844c8d2374/ckanext/harvest/harvesters/base.py
>>
>> >
>> > Best regards,
>> >
>> > Tetsuro Toyoda.
>> >
>> > -----Original Message-----
>> > From: d.t.read at gmail.com [mailto:d.t.read at gmail.com] On Behalf Of David
>> > Read
>> > Sent: Friday, July 15, 2011 7:52 PM
>> > To: Koro NISHIKATA
>> > Cc: ckan-discuss at lists.okfn.org
>> > Subject: Re: [ckan-discuss] Questions for "RDF/XML" of CKAN page
>> >
>> > 2011/7/15 Koro NISHIKATA <koro at base.riken.jp>:
>> >> Dear all of CKAN discuss mailing list,
>> >>
>> >> Hello, nice to meet you.
>> >> I am a new member of this mailing list, "Koro NISHIKATA".
>> >
>> > Welcome Koro!
>> >
>> >> I have a question about "the Data Hub API / datapkg" section of CKAN
>> >> page.
>> >>　 http://ckan.net/package/biolod-pdb
>> >>
>> >> Can a user register data to "RDF/XML" ?
>> >> Or automatically produced after register or editing ?
>> >
>> > CKAN accepts data via a number of ways:
>> > * the Web page form
>> > * API (JSON format)
>> > * harvesters (INSPIRE, custom formats, etc)
>> >
>> > So you are correct - we don't accept RDF/XML. This format is one of
>> > the exported formats, derived from the raw data.
>> >
>> > David
>> >
>> >>
>> >> [RDF/XML]
>>
>> >> >>　 http://semantic.ckan.net/record//be684dad-b24a-4f59-b8e2-969229200c63.rdf
>> >>
>> >> Best regard,
>> >> Koro NISHIKATA
>> >> E-mail: koro at base.riken.jp
>> >>
>
>