[ckan-discuss] Questions for "RDF/XML" of CKAN page

Sat Jul 16 12:03:20 BST 2011

William,
Thanks for your message. That does sound like an interesting possibility. I
will discuss with my colleagues which approach to take for now. I will ping
you later about the possibility to add the quality metrics into
semantic.ckan.net

Cheers,
Pablo

On Sat, Jul 16, 2011 at 11:38 AM, William Waites <ww at styx.org> wrote:

> Pablo, these should already be in semantic.ckan.net harvested from
> catalogue.data.gov.uk. They don't use the lodcloud conventions but
> some cleverly composed SPARQL queries should give you the results
> you want very quickly.
>
> If you want to store extra information that you calculate I can
> also happily give you update rights on the triplestore as long as
> you promise not to stomp on any existing named graphs :P
>
> Cheers,
> -w
>
> * [2011-07-15 17:47:10 +0200] Pablo Mendes <pablomendes at gmail.com> écrit:
>
> > David,
> > Thanks for the prompt answer.
> >
> > We are in the process of cataloging all datasets following the Linked
> Data
> > principles (linkeddata.org). Besides what is already in data.gov.uk, we
> also
> > collect some provenance, accessibility, interpretability, availability,
> > among other quality indicators.
> > http://www4.wiwiss.fu-berlin.de/lodcloud/ckan/validator/levels.html
> >
> > We could have pulled them into our own in-house database, but we thought
> it
> > would be a much useful effort if we contributed our metadata to CKAN
> (which
> > also allows dataset owners to add some more information they might have,
> and
> > potential users to rate the content somehow).
> >
> > We currently have some crowdsourcing going on:
> > http://lists.w3.org/Archives/Public/public-lod/2011Jul/0059.html
> > And in parallel I am trying to ingest other catalogs to have the info all
> in
> > one place.
> >
> > As a result of this effort, we will release a new version of the LOD
> cloud
> > diagram:
> > http://www4.wiwiss.fu-berlin.de/lodcloud/state/
> >
> > We could discuss better ways to do these things (e.g. we could keep the
> > quality metadata as voID files in our own servers, and the federated CKAN
> > search attaches them to the search results on-the-fly). However, we need
> a
> > short-term solution.
> >
> > We'd like to get this nice overview of how open data is interconnected on
> > the Web by next week. And, to the extend of my knowledge, pulling the
> > metadata for these catalogs into CKAN would be the best (temporary)
> > solution.
> >
> > What do you think?
> >
> > Cheers,
> > Pablo
> > On Fri, Jul 15, 2011 at 5:23 PM, David Read <david.read at okfn.org> wrote:
> >
> > > On 15 July 2011 15:39, Pablo Mendes <pablomendes at gmail.com> wrote:
> > > > David,
> > > > very helpful message for all of us! I have a similar need. I want to
> pull
> > > > all of the metadata in data.gov.uk about datasets that contain RDF
> into
> > > > CKAN.net. There are about 160 of them.
> > > >
> > >
> http://data.gov.uk/search/apachesolr_search?filters=sm_resource_formats%3ARDF
> > > > Would the same advice apply?
> > >
> > > The data.gov.uk datasets are in CKAN format already. e.g.
> > >
> > >
> http://catalogue.data.gov.uk/api/rest/package/ni-115-substance-misuse-by-young-people
> > > so it would be relatively simple to pull data.gov.uk datasets into
> > > another CKAN, such as thedatahub.org. We've been considering this, but
> > > not sure if the 7000 of those would flood thedatahub.org somewhat. And
> > > since you want to stay in step, how do you deal with edits at both
> > > ends?
> > >
> > > Our current thinking is to aim at:
> > > a) allowing the public to edit/correction metadata on data.gov.uk
> > > (with moderation)
> > > b) use federated search across all CKAN instances
> > >
> > > I'd be interested to hear why you suggested pulling the RDF datasets
> > > from data.gov.uk into thedatahub.org/ckan.net - do let us know your
> > > thoughts.
> > >
> > > David
> > >
> > > > Cheers,
> > > > Pablo
> > > > On Fri, Jul 15, 2011 at 4:16 PM, David Read <david.read at okfn.org>
> wrote:
> > > >>
> > > >> On 15 July 2011 12:18, Tetsuro TOYODA <toyoda at base.riken.jp> wrote:
> > > >> > Dear David,
> > > >> >
> > > >> > Thank you for giving us your quick reply.
> > > >> >
> > > >> > http://biolod.org is the download site of biological databases
> funded
> > > by
> > > >> > the National Database Integration Project of Japan.
> > > >> >
> > > >> > RDF files are already generated and supplied from the site.
> > > >> >
> > > >> > Could you advise us how to register the data files including RDFs
> into
> > > >> > CKAN.
> > > >>
> > > >> Tetsuro
> > > >> Basically we need to write a little code that transforms the biolod
> > > >> RDF metadata to the CKAN JSON format.
> > > >>
> > > >> To do a one-off import then you should use the CKAN API [1]. You
> could
> > > >> do it in Python and use the ckanclient library [2].
> > > >>
> > > >> Or to do a daily automatic import you would write a harvester.
> Derive
> > > >> from the base class [3]
> > > >> and write a custom "_create_or_update_package" method.
> > > >>
> > > >> This way has the advantage that the original RDF is stored alongside
> > > >> the CKAN version. We'd be happy to install this harvester in
> > > >> thedatahub.org or another CKAN instance.
> > > >>
> > > >> David
> > > >>
> > > >> [1] http://packages.python.org/ckan/api.html
> > > >> [2] https://bitbucket.org/okfn/ckanclient
> > > >> [3]
> > > >>
> > >
> https://bitbucket.org/okfn/ckanext-harvest/src/61844c8d2374/ckanext/harvest/harvesters/base.py
> > > >>
> > > >> >
> > > >> > Best regards,
> > > >> >
> > > >> > Tetsuro Toyoda.
> > > >> >
> > > >> > -----Original Message-----
> > > >> > From: d.t.read at gmail.com [mailto:d.t.read at gmail.com] On Behalf Of
> > > David
> > > >> > Read
> > > >> > Sent: Friday, July 15, 2011 7:52 PM
> > > >> > To: Koro NISHIKATA
> > > >> > Cc: ckan-discuss at lists.okfn.org
> > > >> > Subject: Re: [ckan-discuss] Questions for "RDF/XML" of CKAN page
> > > >> >
> > > >> > 2011/7/15 Koro NISHIKATA <koro at base.riken.jp>:
> > > >> >> Dear all of CKAN discuss mailing list,
> > > >> >>
> > > >> >> Hello, nice to meet you.
> > > >> >> I am a new member of this mailing list, "Koro NISHIKATA".
> > > >> >
> > > >> > Welcome Koro!
> > > >> >
> > > >> >> I have a question about "the Data Hub API / datapkg" section of
> CKAN
> > > >> >> page.
> > > >> >>  http://ckan.net/package/biolod-pdb
> > > >> >>
> > > >> >> Can a user register data to "RDF/XML" ?
> > > >> >> Or automatically produced after register or editing ?
> > > >> >
> > > >> > CKAN accepts data via a number of ways:
> > > >> > * the Web page form
> > > >> > * API (JSON format)
> > > >> > * harvesters (INSPIRE, custom formats, etc)
> > > >> >
> > > >> > So you are correct - we don't accept RDF/XML. This format is one
> of
> > > >> > the exported formats, derived from the raw data.
> > > >> >
> > > >> > David
> > > >> >
> > > >> >>
> > > >> >> [RDF/XML]
> > > >>
> > > >> >> >>
> > >
> http://semantic.ckan.net/record//be684dad-b24a-4f59-b8e2-969229200c63.rdf
> > > >> >>
> > > >> >> Best regard,
> > > >> >> Koro NISHIKATA
> > > >> >> E-mail: koro at base.riken.jp
> > > >> >>
> > > >
> > > >
> > >
>
> > _______________________________________________
> > ckan-discuss mailing list
> > ckan-discuss at lists.okfn.org
> > http://lists.okfn.org/mailman/listinfo/ckan-discuss
>
>
> --
> William Waites                <mailto:ww at styx.org>
> http://river.styx.org/ww/        <sip:ww at styx.org>
> F4B3 39BF E775 CF42 0BAB  3DF0 BE40 A6DF B06F FD45
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-discuss/attachments/20110716/53c27b2f/attachment.htm>