[ckan-dev] DCAT JSON harvester license property not importing

Thu Feb 16 19:58:20 UTC 2017

Nice one, I'll check out that extension Adrià.

Epic high fives for all the help everyone.

+1 open source community :D

On 16 February 2017 at 21:48, Adrià Mercader <adria.mercader at okfn.org>
wrote:

> Hi Cam,
>
> GSA maintain an extension to harvest data.json files:
> https://github.com/GSA/ckanext-datajson
>
> It might be worth investigating.
>
> Best,
>
> Adrià
>
> On 15 February 2017 at 20:41, Cam Findlay <cam at camfindlay.com> wrote:
> > Thanks for all the responses :)
> >
> > The json we are looking to harvest conform to the US Project Open Data
> > schema https://project-open-data.cio.gov/v1.1/schema/ (this seems to
> have a
> > license property at the dataset level).
> >
> > We've found that ESRI GIS open data portal have this baked in as a
> standard
> > endpoint if you simply add data.json to the end of the portal URL. Eg.
> > http://aucklandopendata-aucklandcouncil.opendata.arcgis.com/data.json
> >
> > Has anyone put together a custom profile for this schema we could reuse?
> >
> >
> > Many thanks,
> >
> > Cam.
> >
> >
> >
> >
> > On 15 February 2017 at 23:29, Adrià Mercader <adria.mercader at okfn.org>
> > wrote:
> >>
> >> That's a really interesting approach!
> >>
> >> I guess it's possible because a very clearly limited number of options
> >> which are easily ordered. It would be difficult to apply to a more
> >> generic list like the ones listed here
> >> http://opendefinition.org/licenses/
> >>
> >> But well worth a try if you have limited licensing options
> >>
> >> Adrià
> >>
> >> On 15 February 2017 at 10:15, Stefan Oderbolz <stefan.oderbolz at liip.ch>
> >> wrote:
> >> > For our portal we followed the approach of defining a "hierarchy of
> >> > openess"
> >> > of our terms of use (for legal reasons we don't have licenses, but
> terms
> >> > of
> >> > use, but it's basically the same).
> >> >
> >> > So the "terms of use" of a dataset is defined as the "least open terms
> >> > of
> >> > use of one it's resources", you can find the code here:
> >> >
> >> > https://github.com/opendata-swiss/ckanext-switzerland/blob/
> 74fc50090d5e4e27af9b44802bb8d693e37d4894/ckanext/
> switzerland/logic.py#L52-L88
> >> >
> >> >
> >> >
> >> > On Wed, Feb 15, 2017 at 10:32 AM, Adrià Mercader
> >> > <adria.mercader at okfn.org>
> >> > wrote:
> >> >>
> >> >> Hi,
> >> >>
> >> >> Yes, annoyingly DCAT defines license at the resource level and there
> >> >> isn't one at the dataset level that resources can inherit from (but
> >> >> there is one in catalog!)
> >> >>
> >> >> https://github.com/ckan/ckanext-dcat/issues/42
> >> >>
> >> >> Anyway, I think that the use case of grabbing the first license found
> >> >> on a resource and applying it to the dataset is a common enough one
> to
> >> >> implement it on the default profile, and people can override it if
> not
> >> >> happy with it (it will be better than no license at all).
> >> >>
> >> >> I'd be more than happy to review and merge a contribution if someone
> >> >> is up for it.
> >> >>
> >> >> Adrià
> >> >>
> >> >>
> >> >> On 15 February 2017 at 08:16, Stefan Oderbolz <
> stefan.oderbolz at liip.ch>
> >> >> wrote:
> >> >> > Hi Cam,
> >> >> >
> >> >> > I guess you are not using a custom DCAT profile, because then the
> >> >> > default
> >> >> > profile is being used which is "euro_dcat_ap", which is defined
> here:
> >> >> >
> >> >> >
> >> >> > https://github.com/ckan/ckanext-dcat/blob/
> e1245cdb5e29194787c33430db637be74e5bc2e1/ckanext/dcat/profiles.py#L577
> >> >> > This profile does not define licenses on a dataset level, but only
> in
> >> >> > a
> >> >> > level of a distribution (a.k.a. resources). Which DCAT-AP standard
> do
> >> >> > you
> >> >> > want to harvest?
> >> >> >
> >> >> > It might be needed to create your own profile to define exactly
> this
> >> >> > specifities. The process to do so is described in the README:
> >> >> > https://github.com/ckan/ckanext-dcat#profiles
> >> >> >
> >> >> > I hope this helps.
> >> >> >
> >> >> > Best regards Stefan
> >> >> >
> >> >> > On Tue, Feb 14, 2017 at 7:57 PM, Cam Findlay <cam at camfindlay.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> Hi all.
> >> >> >>
> >> >> >> Has anyone else had trouble with the DCAT JSON harvester in
> >> >> >> particular
> >> >> >> the
> >> >> >> license property not being set on the harvested dataset?
> >> >> >>
> >> >> >> Our JSON source includes at the dataset level:
> >> >> >>
> >> >> >> "license": "https://creativecommons.org/licenses/by/4.0/",
> >> >> >>
> >> >> >> However the final result is no license gets set for the dataset
> post
> >> >> >> harvest. I'd expect "Creative Commons Attribution" to show up but
> we
> >> >> >> get
> >> >> >> nothing. If I edit the dataset and manually set the license then
> it
> >> >> >> shows
> >> >> >> fine. So it seems to be lost on import for some reason.
> >> >> >>
> >> >> >> No problems with the CKAN to CKAN harvester, we get the license
> >> >> >> correctly
> >> >> >> so just checking if there is any known issues around DCAT JSON and
> >> >> >> license
> >> >> >> properties.
> >> >> >>
> >> >> >> Trying to determine if it's extension related or a problem at our
> >> >> >> end.
> >> >> >>
> >> >> >> (Running CKAN 2.6.X, Ubuntu 14.04 LTS)
> >> >> >>
> >> >> >> Thanks in advance :)
> >> >> >>
> >> >> >> Cam.
> >> >> >>
> >> >> >> _______________________________________________
> >> >> >> ckan-dev mailing list
> >> >> >> ckan-dev at lists.okfn.org
> >> >> >> https://lists.okfn.org/mailman/listinfo/ckan-dev
> >> >> >> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> >> >> >>
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Liip AG  // Limmatstrasse 183 //  CH-8005 Zürich
> >> >> > Tel +41 43 500 39 80 // GnuPG 0x7B588C67 // www.liip.ch
> >> >> >
> >> >> > _______________________________________________
> >> >> > ckan-dev mailing list
> >> >> > ckan-dev at lists.okfn.org
> >> >> > https://lists.okfn.org/mailman/listinfo/ckan-dev
> >> >> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> >> >> >
> >> >> _______________________________________________
> >> >> ckan-dev mailing list
> >> >> ckan-dev at lists.okfn.org
> >> >> https://lists.okfn.org/mailman/listinfo/ckan-dev
> >> >> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Liip AG  // Limmatstrasse 183 //  CH-8005 Zürich
> >> > Tel +41 43 500 39 80 // GnuPG 0x7B588C67 // www.liip.ch
> >> >
> >> > _______________________________________________
> >> > ckan-dev mailing list
> >> > ckan-dev at lists.okfn.org
> >> > https://lists.okfn.org/mailman/listinfo/ckan-dev
> >> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> >> >
> >> _______________________________________________
> >> ckan-dev mailing list
> >> ckan-dev at lists.okfn.org
> >> https://lists.okfn.org/mailman/listinfo/ckan-dev
> >> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> >
> >
> >
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > https://lists.okfn.org/mailman/listinfo/ckan-dev
> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> >
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20170217/5b67b54e/attachment-0003.html>