[ckan-dev] DCAT JSON harvester license property not importing

Thu Feb 16 08:48:29 UTC 2017

Hi Cam,

GSA maintain an extension to harvest data.json files:
https://github.com/GSA/ckanext-datajson

It might be worth investigating.

Best,

Adrià

On 15 February 2017 at 20:41, Cam Findlay <cam at camfindlay.com> wrote:
> Thanks for all the responses :)
>
> The json we are looking to harvest conform to the US Project Open Data
> schema https://project-open-data.cio.gov/v1.1/schema/ (this seems to have a
> license property at the dataset level).
>
> We've found that ESRI GIS open data portal have this baked in as a standard
> endpoint if you simply add data.json to the end of the portal URL. Eg.
> http://aucklandopendata-aucklandcouncil.opendata.arcgis.com/data.json
>
> Has anyone put together a custom profile for this schema we could reuse?
>
>
> Many thanks,
>
> Cam.
>
>
>
>
> On 15 February 2017 at 23:29, Adrià Mercader <adria.mercader at okfn.org>
> wrote:
>>
>> That's a really interesting approach!
>>
>> I guess it's possible because a very clearly limited number of options
>> which are easily ordered. It would be difficult to apply to a more
>> generic list like the ones listed here
>> http://opendefinition.org/licenses/
>>
>> But well worth a try if you have limited licensing options
>>
>> Adrià
>>
>> On 15 February 2017 at 10:15, Stefan Oderbolz <stefan.oderbolz at liip.ch>
>> wrote:
>> > For our portal we followed the approach of defining a "hierarchy of
>> > openess"
>> > of our terms of use (for legal reasons we don't have licenses, but terms
>> > of
>> > use, but it's basically the same).
>> >
>> > So the "terms of use" of a dataset is defined as the "least open terms
>> > of
>> > use of one it's resources", you can find the code here:
>> >
>> > https://github.com/opendata-swiss/ckanext-switzerland/blob/74fc50090d5e4e27af9b44802bb8d693e37d4894/ckanext/switzerland/logic.py#L52-L88
>> >
>> >
>> >
>> > On Wed, Feb 15, 2017 at 10:32 AM, Adrià Mercader
>> > <adria.mercader at okfn.org>
>> > wrote:
>> >>
>> >> Hi,
>> >>
>> >> Yes, annoyingly DCAT defines license at the resource level and there
>> >> isn't one at the dataset level that resources can inherit from (but
>> >> there is one in catalog!)
>> >>
>> >> https://github.com/ckan/ckanext-dcat/issues/42
>> >>
>> >> Anyway, I think that the use case of grabbing the first license found
>> >> on a resource and applying it to the dataset is a common enough one to
>> >> implement it on the default profile, and people can override it if not
>> >> happy with it (it will be better than no license at all).
>> >>
>> >> I'd be more than happy to review and merge a contribution if someone
>> >> is up for it.
>> >>
>> >> Adrià
>> >>
>> >>
>> >> On 15 February 2017 at 08:16, Stefan Oderbolz <stefan.oderbolz at liip.ch>
>> >> wrote:
>> >> > Hi Cam,
>> >> >
>> >> > I guess you are not using a custom DCAT profile, because then the
>> >> > default
>> >> > profile is being used which is "euro_dcat_ap", which is defined here:
>> >> >
>> >> >
>> >> > https://github.com/ckan/ckanext-dcat/blob/e1245cdb5e29194787c33430db637be74e5bc2e1/ckanext/dcat/profiles.py#L577
>> >> > This profile does not define licenses on a dataset level, but only in
>> >> > a
>> >> > level of a distribution (a.k.a. resources). Which DCAT-AP standard do
>> >> > you
>> >> > want to harvest?
>> >> >
>> >> > It might be needed to create your own profile to define exactly this
>> >> > specifities. The process to do so is described in the README:
>> >> > https://github.com/ckan/ckanext-dcat#profiles
>> >> >
>> >> > I hope this helps.
>> >> >
>> >> > Best regards Stefan
>> >> >
>> >> > On Tue, Feb 14, 2017 at 7:57 PM, Cam Findlay <cam at camfindlay.com>
>> >> > wrote:
>> >> >>
>> >> >> Hi all.
>> >> >>
>> >> >> Has anyone else had trouble with the DCAT JSON harvester in
>> >> >> particular
>> >> >> the
>> >> >> license property not being set on the harvested dataset?
>> >> >>
>> >> >> Our JSON source includes at the dataset level:
>> >> >>
>> >> >> "license": "https://creativecommons.org/licenses/by/4.0/",
>> >> >>
>> >> >> However the final result is no license gets set for the dataset post
>> >> >> harvest. I'd expect "Creative Commons Attribution" to show up but we
>> >> >> get
>> >> >> nothing. If I edit the dataset and manually set the license then it
>> >> >> shows
>> >> >> fine. So it seems to be lost on import for some reason.
>> >> >>
>> >> >> No problems with the CKAN to CKAN harvester, we get the license
>> >> >> correctly
>> >> >> so just checking if there is any known issues around DCAT JSON and
>> >> >> license
>> >> >> properties.
>> >> >>
>> >> >> Trying to determine if it's extension related or a problem at our
>> >> >> end.
>> >> >>
>> >> >> (Running CKAN 2.6.X, Ubuntu 14.04 LTS)
>> >> >>
>> >> >> Thanks in advance :)
>> >> >>
>> >> >> Cam.
>> >> >>
>> >> >> _______________________________________________
>> >> >> ckan-dev mailing list
>> >> >> ckan-dev at lists.okfn.org
>> >> >> https://lists.okfn.org/mailman/listinfo/ckan-dev
>> >> >> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Liip AG  // Limmatstrasse 183 //  CH-8005 Zürich
>> >> > Tel +41 43 500 39 80 // GnuPG 0x7B588C67 // www.liip.ch
>> >> >
>> >> > _______________________________________________
>> >> > ckan-dev mailing list
>> >> > ckan-dev at lists.okfn.org
>> >> > https://lists.okfn.org/mailman/listinfo/ckan-dev
>> >> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>> >> >
>> >> _______________________________________________
>> >> ckan-dev mailing list
>> >> ckan-dev at lists.okfn.org
>> >> https://lists.okfn.org/mailman/listinfo/ckan-dev
>> >> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>> >
>> >
>> >
>> >
>> > --
>> > Liip AG  // Limmatstrasse 183 //  CH-8005 Zürich
>> > Tel +41 43 500 39 80 // GnuPG 0x7B588C67 // www.liip.ch
>> >
>> > _______________________________________________
>> > ckan-dev mailing list
>> > ckan-dev at lists.okfn.org
>> > https://lists.okfn.org/mailman/listinfo/ckan-dev
>> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>> >
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>