[ECODP-dev] no license causes python exception

Bert Van Nuffelen bert.van.nuffelen at tenforce.com
Tue Aug 20 14:17:07 UTC 2013


Hi John,

that is fine. The problem is the python exception for me. We can block at
the RDF2CKAN the absence of a license if PO requires it.

Bert




2013/8/20 John Glover <john.glover at okfn.org>

> Thanks Bert,
>
> I can reproduce the bug with that JSON and will fix it now. Note that we
> do not currently ensure that the license field exists, so this call will
> succeed.
>
> Regards,
> John
>
>
> On 20 August 2013 15:49, Bert Van Nuffelen <bert.van.nuffelen at tenforce.com
> > wrote:
>
>> Hi John,
>>
>> the post of this json
>>
>> {"geographical_coverage":[],"published_by":"publ","status":"
>> http://ec.europa.eu/open-data/kos/dataset-status/Completed
>> ","keyword_string":[],"url":"http://example.com/test-nolicense","title":"Test
>> dataset nolicense","type_of_dataset":[],"description":"This is a
>> description about the test dataset
>> nolicense.","interoperability_level":[],"name":"PXYNt05Kle3DfqEZaovphQ","rdf":"<?xml
>> version=\"1.0\"?>\n<rdf:RDF\n    xmlns:ecodp=\"
>> http://ec.europa.eu/open-data/ontologies/ec-odp#\"\n    xmlns:rdf=\"
>> http://www.w3.org/1999/02/22-rdf-syntax-ns#\"\n    xmlns:foaf=\"
>> http://xmlns.com/foaf/0.1/\"\n    xmlns:dct=\"http://purl.org/dc/terms/\"\n
>> xmlns:xsd=\"http://www.w3.org/2001/XMLSchema#\"\n    xmlns:skos=\"
>> http://www.w3.org/2004/02/skos/core#\"\n    xmlns:dcat=\"
>> http://www.w3.org/ns/dcat#\"\n    xmlns:dc=\"
>> http://purl.org/dc/elements/1.1/\"\n    xmlns=\"
>> http://ec.europa.eu/open-data/ontologies/ec-odp-example#\"\n
>> xmlns:owl=\"http://www.w3.org/2002/07/owl#\"\n    xmlns:skos-xl=\"
>> http://www.w3.org/2008/05/skos-xl#\"\n    xmlns:dcam=\"
>> http://purl.org/dc/dcam/\"\n    xmlns:rdfs=\"
>> http://www.w3.org/2000/01/rdf-schema#\">\n<!-- Dataset type and URI,
>> don't fill in the CKAN catalog identifier here -->\n  <dcat:Dataset
>> rdf:about=\"http://example.com/test-nolicense\">\n  \t\n<!-- Title, 1..1
>> - can be filled in in multiple translations though-->\n    <dct:title
>> xml:lang=\"en\">Test dataset nolicense</dct:title>  \n    \n<!--
>> Description, 1..*-->         \n    <dct:description xml:lang=\"en\">This is
>> a description about the test dataset nolicense.</dct:description>\n\n\n<!--
>> Publisher 1..1 -->    \n    <dct:publisher>\n      <skos:Concept
>> rdf:about=\"
>> http://publications.europa.eu/resource/authority/corporate-body/PUBL\"/>\n
>> </dct:publisher>    \n\n<!-- Dataset issued 0..1 -->   \n
>> <dct:issued>2004-04-06T01:01:01</dct:issued>\n\n<!-- Dataset last
>> modification 0..1 -->       \n
>> <dct:modified>2012-04-07T01:01:01</dct:modified>\n\n<!-- Status 1..1
>> -->      \n    <ecodp:datasetStatus>\n      <skos:Concept rdf:about=\"
>> http://ec.europa.eu/open-data/kos/dataset-status/Completed\"/>\n
>> </ecodp:datasetStatus>\n\n
>> </dcat:Dataset>\n</rdf:RDF>\n","license_id":[],"language":[]}
>>
>> causes this python exception:
>>
>>
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1]   return
>> self.app(environ, start_response)
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] File
>> '/applications/ecodp/users/ecodp/ckan/ecp
>> ortal/lib/python2.6/site-packages/beaker/middleware.py', line 155 in
>> __call__
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1]   return
>> self.wrap_app(environ, session_start_
>> response)
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] File
>> '/applications/ecodp/users/ecodp/ckan/ecp
>> ortal/lib/python2.6/site-packages/repoze/who/middleware.py', line 87 in
>> __call__
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1]   app_iter =
>> app(environ, wrapper.wrap_start_r
>> esponse)
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] File
>> '/applications/ecodp/users/ecodp/ckan/ecp
>> ortal/lib/python2.6/site-packages/routes/middleware.py', line 131 in
>> __call__
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1]   response =
>> self.app(environ, start_response)
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] File
>> '/applications/ecodp/users/ecodp/ckan/ecp
>> ortal/lib/python2.6/site-packages/pylons/wsgiapp.py', line 125 in __call__
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1]   response =
>> self.dispatch(controller, environ
>> , start_response)
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] File
>> '/applications/ecodp/users/ecodp/ckan/ecp
>> ortal/lib/python2.6/site-packages/pylons/wsgiapp.py', line 324 in dispatch
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1]   return
>> controller(environ, start_response)
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] File
>> '/applications/ecodp/users/ecodp/ckan/ecp
>> ortal/src/ckan/ckan/controllers/api.py', line 69 in __call__
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1]   return
>> base.BaseController.__call__(self, en
>> viron, start_response)
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] File
>> '/applications/ecodp/users/ecodp/ckan/ecp
>> ortal/src/ckan/ckan/lib/base.py', line 238 in __call__
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1]   res =
>> WSGIController.__call__(self, environ,
>>  start_response)
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] File
>> '/applications/ecodp/users/ecodp/ckan/ecp
>> ortal/lib/python2.6/site-packages/pylons/controllers/core.py', line 221
>> in __call__
>> :
>> ...
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] File
>> '/applications/ecodp/users/ecodp/ckan/ecportal/src/ckan/ckan/logic/action/create.py',
>> line 132 in package_create
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1]   data, errors =
>> _validate(data_dict, schema, context)
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] File
>> '/applications/ecodp/users/ecodp/ckan/ecportal/src/ckan/ckan/lib/navl/dictization_functions.py',
>> line 228 in validate
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1]   converted_data,
>> errors = _validate(flattened, schema, context)
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] File
>> '/applications/ecodp/users/ecodp/ckan/ecportal/src/ckan/ckan/lib/navl/dictization_functions.py',
>> line 290 in _validate
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1]
>> convert(converter, key, converted_data, errors, context)
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] File
>> '/applications/ecodp/users/ecodp/ckan/ecportal/src/ckan/ckan/lib/navl/dictization_functions.py',
>> line 198 in convert
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1]   value =
>> converter(converted_data.get(key), context)
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] File
>> '/applications/ecodp/users/ecodp/ckan/ecportal/src/ckanext-ecportal-release-v1.8.1/ckanext/ecportal/validators.py',
>> line 369 in map_licenses
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1]   if
>> re.match(_cc_by_re, val):
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] File
>> '/usr/lib64/python2.6/re.py', line 137 in match
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1]   return
>> _compile(pattern, flags).match(string)
>> [Tue Aug 20 15:44:23 2013] [error] [client 127.0.0.1] TypeError: expected
>> string or buffer
>>
>>
>> kind regards,
>>
>> Bert
>>
>>
>>
>> 2013/8/20 John Glover <john.glover at okfn.org>
>>
>>> Hi Bert,
>>>
>>> I am unable to reproduce this, do you have a test case?
>>>
>>> Thanks,
>>> John
>>>
>>>
>>> On 13 July 2013 16:48, John Glover <john.glover at okfn.org> wrote:
>>>
>>>> Thanks Bert, we will look into this.
>>>>
>>>> John
>>>>
>>>>
>>>> On 12 July 2013 17:14, Bert Van Nuffelen <
>>>> bert.van.nuffelen at tenforce.com> wrote:
>>>>
>>>>> Hi John,
>>>>>
>>>>> when a package description does not have a license we get a python
>>>>> exception (noted in httpd logs)  instead of a proper respons.
>>>>>
>>>>> best,
>>>>>
>>>>> Bert
>>>>>
>>>>> --
>>>>> Bert Van Nuffelen
>>>>>
>>>>> Semantic Technologies Software Architect at TenForce
>>>>> www.tenforce.be
>>>>>
>>>>> Bert.Van.Nuffelen at tenforce.com
>>>>> Office: +32 (0)16 31 48 60
>>>>> Mobile:+32 479 06 24 26
>>>>> skype: bert.van.nuffelen
>>>>>
>>>>> _______________________________________________
>>>>> Ecodp-dev mailing list
>>>>> Ecodp-dev at lists.okfn.org
>>>>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>>>>
>>>>>
>>>>
>>>
>>> _______________________________________________
>>> Ecodp-dev mailing list
>>> Ecodp-dev at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>>
>>>
>>
>>
>> --
>> Bert Van Nuffelen
>>
>> Semantic Technologies Software Architect at TenForce
>> www.tenforce.be
>>
>> Bert.Van.Nuffelen at tenforce.com
>> Office: +32 (0)16 31 48 60
>> Mobile:+32 479 06 24 26
>> skype: bert.van.nuffelen
>>
>> _______________________________________________
>> Ecodp-dev mailing list
>> Ecodp-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>
>>
>
> _______________________________________________
> Ecodp-dev mailing list
> Ecodp-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>
>


-- 
Bert Van Nuffelen

Semantic Technologies Software Architect at TenForce
www.tenforce.be

Bert.Van.Nuffelen at tenforce.com
Office: +32 (0)16 31 48 60
Mobile:+32 479 06 24 26
skype: bert.van.nuffelen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130820/23f06b3f/attachment.html>


More information about the ecodp-dev mailing list