[ECODP-dev] Investigation of Estat ingestion problems

Thu Apr 25 09:42:36 UTC 2013

Hi Agnieszka,

That is good news about 00.08.04.

W.r.t. DG COMP (jira 238) I did a test yesterday with your examples.

One example (  COMP_20130411164248.zip ) is not a zip at all: so obviously
the package cannot be processed.
That was mentioned in the some of the logs you have mailed me before.

The other one attached to the jira ticket works indeed.

Bert

2013/4/25 ZAJAC Agnieszka (OP) <Agnieszka.ZAJAC at publications.europa.eu>

>  Hi Bert,****
>
> ** **
>
> Thank you very much for a quick response. ****
>
> ** **
>
> Just to let you know: André has just installed 00.08.04 without problems Jand is in the process of ingesting data.
> ****
>
> ** **
>
> Regarding Estat: we noticed it occurs only with morning packages, the ones
> from afternoon are ingesting without problems.****
>
> Regarding Comp: André managed to ingest on test the latest package so I'm
> a bit confused about your last remark saying that something is still not
> correct. ****
>
> ** **
>
> @ André: when you're done with ingestion could you provide Bert with the
> logs requested?****
>
> ** **
>
> Thank you for looking closer at this issue.****
>
> ** **
>
> Regards,****
>
> Agnieszka****
>
> ** **
>
> *From:* Bert Van Nuffelen [mailto:bert.van.nuffelen at tenforce.com]
> *Sent:* Thursday, April 25, 2013 11:01 AM
> *To:* ZAJAC Agnieszka (OP); Project list for EC ODP CKAN project
> *Cc:* HOHN Norbert (OP)
> *Subject:* Re: Investigation of Estat issue****
>
> ** **
>
> Hi Agnieszka,****
>
> my investigation (jira ODP-239) has led to the conclusion that the
> publisher and its package for eurostat is not the problem, but it is a pure
> system operational problem.****
>
> In the attached logs the error on which RDF2CKAN halts is the last line:**
> **
>
> 2013-04-24 07:40:06,080 main ERROR org.ckan.demo.ecportal.rdf2ckan.http.JsonAPI UPLOAD ERROR: No JSON output found.****
>
>  This means that CKAN/apache did not returned a respons to the request.
> Consequently the process stopped. ****
>
> I could reproduce this problem one time and found then the next error
> message in the next apache logs****
>
>
> /applications/ecodp/users/ecodp/logs/httpd/error.error.log****
>
>
> [Tue Apr 23 10:43:03 2013] [error] Exception KeyError:
> KeyError(139770906142688,) in <module 'threading' from
> '/usr/lib64/python2.6/threading.pyc'> ignored
> [Tue Apr 23 10:43:03 2013] [error] 2013-04-23 10:43:03,948 ERROR
> [ckan.controllers.api] Validation error: "{'__type': 'Validation Error',
> 'name': [u'That URL is already in use.']}"
> [Tue Apr 23 10:43:05 2013] [error] 2013-04-23 10:43:05,146 ERROR
> [ckan.controllers.api] Validation error: "{'__type': 'Validation Error',
> 'name': [u'That URL is already in use.']}"
> [Tue Apr 23 10:43:05 2013] [error] Exception AttributeError: "'NoneType'
> object has no attribute 'clearing'" in****
>
> This indicates an exception created in the python container processes that
> handle the CKAN requests. This is outside the RDF2CKAN tool and has to do
> with a combination of process configuration and CKAN code. Which will be
> investigated. ****
>
> ** **
>
> To accomodate the problem:****
>
> If it happens, lets restart the apache service and start in ingestion
> process again for the same rdf2ckan package.****
>
>
> Secondly, I would like to recieve the content of the above log file that
> covers the period when it happened so that we have more information to
> pinpoint the problem.****
>
> Because reproducing it is not so straightforward as my tests showed. I
> could only reproduce it once.
>
> ****
>
> kind regards,****
>
> Bert****
>
>
>
> ****
>
> ** **
>
> 2013/4/25 ZAJAC Agnieszka (OP) <Agnieszka.ZAJAC at publications.europa.eu>***
> *
>
> Hello Bert,****
>
>  ****
>
> In the view of recurring problems with Estat updates would you have some
> time today to have a look at it so that I could come back to them as soon
> as possible.****
>
>  ****
>
> Thanks in advance,****
>
>  ****
>
> Agnieszka****
>
>  ****
>
> *Agnieszka Zając*****
>
> Open Data Portal Section****
>
> *_**__**_**__**_**__**__**_**__**_**__* ****
>
> *Publications Office of the European Union
> *Dissemination and Reuse Directorate ****
>
> Unit C1 Common Portal and Open Data Portal****
>
>
> L-2985 Luxembourg • LUXEMBOURG
> Tel: +352 2929.42834
> agnieszka.zajac at publications.europa.eu
> http://publications.europa.eu ****
>
> http://open-data.europa.eu****
>
>  ****
>
>  ****
>
>  ****
>
>
>
>
> --
> Bert Van Nuffelen
>
> Semantic Technologies Software Architect at TenForce
> www.tenforce.be
>
> Bert.Van.Nuffelen at tenforce.com
> Office: +32 (0)16 31 48 60
> Mobile:+32 479 06 24 26
> skype: bert.van.nuffelen ****
>

-- 
Bert Van Nuffelen

Semantic Technologies Software Architect at TenForce
www.tenforce.be

Bert.Van.Nuffelen at tenforce.com
Office: +32 (0)16 31 48 60
Mobile:+32 479 06 24 26
skype: bert.van.nuffelen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130425/de0fae8d/attachment.html>