[CKAN-support] ODP-383

[Bert Van Nuffelen]

Hi Adam,

the ESTAT package is a RDF2CKAN package containing about 5500 datasets in
RDF format. The RDF2CKAN tool will interprete those RDF dataset
descriptions and turn them into json to be sent as an update to the EU ODP
CKAN.

So far my tests show on our test systems that the package content is not
the origin of the problem. I suspect that it might be related to the load
the RDF2CKAN execution creates on the wsgi processes.
Here is the default config:

    WSGIScriptAlias /data /applications/ecodp/users/ecodp/ckan/conf/wsgi.py
    WSGIDaemonProcess ecodp display-name=ecodp processes=6 threads=15
maximum-requests=2000
    WSGIProcessGroup ecodp

    # pass authorization info on (needed for rest api)
    WSGIPassAuthorization On

kind regards,

Bert


2014-03-14 16:37 GMT+01:00 Adam McGreggor <adam.mcgreggor at okfn.org>:

> Bert,
>
> I can confirm that we’ve received this ticket, and it does look like
> something the warranty period should cover.
>
> I’ve assigned the task over to the support team, who’ll hopefully have
> more information.
>
> Could you clarify what that ESTAT package is, please.
>
> Adam
>
> On 13 Mar 2014, at 22:17, Bert Van Nuffelen <
> bert.van.nuffelen at tenforce.com> wrote:
>
> > Dear OKF,
> >
> > We recieved the next ticket ODP-383. This ticket reports a CKAN crash on
> production.
> >
> > The problem description is the following:
> >
> > >> start_quote
> >
> > We just discovered that today’s ESTAT package from 11:00 causes the
> webserver to crash. After investigating the log files I noticed a new error
> message right before the first crash, to confirm my suspicion I gave the go
> ahead to Gregory Wassmann to relaunch the ingestion and it crashed again.
> >
> >
> > Here is a snippet from said log file for the 2nd crash:
> >
> >
> > [Tue Mar 11 14:36:49 2014] [error] Exception AttributeError: "'NoneType'
> object has no attribute 'clearing'" in <bound method
> PluginEnvironment.__del__ of  Services for Environment '<default>'
> >
> > [Tue Mar 11 14:36:49 2014] [error] > ignored
> >
> > [Tue Mar 11 14:36:49 2014] [error] Exception AttributeError: "'NoneType'
> object has no attribute 'clearing'" in <bound method
> PluginEnvironment.__del__ of  Services for Environment '<default>'
> >
> > [Tue Mar 11 14:36:49 2014] [error] > ignored
> >
> > [Tue Mar 11 14:36:50 2014] [error] Exception AttributeError: "'NoneType'
> object has no attribute 'clearing'" in <bound method
> PluginEnvironment.__del__ of  Services for Environment '<default>'
> >
> > [Tue Mar 11 14:36:50 2014] [error] > ignored
> >
> > [Tue Mar 11 14:36:50 2014] [error] Exception AttributeError: "'NoneType'
> object has no attribute 'clearing'" in <bound method
> PluginEnvironment.__del__ of  Services for Environment '<default>'
> >
> > [Tue Mar 11 14:36:50 2014] [error] > ignored
> >
> > [Tue Mar 11 14:36:50 2014] [error] Exception AttributeError: "'NoneType'
> object has no attribute 'clearing'" in <bound method
> PluginEnvironment.__del__ of  Services for Environment '<default>'
> >
> > [Tue Mar 11 14:36:50 2014] [error] > ignored
> >
> > [Tue Mar 11 14:36:50 2014] [error] Exception AttributeError: "'NoneType'
> object has no attribute 'clearing'" in <bound method
> PluginEnvironment.__del__ of  Services for Environment '<default>'
> >
> > [Tue Mar 11 14:36:50 2014] [error] > ignored
> >
> > [Tue Mar 11 14:36:52 2014] [error] [client 158.167.99.23] (2)No such
> file or directory: mod_wsgi (pid=21578): Unable to connect to WSGI daemon
> process 'ecodp' on '/var/run/wsgi.21367.0.1.sock' after multiple attempts.
> >
> >
> > As you can see, there are a couple of messages that indicate the usage
> of a ‘clearing’ attribute which seems to cause the problem as right after
> these messages pop up, the server crashes, as indicated by the last line in
> the snippet that I have included here in the email.
> >
> >
> > << end_quote
> >
> >
> > Can you indicate from which CKAN module the above exceptions come? Note
> that I find the above exceptions in older logs and so far they never
> crashed the wsgi daemon (logs dating back to 4 september 2013).  Loading
> the ESTAT on our test system does not result in a WSGI crash.
> >
> >
> >
> >
> >
> > kind regards,
> >
> >
> >
> > Bert
> >
>
> --
> Adam McGreggor
> General Manager  |  skype: adamamyl  |  @adamamyl
> The Open Knowledge Foundation
> Empowering through Open Knowledge
> http://okfn.org/  |  @okfn
>
>


-- 
Bert Van Nuffelen

Semantic Technologies Software Architect at TenForce
www.tenforce.be

Bert.Van.Nuffelen at tenforce.com
Office: +32 (0)16 31 48 60
Mobile:+32 479 06 24 26
skype: bert.van.nuffelen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ckan-support/attachments/20140314/fb21a2db/attachment-0003.html>