[ckan-dev] Problem with user_list

Wed May 17 12:24:54 UTC 2017

Hello,
I am coming back with this error. Time to time, I am getting on our
production server same error but on different lines in ckan core.
..ckan/lib/dictization/model_dictize:559
result_dict['number_of_edits'] = user.number_of_edits()
result_dict['number_created_packages'] = user.number_created_packages(...

Anyone faced this ? We have around 4400 registered users.

You can see below one full stack error:

URL: http://data.humdata.org/api/action/user_list
> Module weberror.errormiddleware:*162* in __call__
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> app_iter = self.application(environ, sr_checker)
> Module webob.dec:*147* in __call__
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> resp = self.call_func(req, *args, **self.kwargs)
> Module webob.dec:*208* in call_func
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return self.func(req, *args, **kwargs)
> Module fanstatic.publisher:*234* in __call__
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return request.get_response(self.app)
> Module webob.request:*1053* in get_response
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> application, catch_exc_info=False)
> Module webob.request:*1022* in call_application
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> app_iter = application(self.environ, start_response)
> Module webob.dec:*147* in __call__
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> resp = self.call_func(req, *args, **self.kwargs)
> Module webob.dec:*208* in call_func
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return self.func(req, *args, **kwargs)
> Module fanstatic.injector:*54* in __call__
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> response = request.get_response(self.app)
> Module webob.request:*1053* in get_response
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> application, catch_exc_info=False)
> Module webob.request:*1022* in call_application
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> app_iter = application(self.environ, start_response)
> Module beaker.middleware:*73* in __call__
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return self.app(environ, start_response)
> Module beaker.middleware:*155* in __call__
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return self.wrap_app(environ, session_start_response)
> Module routes.middleware:*131* in __call__
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> response = self.app(environ, start_response)
> Module newrelic.api.error_trace:*34* in wrapper
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return wrapped(*args, **kwargs)
> Module pylons.wsgiapp:*125* in __call__
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> response = self.dispatch(controller, environ, start_response)
> Module pylons.wsgiapp:*324* in dispatch
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return controller(environ, start_response)
> Module ckan.controllers.api:*70* in __call__
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return base.BaseController.__call__(self, environ, start_response)
> Module ckan.lib.base:*338* in __call__
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> res = WSGIController.__call__(self, environ, start_response)
> Module newrelic.api.function_trace:*110* in literal_wrapper
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return wrapped(*args, **kwargs)
> Module newrelic.api.transaction_name:*38* in dynamic_wrapper
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return wrapped(*args, **kwargs)
> Module pylons.controllers.core:*221* in __call__
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> response = self._dispatch_call()
> Module pylons.controllers.core:*172* in _dispatch_call
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> response = self._inspect_call(func)
> Module pylons.controllers.core:*107* in _inspect_call
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> result = self._perform_call(func, args)
> Module newrelic.hooks.framework_pylons:*35* in __call__
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return self.__wrapped(*args, **kwargs)
> Module newrelic.api.function_trace:*98* in dynamic_wrapper
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return wrapped(*args, **kwargs)
> Module pylons.controllers.core:*60* in _perform_call
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return func(**args)
> Module ckan.controllers.api:*197* in action
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> result = function(context, request_data)
> Module ckan.logic:*424* in wrapped
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> result = _action(context, data_dict, **kw)
> Module ckan.logic.action.get:*839* in user_list
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> result_dict = model_dictize.user_dictize(user[0], context)
> Module ckan.lib.dictization.model_dictize:*579* in user_dictize
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> 'count_private_and_draft_datasets', False))
> Module ckan.model.user:*210* in number_created_packages
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return q.count()
> Module sqlalchemy.orm.query:*2594* in count
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return self.from_self(col).scalar()
> Module sqlalchemy.orm.query:*2393* in scalar
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> ret = self.one()
> Module sqlalchemy.orm.query:*2362* in one
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> ret = list(self)
> Module sqlalchemy.orm.query:*2401* in __iter__
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> context = self._compile_context()
> Module sqlalchemy.orm.query:*2764* in _compile_context
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> entity.setup_context(self, context)
> Module sqlalchemy.orm.query:*3475* in setup_context
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> query, self.column, context)
> Module sqlalchemy.orm.query:*3459* in _resolve_expr_against_query_aliases
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return query._adapt_clause(expr, False, True)
> Module sqlalchemy.orm.query:*281* in _adapt_clause
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> replace
> Module sqlalchemy.sql.visitors:*313* in replacement_traverse
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> obj = clone(obj, **opts)
> Module sqlalchemy.sql.visitors:*309* in clone
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> newelem._copy_internals(clone=clone, **kw)
> Module sqlalchemy.sql.elements:*2827* in _copy_internals
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> self.element = clone(self.element, **kw)
> Module sqlalchemy.sql.visitors:*302* in clone
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> newelem = replace(elem)
> Module sqlalchemy.orm.query:*274* in replace
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> e = adapter(elem)
> Module sqlalchemy.sql.util:*523* in replace
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> self.selectable.is_derived_from(col):
> Module sqlalchemy.sql.selectable:*974* in is_derived_from
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> return self.element.is_derived_from(fromclause)
> Module sqlalchemy.sql.selectable:*2377* in is_derived_from
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> for f in self.locate_all_froms():
> Module sqlalchemy.util.langhelpers:*741* in oneshot
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> result = self.fget(obj, *args, **kw)
> Module sqlalchemy.sql.selectable:*2362* in locate_all_froms
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> froms = self._froms
> Module sqlalchemy.sql.selectable:*2230* in _froms
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> add(_from_objects(self._whereclause))
> Module sqlalchemy.sql.selectable:*2224* in add
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> if not seen.intersection(item._cloned_set):
> Module gunicorn.workers.base:*159* in handle_abort
> >>
> <https://mail.google.com/mail/u/0/?ui=2&ik=1687dee513&view=lg&msg=15c16502572d3954#m_6506355832918824652_>
> sys.exit(1)
>

Thank you!

*Dan Mihaila*, IT Consultant
(M) +40 722 502 304 • (GTalk) dan.mihaila at gmail.com • (Skype) carcotelul
• (Twitter) dan_mihaila

On Tue, Apr 4, 2017 at 11:36 AM, Dan Mihaila <danmihaila at gmail.com> wrote:

> Hello,
> I noticed there is a timeout when trying the api call to retrieve the list
> of users.
> e.g http://demo.ckan.org/api/3/action/user_list
> I tried 3 times on demo.ckan.org and 1 time it worked after aprox 45
> seconds, but i got also timeout.
> Do you know why is so slow? I think demo is running on 2.5.
>
> Thanks,
>
> *Dan Mihaila*, IT Consultant
> (M) +40 722 502 304 <0722%20502%20304> • (GTalk) dan.mihaila at gmail.com • (Skype)
> carcotelul
> • (Twitter) dan_mihaila • (Yahoo) carcotelul
>   <dmihaila at dginternational.org>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20170517/dbdd7e5e/attachment-0002.html>