[ckan-dev] CKAN Timeout error

[David Read]

This is CKAN's SQLAlchemy running out of connections to the database.
You'll see it with when apache sees several simultaneous requests
under a bit of load. You can replicate it using something like 'ab' to
provide the simultaneous load.

The SQLAlchemy pool of connections is shared amongst threads in the
same CKAN process, but each CKAN process has its own pool.

You could increase the numbers of connections in the pool, but
remember that postgres has a limit of 100 connections by default. So
with multiple processes you'll hit a limit reasonably quickly. Or you
can limit the number of CKAN processess. I believe Ian noted a few
days ago one CKAN dependency that is not thread-safe, so best stick to
just processes for now.

If using Apache, you could run the mod_wsgi in daemon mode and specify
limits for the processes/threads. If you're on nginx/uwsgi then this
is naturally limits the number of processes, which is better.

The best solution would be to use a separate app to hold the
connection pool. That has the advantage of working across CKAN
threads. PGPool is an option, but we went for pgbouncer, which I
believe is faster and we've been very pleased with it.

David

On 16 November 2013 18:09, Pabitra Dash <pkdash_reena at hotmail.com> wrote:
> From time to time I am getting a timeout error from my CKAN server. The log
> file shows the following error message:
>
> TimeoutError: QueuePool  limit of size 5 overflow 10 reached.
>
> Any idea what's causing this and how to fix it?
>
> Thanks,
> Pabitra
>
> pkdash_reena at hotmail.com
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>