[ckan-dev] Database connections per worker

Fri May 2 10:20:30 UTC 2014

On 02/05/14 11:15, Alice Heaton wrote:
> On 01/05/14 16:20, Alice Heaton wrote:
>> On 01/05/14 14:33, Alice Heaton wrote:
>>> I was wondering how ckan handles it's PostgreSQL connections.
>>
>> So to answer my question, and as far as I can tell, the pooling is 
>> handled by SqlAlchemy.
>>
>> ckan creates one [*] SqlAlchemy engine that is used for querying 
>> models, and passes it the main config file option. This means that 
>> the pool configuration can be set by setting sqlalchemy.pool_size, 
>> sqlalchemy.max_overflow, etc. This pool is shared across threads.
>
> To continue on the topic:
>
> The default setting for the SqlAlchemy pool is to have up to 15 
> PostgreSQL connections open (pool_size=5,max_overflow=10).
> The default suggested configuration for production is to have 15 
> threads per process.
>
> This suggests that the expected behaviour is that a single thread will 
> not use more than one connection at a time (per SqlAlchemy engine).
>
> Is this correct?
>
> In that case, given that we have three engines (with the datastore 
> extension) then we shouldn't expect more than 3 connections per 
> thread. So, sticking to the defaults, we could have up 3 * 15 * 2 = 90 
> concurrent connections to the PostgreSql server.

Sorry, there are four engines (one created by ckan, three by the 
datastore extension) - so that's actually up to 120 connections - more 
than the default PostgreSQL number of allowed connections I think (100).

>
> Thanks,
> Alice
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev