[ckan-dev] Performance CKAN / Configuration Parameters

Wed Aug 6 17:54:26 UTC 2014

Hi all,

we do performance tests with following setup:
- CKAN 2.1.2
- Search queries via the default ckan-portal and a web-portal
- Ramp tests of 50,100,200...600...1000,...5000 users per second (!)
   (we expect such numbers in the early on-line phase of our portal)
- 24 GB RAM, 8 CPUs

Following parameter configurations in:
1) apache2.conf
  ServerLimits 300
  MaxClients 300
  for all occurrences

2) Jetty Java_Options:
-Xms512M -Xmx4g

3) virtual host ckan_default:
WSGIDaemonProcess ckan_default display-name=ckan_default processes=2 
threads=30

We get response time of ca. mean 30 seconds each of 600 concurrent users 
per second.
And several errors. Which altogether we feel bad with.

The CPUs are 50% active, RAM uses only ca. 5GB (of the 24 GB).
The ckan-portal and web-portal have same results.

Is there somebody who can explain the above parameters and optimal 
settings of them and their influences?
E.g. do apache workers correspond to threads? Are there multiple jetty 
processes or only one if CKAN is running?
Has anybody experiences in this direction or hints to further information?

Best wishes,
Lothar

-- 
Dr. Lothar Hotz
HITeC e.V., University of Hamburg
Vogt-Kölln-Str. 30, 22527 Hamburg
Tel: 040/42883-2605; Fax: 040/42883-2572
E-Mail: hotz at informatik.uni-hamburg.de
WWW: www.hitec-hh.de
Private page: kogs-www.informatik.uni-hamburg.de/~hotz