[ECODP-dev] performance CKAN inserts

Mon Jul 8 12:39:37 UTC 2013

Hi,

On reaction to the remark of PO we do ESTAT insert.
We have at this moment in our setup on odp.tenforce.com approximal 20s
per package create.

127.0.0.1 - - [08/Jul/2013:14:28:22 +0200] "POST
/data/api/action/package_create HTTP/1.1" 200 39145
1 "-" "Apache-HttpClient/4.1.3 (java 1.5)"
127.0.0.1 - - [08/Jul/2013:14:28:41 +0200] "POST
/data/api/action/term_translation_update_many HTTP/

This means for 5000 datasets approximal 27 hours!
I attached the httpd logs and a first set of Estat insertion logs
(contains sample json files)

best,

Bert

--
Bert Van Nuffelen

Semantic Technologies Software Architect at TenForce
www.tenforce.be

Bert.Van.Nuffelen at tenforce.com
Office: +32 (0)16 31 48 60
Mobile:+32 479 06 24 26
skype: bert.van.nuffelen
-------------- next part --------------
A non-text attachment was scrubbed...
Name: okf.log.gz
Type: application/x-gzip
Size: 50874 bytes
Desc: not available
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130708/b631db76/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: estat.log.gz
Type: application/x-gzip
Size: 2058736 bytes
Desc: not available
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130708/b631db76/attachment-0003.bin>