[ECODP-dev] Fwd: FW: [ODP] 09.00 Installation update - general status - comments

Tue Jul 9 10:05:04 UTC 2013

Here is an extract of the log trace on httpd:

127.0.0.1 - - [09/Jul/2013:12:02:51 +0200] "POST
/data/api/action/term_translation_update_many HTTP/1.1" 200 493 "-"
"Apache-HttpClient/4.1.3 (java 1.5)"
127.0.0.1 - - [09/Jul/2013:12:02:51 +0200] "POST
/data/api/action/term_translation_update_many HTTP/1.1" 200 493 "-"
"Apache-HttpClient/4.1.3 (java 1.5)"
127.0.0.1 - - [09/Jul/2013:12:02:51 +0200] "POST
/data/api/action/term_translation_update_many HTTP/1.1" 200 493 "-"
"Apache-HttpClient/4.1.3 (java 1.5)"
127.0.0.1 - - [09/Jul/2013:12:02:52 +0200] "POST
/data/api/action/package_update HTTP/1.1" 200 280657 "-"
"Apache-HttpClient/4.1.3 (java 1.5)"
127.0.0.1 - - [09/Jul/2013:12:03:06 +0200] "POST
/data/api/action/term_translation_update_many HTTP/1.1" 200 493 "-"
"Apache-HttpClient/4.1.3 (java 1.5)"
127.0.0.1 - - [09/Jul/2013:12:03:06 +0200] "POST
/data/api/action/term_translation_update_many HTTP/1.1" 200 493 "-"
"Apache-HttpClient/4.1.3 (java 1.5)"
127.0.0.1 - - [09/Jul/2013:12:03:06 +0200] "POST
/data/api/action/term_translation_update_many HTTP/1.1" 200 493 "-"
"Apache-HttpClient/4.1.3 (java 1.5)"
127.0.0.1 - - [09/Jul/2013:12:03:06 +0200] "POST
/data/api/action/package_update HTTP/1.1" 200 282090 "-"
"Apache-HttpClient/4.1.3 (java 1.5)"
127.0.0.1 - - [09/Jul/2013:12:03:21 +0200] "POST
/data/api/action/term_translation_update_many HTTP/1.1" 200 493 "-"
"Apache-HttpClient/4.1.3 (java 1.5)"
127.0.0.1 - - [09/Jul/2013:12:03:21 +0200] "POST
/data/api/action/term_translation_update_many HTTP/1.1" 200 493 "-"
"Apache-HttpClient/4.1.3 (java 1.5)"
127.0.0.1 - - [09/Jul/2013:12:03:21 +0200] "POST
/data/api/action/term_translation_update_many HTTP/1.1" 200 493 "-"
"Apache-HttpClient/4.1.3 (java 1.5)"

2013/7/9 Bert Van Nuffelen <bert.van.nuffelen at tenforce.com>

> Hi Darwin,
>
> the first 20 calls were 3 seconds, the next 20 went to 4 seconds and now
> we are at 10 seconds per package update call...
>
> There is a log-N factor going on...
>
> Bert
>
>
> 2013/7/9 Darwin Peltan <darwin.peltan at okfn.org>
>
>> That's great. How quick is the import when you run it on this server?
>>
>> Best,
>>
>> Darwin
>>
>>
>> On Tuesday, July 9, 2013, Dimitrios Mexis wrote:
>>
>>> on 212 machine, rdf2ckan is installed. On  /root there is the ESTAT zip
>>> package for making copies.
>>> You can copy it anytime to the
>>> /applications/ecodp/users/ecodp/rdf2ckan/work/import folder. before you run
>>> the software
>>>
>>> Go to /applications/ecodp/users/ecodp/rdf2ckan/bin
>>> and issue the command ./rdf2ckan.sh -c
>>>
>>> this flag is one of the options you can have from the utility, and will
>>> validate and try to import the ESTAT zip package.
>>>
>>> Currently we have already started the software running.
>>> In the logs you can see relevant output.
>>>
>>> _WORKFLOW_ is the more detailed
>>> _BUSINESSFLOW_ is little less output than previous
>>>
>>> Regards
>>> Dimitrios
>>>
>>> On Tue, Jul 9, 2013 at 11:09 AM, Bert Van Nuffelen <
>>> bert.van.nuffelen at tenforce.com> wrote:
>>>
>>> We will do this.
>>>
>>> Bert
>>>
>>>
>>> 2013/7/9 Darwin Peltan <darwin.peltan at okfn.org>
>>>
>>> Hi Bert,
>>>
>>> It would be very helpful if you could try the import on the 212.x server
>>> please and see if the issue is the same there. If so we can quickly rule
>>> out the possibility that it's a config issue and look at other causes.
>>>
>>> Would that be OK please?
>>>
>>> Thanks,
>>>
>>> Darwin
>>>
>>> On Tuesday, July 9, 2013, Bert Van Nuffelen wrote:
>>>
>>> Hi Darwin,
>>>
>>> this is indeed on the case on odp.tenforce.com too. We did our tests
>>> yesterday with the information provided by PO and we obtain similar
>>> performance results.
>>> On the test machine, I do not know what John did. So no statement from
>>> my side if that is representable.
>>>
>>> Note that this will block probably the acceptance of release 00.09.00
>>> and might lead to a rejection of the release.
>>>
>>> best regards,
>>>
>>> Bert
>>>
>>>
>>>
>>>
>>> 2013/7/9 Darwin Peltan <darwin.peltan at okfn.org>
>>>
>>> Hi Bert,
>>>
>>> I am very surprised by this as we obviously  wouldn't expect this. We
>>> will do some initial investigations this week and come back to you. However
>>> as discussed we have limited availability this week so if we're not able to
>>> diagnose the issue we'll need to continue investigating next week.
>>>
>>> Is the performance the same on our test server (212.x) as
>>> ODP.tenforce.con?
>>>
>>> Best,
>>>
>>> Darwin
>>>
>>>
>>>
>>> On Tuesday, July 9, 2013, Bert Van Nuffelen wrote:
>>>
>>> Hi Darwin, John and David,
>>>
>>> Can you have a look to the performance. Currentltly release 00.09.00 is
>>> 80 times slower on ingest (from 250ms to 20s).
>>> That is a serious regression. Would it be possible to give a short
>>> status for this afternoon (call with PO at 1400h Brussels time) on the
>>> investigation and what is the action plan to get further improvements.
>>>
>>> best regards,
>>>
>>> Bert
>>>
>>> ---------- Forwarded message ----------
>>> From: *ZAJAC Agnieszka (OP)* <Agnieszka.ZAJAC at publications.europa.eu>
>>> Date: 2013/7/8
>>> Subject: FW: [ODP] 09.00 Installation update - general status - comments
>>> To: "Bert.Van.Nuffelen at tenforce.com" <Bert.Van.Nuffelen at tenforce.com>
>>> Cc: "MEYER André (OP-EXT)" <Andre.MEYER at ext.publications.europa.eu>,
>>> "SABETE Vafa (OP)" <Vafa.SABETE at publications.europa.eu>, "ISOARD
>>> Olivier (OP)" <Olivier.ISOARD at publications.europa.eu>, "HOHN Norbert
>>> (OP)" <Norbert.HOHN at publications.europa.eu>
>>>
>>>
>>>  Hello Bert,****
>>>
>>>
>>
>> --
>>
>> *Darwin Peltan*
>>
>> *Project Manager  |  skype: darwinp  |  twitter: @darwin<http://twitter.com/darwin>
>> *
>>
>> *The Open Knowledge Foundation <http://okfn.org/>*
>>
>> *Empowering through Open Knowledge*
>>
>> *http://okfn.org/  |  @okfn <http://twitter.com/OKFN>  |  OKF on Facebook<https://www.facebook.com/OKFNetwork> |
>> Blog <http://blog.okfn.org/>  |  Newsletter<http://okfn.org/about/newsletter>
>> *
>> *
>> CKAN | http://ckan.org | @ckanproject <http://twitter.com/ckanproject> |open source data management platform
>> *
>>
>>
>> _______________________________________________
>> Ecodp-dev mailing list
>> Ecodp-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>
>>
>
>
> --
> Bert Van Nuffelen
>
> Semantic Technologies Software Architect at TenForce
> www.tenforce.be
>
> Bert.Van.Nuffelen at tenforce.com
> Office: +32 (0)16 31 48 60
> Mobile:+32 479 06 24 26
> skype: bert.van.nuffelen
>

-- 
Bert Van Nuffelen

Semantic Technologies Software Architect at TenForce
www.tenforce.be

Bert.Van.Nuffelen at tenforce.com
Office: +32 (0)16 31 48 60
Mobile:+32 479 06 24 26
skype: bert.van.nuffelen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130709/7cf63dcd/attachment.html>