[ECODP-dev] quick fix for the production database - Release 00.07.50.SP1

Thu Mar 7 15:04:29 UTC 2013

Hi

as promised a quick update from our side w.r.t. the upload of eurostat.
The processing finished properly.

I will trigger an additional run with some other eurostat zips to see if
the behaviour is reoccurring.

Bert

2013/3/7 Bert Van Nuffelen <bert.van.nuffelen at tenforce.com>

> Hi Olivier,
>
> it does not matter the release version. It is a valid operation for all
> CKAN databases.
>
> kind regards,
>
> Bert
>
> 2013/3/7 ISOARD Olivier (OP) <Olivier.ISOARD at publications.europa.eu>
>
> Hello Bert,****
>>
>> ** **
>>
>> Thanks for this update. We already got the 2 scripts and are about to run
>> them on our TEST env first. As this is pure db cleaning, it does not matter
>> if we run it on 00.07.50 ?****
>>
>> ** **
>>
>> Please note that we named this *delivery 00.07.50.SP1*. (I did the
>> update and tagging in SVN myself, preferring you’re still focusing on the
>> complementary investigations J )****
>>
>> ** **
>>
>> Best regards,****
>>
>> ** **
>>
>> *Olivier ISOARD **
>> **Projets Informatiques*****
>>
>> *_**__**_**__**_**__**__**_**__**_**__***
>>
>> * *
>>
>> *Office des publications de l'Union européenne *
>> Direction A – Core Business Services
>> Unité A3 - Projets Informatiques
>>
>> 2985 Luxembourg • LUXEMBOURG
>> Tel: +352 292942360 • Fax: +352 292944604
>> olivier.isoard at publications.europa.eu*
>> **http://publications.europa.eu*****
>>
>> ** **
>>
>> *From:* Bert Van Nuffelen [mailto:bert.van.nuffelen at tenforce.com]
>> *Sent:* Thursday, March 07, 2013 12:43 PM
>> *To:* ISOARD Olivier (OP); ZAJAC Agnieszka (OP); HOHN Norbert (OP);
>> MEYER André (OP-EXT)
>> *Cc:* Project list for EC ODP CKAN project
>> *Subject:* quick fix for the production database****
>>
>> ** **
>>
>> Dear,
>>
>> our investigations has lead to the first conclusion. The regular upload
>> of (new) data increased the CKAN database table activity to a high number.
>> Very lickely this causes the instability of the CKAN system.
>>
>> On our test system (a release 00.07.01) we activated the production dump.
>> During the indexing (a process step required after changing the database in
>> the back)
>> the CKAN system halted. After emptying the activity table the indexing
>> was restarted and it finished properly. After that we have run rdf2ckan
>> with the latest zip you have provided.
>> At this moment the EEA and SANCO zips passed. Currently a Eurostat zip is
>> running.
>>
>> In the svn you find a script to clean the appropriate tables. This is a
>> quick-win step towards having back a stable system.
>>
>>
>> https://webgate.ec.europa.eu/publications/svn/ODP/trunk/ckan/management_scripts/ckan_clear_activity_table.sh
>>
>> Copy this script to $USERHOME/ckan_management and complete the necessary
>> parameters. These are the same as for a CKAN backup. ****
>>
>> ·         TARGETBACKUPDIR****
>>
>> ·         CKAN_PWD****
>>
>> ·         POSTGRES_HOST****
>>
>> The script will first backup the table and afterwards empty the tables.
>>
>> The execution on our site costs +- 45 minutes and resulted in a file of
>> 7.6 GB.
>>
>> After the script execution, you have to trigger for this time the
>> reindexing of the database.
>> In $USERHOME/ckan_management
>>
>>      ./ckan_reindex.sh
>>
>> (It  costs +- 1hour on our site)
>>
>> We keep you informed about the outcome of the Eurostat test.
>>
>> ps. I inserted the description of the new script also in the OM on
>> section 7.10.
>>
>> kind regards,
>>
>> Bert
>>
>>
>> --
>> Bert Van Nuffelen
>>
>> Semantic Technologies Software Architect at TenForce
>> www.tenforce.be
>>
>> Bert.Van.Nuffelen at tenforce.com
>> Office: +32 (0)16 31 48 60
>> Mobile:+32 479 06 24 26
>> skype: bert.van.nuffelen ****
>>
>
>
>
> --
> Bert Van Nuffelen
>
> Semantic Technologies Software Architect at TenForce
> www.tenforce.be
>
> Bert.Van.Nuffelen at tenforce.com
> Office: +32 (0)16 31 48 60
> Mobile:+32 479 06 24 26
> skype: bert.van.nuffelen
>

-- 
Bert Van Nuffelen

Semantic Technologies Software Architect at TenForce
www.tenforce.be

Bert.Van.Nuffelen at tenforce.com
Office: +32 (0)16 31 48 60
Mobile:+32 479 06 24 26
skype: bert.van.nuffelen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130307/c812af22/attachment.html>