[ECODP-dev] quick fix for the production database - Release 00.07.50.SP1

Thu Mar 7 16:20:44 UTC 2013

Hi Olivier,

I'm glad that the scripts we developed with 10F yesterday and last night
seem to be working. As discussed earlier in the week David and I are both
out of the office tomorrow so I'm afraid we won't be able to join the
status call tomorrow morning.

Best,

Darwin

Darwin Peltan
Project Manager

The Open Knowledge Foundation
http://www.okfn.org

Skype: darwinp
Twitter: @darwin

On 7 March 2013 16:05, ISOARD Olivier (OP) <
Olivier.ISOARD at publications.europa.eu> wrote:

> Hi Bert,****
>
> ** **
>
> Thanks for this new update.****
>
> ** **
>
> From our side, the run of the 2 scripts on our TEST environment was also
> conclusive and we now are managing the deployment and run of those scripts
> also on PROD, after having restored the last backup of prod 00.07.05.
> restore is done but cleaning (scripts) is still on going for the moment.**
> **
>
> ** **
>
> See u tomorrow to doing a new status via telco.****
>
> ** **
>
> Best regards,****
>
> ** **
>
> *Olivier ISOARD **
> **Projets Informatiques*****
>
> *_**__**_**__**_**__**__**_**__**_**__***
>
> * *
>
> *Office des publications de l'Union européenne *
> Direction A – Core Business Services
> Unité A3 - Projets Informatiques
>
> 2985 Luxembourg • LUXEMBOURG
> Tel: +352 292942360 • Fax: +352 292944604
> olivier.isoard at publications.europa.eu*
> **http://publications.europa.eu*****
>
> ** **
>
> *From:* Bert Van Nuffelen [mailto:bert.van.nuffelen at tenforce.com]
> *Sent:* Thursday, March 07, 2013 4:04 PM
> *To:* ISOARD Olivier (OP)
> *Cc:* Project list for EC ODP CKAN project; ZAJAC Agnieszka (OP); HOHN
> Norbert (OP); MEYER André (OP-EXT)
> *Subject:* Re: quick fix for the production database - Release
> 00.07.50.SP1****
>
> ** **
>
> Hi
>
> as promised a quick update from our side w.r.t. the upload of eurostat.
> The processing finished properly.
>
> I will trigger an additional run with some other eurostat zips to see if
> the behaviour is reoccurring.
>
> Bert****
>
> 2013/3/7 Bert Van Nuffelen <bert.van.nuffelen at tenforce.com>****
>
> Hi Olivier,
>
> it does not matter the release version. It is a valid operation for all
> CKAN databases.
>
> kind regards,
>
> Bert****
>
> 2013/3/7 ISOARD Olivier (OP) <Olivier.ISOARD at publications.europa.eu>****
>
> ** **
>
> Hello Bert,****
>
>  ****
>
> Thanks for this update. We already got the 2 scripts and are about to run
> them on our TEST env first. As this is pure db cleaning, it does not matter
> if we run it on 00.07.50 ?****
>
>  ****
>
> Please note that we named this *delivery 00.07.50.SP1*. (I did the update
> and tagging in SVN myself, preferring you’re still focusing on the
> complementary investigations J )****
>
>  ****
>
> Best regards,****
>
>  ****
>
> *Olivier ISOARD **
> **Projets Informatiques*****
>
> *_**__**_**__**_**__**__**_**__**_**__*****
>
> * *****
>
> *Office des publications de l'Union européenne *
> Direction A – Core Business Services
> Unité A3 - Projets Informatiques
>
> 2985 Luxembourg • LUXEMBOURG
> Tel: +352 292942360 • Fax: +352 292944604
> olivier.isoard at publications.europa.eu*
> **http://publications.europa.eu*****
>
>  ****
>
> *From:* Bert Van Nuffelen [mailto:bert.van.nuffelen at tenforce.com]
> *Sent:* Thursday, March 07, 2013 12:43 PM
> *To:* ISOARD Olivier (OP); ZAJAC Agnieszka (OP); HOHN Norbert (OP); MEYER
> André (OP-EXT)
> *Cc:* Project list for EC ODP CKAN project
> *Subject:* quick fix for the production database****
>
>  ****
>
> Dear,
>
> our investigations has lead to the first conclusion. The regular upload of
> (new) data increased the CKAN database table activity to a high number.
> Very lickely this causes the instability of the CKAN system.
>
> On our test system (a release 00.07.01) we activated the production dump.
> During the indexing (a process step required after changing the database in
> the back)
> the CKAN system halted. After emptying the activity table the indexing was
> restarted and it finished properly. After that we have run rdf2ckan with
> the latest zip you have provided.
> At this moment the EEA and SANCO zips passed. Currently a Eurostat zip is
> running.
>
> In the svn you find a script to clean the appropriate tables. This is a
> quick-win step towards having back a stable system.
>
>
> https://webgate.ec.europa.eu/publications/svn/ODP/trunk/ckan/management_scripts/ckan_clear_activity_table.sh
>
> Copy this script to $USERHOME/ckan_management and complete the necessary
> parameters. These are the same as for a CKAN backup. ****
>
> ·         TARGETBACKUPDIR****
>
> ·         CKAN_PWD****
>
> ·         POSTGRES_HOST****
>
> The script will first backup the table and afterwards empty the tables.
>
> The execution on our site costs +- 45 minutes and resulted in a file of
> 7.6 GB.
>
> After the script execution, you have to trigger for this time the
> reindexing of the database.
> In $USERHOME/ckan_management
>
>      ./ckan_reindex.sh
>
> (It  costs +- 1hour on our site)
>
> We keep you informed about the outcome of the Eurostat test.
>
> ps. I inserted the description of the new script also in the OM on section
> 7.10.
>
> kind regards,
>
> Bert
>
>
> --
> Bert Van Nuffelen
>
> Semantic Technologies Software Architect at TenForce
> www.tenforce.be
>
> Bert.Van.Nuffelen at tenforce.com
> Office: +32 (0)16 31 48 60
> Mobile:+32 479 06 24 26
> skype: bert.van.nuffelen ****
>
>
>
>
> --
> Bert Van Nuffelen
>
> Semantic Technologies Software Architect at TenForce
> www.tenforce.be
>
> Bert.Van.Nuffelen at tenforce.com
> Office: +32 (0)16 31 48 60
> Mobile:+32 479 06 24 26
> skype: bert.van.nuffelen ****
>
>
>
>
> --
> Bert Van Nuffelen
>
> Semantic Technologies Software Architect at TenForce
> www.tenforce.be
>
> Bert.Van.Nuffelen at tenforce.com
> Office: +32 (0)16 31 48 60
> Mobile:+32 479 06 24 26
> skype: bert.van.nuffelen ****
>
> _______________________________________________
> Ecodp-dev mailing list
> Ecodp-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130307/663384ba/attachment.html>