[ECODP-dev] CKAN release notes for 09.00

Thu Jun 13 21:26:34 UTC 2013

Hi john,

thanks for the release notes and the associated instructions.

Some remarks.
The CKAN database update will not happen on the "life production database".
As this is a complete reinstall from zero (because of the rpm
packaging) the process to support migration is:
   * install a release 00.09.00 in test
   * migrate the production db (version v.1.7) to the release 00.09.00 in test
   * upgrade the content to the v.1.8
   => this process will be repeated for a new production system
whenever PO accepts this release and migration path.
   So if you see any issues here, let us know.

On the translations for the vocabularies.
=> how is that related to the json's I pointed you in the /ckan/data
directory. Does the process imports them from there? If not, how do we
configure CKAN to take the data from there.
=> For the Operations Manual. How does PO can update the translations
themselves?

does this affects the management of the publishers with their translations?

FYI, the EU ODP system is installed on 3 machines so "copying" cannot be done.

Kind regards,

Bert

2013/6/13 John Glover <john.glover at okfn.org>:
> Hello,
>
> I am currently writing the release notes for release 09.00. The latest draft
> of the setup instructions is given below, it would be good if you could
> verify that this process works on your acceptance server (starting with the
> CKAN 1.7 package and database). I am still working on the changelog so you
> should get the full version early next week.
>
> Let me know of any problems.
>
> Regards,
> John
>
> -----
>
> Release 09.00
> =============
>
> Unlike previous releases, the RPM is created by TenForce.
> Nothing has been added to the odp Git repository by the OKF.
>
> The CKAN codebase is available at:
> https://github.com/okfn/ckan
> branch: release-v1.8.1-ecportal
> commit hash: 1143f5c074dfffdd1409da289a609e7cf323567d
>
> The CKAN EC Portal Extension is available at:
> https://github.com/okfn/ckanext-ecportal
> branch: master
> commit hash: e3b7f35df98baec1412fa584ec8227e9d404ad3f
>
> Note: As this is only a draft of the release notes, both of thesse
> commit hashes may change.
>
>
> Installation
> ============
>
> * Install the new RPM.
>
> * Upgrade the database for CKAN 1.8.
>
> /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster --plugin=ckan db
> upgrade -c /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
>
> This command should return:
> <date> INFO  [ckan.model] CKAN database version upgraded: 57 -> 59
>
> * Add the database tables for the new search cloud feature.
>
> /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster
> --plugin=ckanext-ecportal ecportal searchcloud-install-tables -c
> /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
>
> This command should return:
> Creating the search_query table ...
> done.
> Creating the search_popular_latest table ...
> done.
> Creating the search_popular_approved table ...
> done.
> Creating the search_query_date index ...
> done.
>
> * Import the new translations of the fixed vocabularies.
>
> /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster
> --plugin=ckanext-ecportal ecportal import-csv-translations -c
> /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
>
> This command does not return anything.
>
> * Make sure that Solr is using the latest schema.xml file from the ckanext
> ecportal extension.
> This is at:
> /applications/ecodp/users/ecodp/ckan/ecportal/src/ckanext-ecportal/ckanext/ecportal/solr
> This can be copied to the solr conf directory:
>
> cp
> /applications/ecodp/users/ecodp/ckan/ecportal/src/ckanext-ecportal/ckanext/ecportal/solr/*
> /applications/ecodp/users/ecodp/solr/solr/conf
>
> * Restart tomcat.
>
> * Restart Apache.
>
> * Rebuild the search index (may take a while to run).
>
> /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster --plugin=ckan
> search-index -r rebuild -c
> /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
>
>
> CRON jobs
> ---------
>
> * The following should be run to update the list of recently searched terms:
>
> /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster
> --plugin=ckanext-ecportal ecportal
> searchcloud-generate-unapproved-search-list -c
> /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
>
> We recommend running this daily at midnight.
>
> * The following should be run to update the tracking information for the
> previous day:
>
> /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster --plugin=ckan
> tracking update -c /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
>
> We recommend running this daily at 1 minute past midnight.
>
>
> Database cleanup
> ----------------
>
> When investigating the issues with the rdf import failing, we noticed that
> some database tables had grown quite large and can be cleaned up in this
> release.
>
> 1. The package_extra_revision table contains data on a lot of old
> revisions that are not currently needed. These can be removed with the
> following
> paster command:
>
> /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster
> --plugin=ckanext-ecportal ecportal purge-package-extra-revision -c
> /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
>
> This command should return:
> <date> WARNI [root] {'revision_rows_deleted': <number of rows>}
>
> This table is in constant use, so this command could be scheduled to be run
> regularly. It should not need to be run too frequently however, once every
> week or two should be fine.
>
>
> 2. Some unused CKAN extensions (qa and datastorer) were installed in
> previous
> releases. These have been removed in this release, and so the tables that
> they were writing to can now be cleared. To do this, run:
>
> /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster
> --plugin=ckanext-ecportal ecportal purge-task-data -c
> /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
>
> This command should return:
> <date> WARNI [root] {'celery_rows_deleted': <number of rows>,
> 'task_status_rows_deleted': <number of rows>}
>
> The tables cleared by this command should not be used at all with the code
> in
> this release, so this command will only have to be run once.
>
>
> _______________________________________________
> Ecodp-dev mailing list
> Ecodp-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>

--
Bert Van Nuffelen

Semantic Technologies Software Architect at TenForce
www.tenforce.be

Bert.Van.Nuffelen at tenforce.com
Office: +32 (0)16 31 48 60
Mobile:+32 479 06 24 26
skype: bert.van.nuffelen