[ECODP-dev] CKAN release notes for 09.00

Mon Jun 17 08:21:11 UTC 2013

Hi Bert,

Yes I would not expect the live database to be upgraded in place, and the
procedure that you describe sounds fine to me.

> => how is that related to the json's I pointed you in the /ckan/data
> directory. Does the process imports them from there? If not, how do we
> configure CKAN to take the data from there.

I don't follow your question about the vocabularies. I mentioned in a
previous email that these can be updated using our API, copied again here
for reference:

Those JSON files were not intended to be the official way of updating the
vocabularies. We use them to do the initial database setup, but I did not
even know that you had your own copy of them in the RPM. However, as all of
this information is stored in the database, it can be updated using the API.

To view or update vocabularies, see:
http://docs.ckan.org/en/ckan-1.8/apiv3.html#ckan.logic.action.get.vocabulary_list
http://docs.ckan.org/en/ckan-1.8/apiv3.html#ckan.logic.action.get.vocabulary_show
http://docs.ckan.org/en/ckan-1.8/apiv3.html#ckan.logic.action.create.vocabulary_create
http://docs.ckan.org/en/ckan-1.8/apiv3.html#ckan.logic.action.update.vocabulary_update

To view or update translations, see:
http://docs.ckan.org/en/ckan-1.8/apiv3.html#ckan.logic.action.get.term_translation_show
http://docs.ckan.org/en/ckan-1.8/apiv3.html#ckan.logic.action.update.term_translation_update_many

> => For the Operations Manual. How does PO can update the translations
> themselves?
> does this affects the management of the publishers with their
translations?

I am also not sure what you mean here, are you asking how the PO can update
the vocabulary translations? This would be the same as the answer above,
using the API.

Regards,
John

On 13 June 2013 23:26, Bert Van Nuffelen <bert.van.nuffelen at tenforce.com>wrote:

> Hi john,
>
> thanks for the release notes and the associated instructions.
>
> Some remarks.
> The CKAN database update will not happen on the "life production database".
> As this is a complete reinstall from zero (because of the rpm
> packaging) the process to support migration is:
>    * install a release 00.09.00 in test
>    * migrate the production db (version v.1.7) to the release 00.09.00 in
> test
>    * upgrade the content to the v.1.8
>    => this process will be repeated for a new production system
> whenever PO accepts this release and migration path.
>    So if you see any issues here, let us know.
>
> On the translations for the vocabularies.
> => how is that related to the json's I pointed you in the /ckan/data
> directory. Does the process imports them from there? If not, how do we
> configure CKAN to take the data from there.
> => For the Operations Manual. How does PO can update the translations
> themselves?
>
> does this affects the management of the publishers with their translations?
>
> FYI, the EU ODP system is installed on 3 machines so "copying" cannot be
> done.
>
> Kind regards,
>
> Bert
>
>
>
> 2013/6/13 John Glover <john.glover at okfn.org>:
> > Hello,
> >
> > I am currently writing the release notes for release 09.00. The latest
> draft
> > of the setup instructions is given below, it would be good if you could
> > verify that this process works on your acceptance server (starting with
> the
> > CKAN 1.7 package and database). I am still working on the changelog so
> you
> > should get the full version early next week.
> >
> > Let me know of any problems.
> >
> > Regards,
> > John
> >
> > -----
> >
> > Release 09.00
> > =============
> >
> > Unlike previous releases, the RPM is created by TenForce.
> > Nothing has been added to the odp Git repository by the OKF.
> >
> > The CKAN codebase is available at:
> > https://github.com/okfn/ckan
> > branch: release-v1.8.1-ecportal
> > commit hash: 1143f5c074dfffdd1409da289a609e7cf323567d
> >
> > The CKAN EC Portal Extension is available at:
> > https://github.com/okfn/ckanext-ecportal
> > branch: master
> > commit hash: e3b7f35df98baec1412fa584ec8227e9d404ad3f
> >
> > Note: As this is only a draft of the release notes, both of thesse
> > commit hashes may change.
> >
> >
> > Installation
> > ============
> >
> > * Install the new RPM.
> >
> > * Upgrade the database for CKAN 1.8.
> >
> > /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster --plugin=ckan db
> > upgrade -c /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
> >
> > This command should return:
> > <date> INFO  [ckan.model] CKAN database version upgraded: 57 -> 59
> >
> > * Add the database tables for the new search cloud feature.
> >
> > /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster
> > --plugin=ckanext-ecportal ecportal searchcloud-install-tables -c
> > /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
> >
> > This command should return:
> > Creating the search_query table ...
> > done.
> > Creating the search_popular_latest table ...
> > done.
> > Creating the search_popular_approved table ...
> > done.
> > Creating the search_query_date index ...
> > done.
> >
> > * Import the new translations of the fixed vocabularies.
> >
> > /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster
> > --plugin=ckanext-ecportal ecportal import-csv-translations -c
> > /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
> >
> > This command does not return anything.
> >
> > * Make sure that Solr is using the latest schema.xml file from the
> ckanext
> > ecportal extension.
> > This is at:
> >
> /applications/ecodp/users/ecodp/ckan/ecportal/src/ckanext-ecportal/ckanext/ecportal/solr
> > This can be copied to the solr conf directory:
> >
> > cp
> >
> /applications/ecodp/users/ecodp/ckan/ecportal/src/ckanext-ecportal/ckanext/ecportal/solr/*
> > /applications/ecodp/users/ecodp/solr/solr/conf
> >
> > * Restart tomcat.
> >
> > * Restart Apache.
> >
> > * Rebuild the search index (may take a while to run).
> >
> > /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster --plugin=ckan
> > search-index -r rebuild -c
> > /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
> >
> >
> > CRON jobs
> > ---------
> >
> > * The following should be run to update the list of recently searched
> terms:
> >
> > /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster
> > --plugin=ckanext-ecportal ecportal
> > searchcloud-generate-unapproved-search-list -c
> > /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
> >
> > We recommend running this daily at midnight.
> >
> > * The following should be run to update the tracking information for the
> > previous day:
> >
> > /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster --plugin=ckan
> > tracking update -c /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
> >
> > We recommend running this daily at 1 minute past midnight.
> >
> >
> > Database cleanup
> > ----------------
> >
> > When investigating the issues with the rdf import failing, we noticed
> that
> > some database tables had grown quite large and can be cleaned up in this
> > release.
> >
> > 1. The package_extra_revision table contains data on a lot of old
> > revisions that are not currently needed. These can be removed with the
> > following
> > paster command:
> >
> > /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster
> > --plugin=ckanext-ecportal ecportal purge-package-extra-revision -c
> > /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
> >
> > This command should return:
> > <date> WARNI [root] {'revision_rows_deleted': <number of rows>}
> >
> > This table is in constant use, so this command could be scheduled to be
> run
> > regularly. It should not need to be run too frequently however, once
> every
> > week or two should be fine.
> >
> >
> > 2. Some unused CKAN extensions (qa and datastorer) were installed in
> > previous
> > releases. These have been removed in this release, and so the tables that
> > they were writing to can now be cleared. To do this, run:
> >
> > /applications/ecodp/users/ecodp/ckan/ecportal/bin/paster
> > --plugin=ckanext-ecportal ecportal purge-task-data -c
> > /applications/ecodp/users/ecodp/ckan/conf/ecportal.ini
> >
> > This command should return:
> > <date> WARNI [root] {'celery_rows_deleted': <number of rows>,
> > 'task_status_rows_deleted': <number of rows>}
> >
> > The tables cleared by this command should not be used at all with the
> code
> > in
> > this release, so this command will only have to be run once.
> >
> >
> > _______________________________________________
> > Ecodp-dev mailing list
> > Ecodp-dev at lists.okfn.org
> > http://lists.okfn.org/mailman/listinfo/ecodp-dev
> >
>
>
>
> --
> Bert Van Nuffelen
>
> Semantic Technologies Software Architect at TenForce
> www.tenforce.be
>
> Bert.Van.Nuffelen at tenforce.com
> Office: +32 (0)16 31 48 60
> Mobile:+32 479 06 24 26
> skype: bert.van.nuffelen
>
> _______________________________________________
> Ecodp-dev mailing list
> Ecodp-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130617/12fad69c/attachment.html>