[ECODP-dev] PRODUCTION 09 upgrade
Dimitrios Mexis
dimitrios.mexis at tenforce.com
Thu Oct 10 10:39:44 UTC 2013
Hi John
good to know that you made it.
Can you check the paragraph of the document in google that refers to the
upgrade of 09 version, with what ever needs to be done in the end ?
Thanks
Dimitrios
On 10/10/2013 11:52, John Glover wrote:
> Hi Dimitrios,
>
> I have successfully ran through the namespace migration without any
> problems. I have reindexed the site and it is now back up.
>
> First however, I made sure to run the following commands (as
> documented previously in relation to the database size):
>
> paster --plugin=ckanext-ecportal ecportal purge-task-data -c <config>
> paster --plugin=ckanext-ecportal ecportal purge-package-extra-revision
> -c <config>
>
> I believe that the second command in particular is responsible for the
> speed up in the migration (for me it took just a couple of minutes).
> The reason is that we have to update the metadata for all datasets
> (and all of their revisions) in the database, and as no one had
> cleared out the old entries from the package_extra_revision table, it
> was performing individual update statements over about 2 million
> records (instead of the 60 thousand that are there now).
>
> Regards,
> John
>
>
> On 10 October 2013 10:12, Dimitrios Mexis
> <dimitrios.mexis at tenforce.com <mailto:dimitrios.mexis at tenforce.com>>
> wrote:
>
> Ok.
>
> Thanks.
> Dimitrios
>
>
> On 10/10/2013 10:08, John Glover wrote:
>> Ok thanks. Could you please not run any of the steps again for
>> now so I can have a look at the database?
>>
>> Thanks,
>> John
>>
>>
>> On 10 October 2013 10:06, Dimitrios Mexis
>> <dimitrios.mexis at tenforce.com
>> <mailto:dimitrios.mexis at tenforce.com>> wrote:
>>
>> John no ,
>> no I found it "killed"
>>
>> Regards
>> Dimitrios
>>
>>
>> On 10/10/2013 10:04, John Glover wrote:
>>> Hi Dimitrios,
>>>
>>> Did the namespace migration step finish the last time you
>>> ran it or did you kill it?
>>>
>>> Thanks,
>>> John
>>>
>>>
>>> On 10 October 2013 09:53, Dimitrios Mexis
>>> <dimitrios.mexis at tenforce.com
>>> <mailto:dimitrios.mexis at tenforce.com>> wrote:
>>>
>>> Hi Glen,
>>>
>>> Ok ight now I am feeding again the database with data
>>> (pg_restore ).
>>> The back up is under $USERHOME/ckan/backups called,
>>> PO-release09.tgz
>>>
>>> We will try to rename this publisher that caused issues
>>> before we start the procedure.
>>>
>>> regards
>>> Dimitrios
>>>
>>>
>>>
>>> On 10/10/2013 9:46, John Glover wrote:
>>>> Hi Dimitrios,
>>>>
>>>> Did you have to kill the migration step again? Can you
>>>> tell me the path to the database dump on the postgres
>>>> machine so I can try loading and running it myself a
>>>> few times?
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>>
>>>> On 10 October 2013 09:12, Dimitrios Mexis
>>>> <dimitrios.mexis at tenforce.com
>>>> <mailto:dimitrios.mexis at tenforce.com>> wrote:
>>>>
>>>> Good morning.
>>>>
>>>> Here can you explain the issue ? How do we de-block
>>>> it ?
>>>>
>>>>
>>>> [root at tffvirt-rh-template ~]# screen -x 25060
>>>> +++ unset pydoc
>>>> +++ '[' -n '' ']'
>>>> +++ '[' -n '' ']'
>>>> +++ '[' -n /bin/bash -o -n '' ']'
>>>> +++ hash -r
>>>> +++ '[' -n '' ']'
>>>> +++ unset VIRTUAL_ENV
>>>> +++ '[' '!' nondestructive = nondestructive ']'
>>>> +++
>>>> VIRTUAL_ENV=/applications/ecodp/users/ecodp/ckan/ecportal
>>>> +++ export VIRTUAL_ENV
>>>> +++
>>>> _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
>>>> +++
>>>> PATH=/applications/ecodp/users/ecodp/ckan/ecportal/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
>>>> +++ export PATH
>>>> +++ '[' -n '' ']'
>>>> +++ '[' -z '' ']'
>>>> +++ _OLD_VIRTUAL_PS1=
>>>> +++ '[' x '!=' x ']'
>>>> ++++ basename
>>>> /applications/ecodp/users/ecodp/ckan/ecportal
>>>> +++ '[' ecportal = __ ']'
>>>> ++++ basename
>>>> /applications/ecodp/users/ecodp/ckan/ecportal
>>>> +++ PS1='(ecportal)'
>>>> +++ export PS1
>>>> +++ alias 'pydoc=python -m pydoc'
>>>> +++ '[' -n /bin/bash -o -n '' ']'
>>>> +++ hash -r
>>>> ++ cd
>>>> /applications/ecodp/users/ecodp//ckan/ecportal/src/ckan
>>>> + set -x
>>>> + paster --plugin=ckan search-index rebuild -r -c
>>>> /applications/ecodp/users/ecodp//ckan/conf/ecportal.ini
>>>> 2013-10-09 12:40:58,527 INFO [ckan.lib.search]
>>>> Rebuilding search index...
>>>> 2013-10-09 12:40:58,946 INFO [ckan.lib.search]
>>>> Rebuilding the whole index...
>>>> 2013-10-09 12:59:56,800 INFO [ckan.lib.search]
>>>> Finished rebuilding search index.
>>>> 2013-10-09 12:59:59,174 INFO [ckan.lib.search]
>>>> Commited pending changes on the search index
>>>> + paster --plugin=ckanext-ecportal ecportal
>>>> migrate-odp-namespace -c
>>>> /applications/ecodp/users/ecodp//ckan/conf/ecportal.ini
>>>> ./ckan_upgrade_09to01.sh: line 21: 5031 Killed
>>>> paster --plugin=ckanext-ecportal ecportal
>>>> migrate-odp-namespace -c $CKAN_INI
>>>> + paster --plugin=ckanext-ecportal ecportal
>>>> update-all-vocabs -c
>>>> /applications/ecodp/users/ecodp//ckan/conf/ecportal.ini
>>>> + paster --plugin=ckanext-ecportal ecportal
>>>> update-publishers -c
>>>> /applications/ecodp/users/ecodp//ckan/conf/ecportal.ini
>>>> Traceback (most recent call last):
>>>> File
>>>> "/applications/ecodp/users/ecodp/ckan/ecportal/bin/paster",
>>>> line 9, in <module>
>>>> load_entry_point('PasteScript==1.7.5',
>>>> 'console_scripts', 'paster')()
>>>> File
>>>> "/applications/ecodp/users/ecodp/ckan/ecportal/lib/python2.6/site-packages/paste/script/command.py",
>>>> line 104, in run
>>>> invoke(command, command_name, options, args[1:])
>>>> File
>>>> "/applications/ecodp/users/ecodp/ckan/ecportal/lib/python2.6/site-packages/paste/script/command.py",
>>>> line 143, in invoke
>>>> exit_code = runner.run(args)
>>>> File
>>>> "/applications/ecodp/users/ecodp/ckan/ecportal/lib/python2.6/site-packages/paste/script/command.py",
>>>> line 238, in run
>>>> result = self.command()
>>>> File
>>>> "/applications/ecodp/users/ecodp/ckan/ecportal/src/ckanext-ecportal/ckanext/ecportal/commands.py",
>>>> line 109, in command
>>>> self.update_publishers(file_path)
>>>> File
>>>> "/applications/ecodp/users/ecodp/ckan/ecportal/src/ckanext-ecportal/ckanext/ecportal/commands.py",
>>>> line 527, in update_publishers
>>>> self._add_publishers(new_publishers)
>>>> File
>>>> "/applications/ecodp/users/ecodp/ckan/ecportal/src/ckanext-ecportal/ckanext/ecportal/commands.py",
>>>> line 213, in _add_publishers
>>>> plugins.toolkit.get_action('group_create')(context,
>>>> group)
>>>> File
>>>> "/applications/ecodp/users/ecodp/ckan/ecportal/src/ckan/ckan/logic/action/create.py",
>>>> line 538, in group_create
>>>> raise ValidationError(errors)
>>>> ckan.logic.ValidationError: {'Name': 'Group name
>>>> already exists in database'}
>>>> + paster --plugin=ckanext-ecportal ecportal
>>>> import-csv-translations -c
>>>> /applications/ecodp/users/ecodp//ckan/conf/ecportal.ini
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20131010/46eb2ca2/attachment.html>
More information about the ecodp-dev
mailing list