[ckan-dev] crashing my CKAN database

Mon Jul 1 15:19:05 UTC 2013

Hello,

I have installed CKAN + harvest extension + spatial extension.

I had added some test datasets with paster create-test-data and also 
manually.

Then I ran this command to test the harvest extension. Is this wrong ?
nosetests --ckan --with-pylons=test-core.ini -l ckanext 
ckanext/harvest/tests

My database crashed. It deletes the postgis tables and also the contents 
in the other tables (users, datasets).

What am I doing wrong ? It happened to me before, probably when I ran 
this nosetests command for the spatial extension

https://github.com/okfn/ckanext-spatial#tests

I am getting mad reinstalling my database all the time. How can I test 
these extensions ?

See log below.

And a second question:
How do I set up a csw harvest request ?
http://localhost/harvest
A job had been created but it had never run.

output of the command   run in directory  pyenv/src/ckanext-harvest
nosetests --ckan --with-pylons=test-core.ini -l ckanext 
ckanext/harvest/tests

ckanext.harvest.model: DEBUG: Harvest tables defined in memory
ckanext.harvest.model: DEBUG: Harvest tables already exist
/usr/local/mtester.ies.jrc.it/pyenv/src/ckan/ckan/lib/app_globals.py:143: DeprecationWarning: 
The attribute 'config.debug' has mov                                ed 
to the pylons.config dictionary: Please access it via pylons.config['debug']
   set_main_css(main_css)
/usr/local/mtester.ies.jrc.it/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/reflection.py:47: 
SAWarning: Did not recognize t                                ype 
'geometry' of column 'the_geom'
   ret = fn(self, con, *args, **kw)
ckanext.harvest.logic.action.create: INFO: Creating harvest source: 
{u'name': u'test-source-action', u'title': u'Test source 
actio                                n', u'url': 
u'http://test.action.com', u'notes': u'Test source action desc', 
u'source_type': u'test-for-action', u'frequency': 
u'M                                ANUAL', u'config': 
u'{"custom_option": ["a", "b"]}'}
ckanext.harvest.logic.action.create: INFO: Creating harvest source: 
{u'name': u'test-source-action-new', u'title': u'Test source 
a                                ction', u'url': 
u'http://test.action.com', u'notes': u'Test source action desc', 
u'source_type': u'test-for-action', 
u'frequency':                                 u'MANUAL', u'config': 
u'{"custom_option": ["a", "b"]}'}
.ckanext.harvest.logic.action.create: INFO: Creating harvest source: {}
.ckanext.harvest.logic.action.create: INFO: Creating harvest source: 
{u'name': u'test-source-action', u'title': u'Test source 
acti                                on', u'url': 
u'http://test.action.com', u'notes': u'Test source action desc', 
u'source_type': u'test-for-action', u'frequency': 
u'                                ANNUALLY', u'config': 
u'{"custom_option": ["a", "b"]}'}
.ckanext.harvest.logic.action.create: INFO: Creating harvest source: 
{u'name': u'test-source-action', u'title': u'Test source 
acti                                on', u'url': 
u'http://test.action.com', u'notes': u'Test source action desc', 
u'source_type': u'unknown', u'frequency': 
u'MANUAL',                                 u'config': 
u'{"custom_option": ["a", "b"]}'}
.ckanext.harvest.logic.action.create: INFO: Creating harvest source: 
{u'name': u'test-source-action', u'title': u'Test source 
acti                                on', u'url': 
u'http://test.action.com', u'notes': u'Test source action desc', 
u'source_type': u'test-for-action', u'frequency': 
u'                                MANUAL', u'config': u'not_json'}
ckanext.harvest.logic.validators: ERROR: Wrong JSON provided in config, 
skipping
ckanext.harvest.logic.action.create: INFO: Creating harvest source: 
{u'name': u'test-source-action', u'title': u'Test source 
actio                                n', u'url': 
u'http://test.action.com', u'notes': u'Test source action desc', 
u'source_type': u'test-for-action', u'frequency': 
u'M                                ANUAL', u'config': 
u'{"custom_option": "not_a_list"}'}
.ckanext.harvest.logic.action.create: INFO: Creating harvest source: 
{u'name': u'test-source-action', u'title': u'Test source 
acti                                on', u'url': 
u'http://test.action.com', u'notes': u'Test source action desc', 
u'source_type': u'test-for-action', u'frequency': 
u'                                MANUAL', u'config': 
u'{"custom_option": ["a", "b"]}'}
ckanext.harvest.logic.action.update: INFO: Updating harvest source: 
{u'id': u'3b7e44fa-7621-4714-b09b-f1a6da6dbac0'}
.ckanext.harvest.logic.action.update: INFO: Updating harvest source: 
{u'name': u'test-source-action', u'title': u'Test source 
acti                                on', u'url': 
u'http://test.action.com', u'notes': u'Test source action desc', 
u'source_type': u'test-for-action', u'frequency': 
u'                                ANNUALLY', u'config': 
u'{"custom_option": ["a", "b"]}', u'id': 
u'3b7e44fa-7621-4714-b09b-f1a6da6dbac0'}
.ckanext.harvest.logic.action.update: INFO: Updating harvest source: 
{u'name': u'test-source-action', u'title': u'Test source 
acti                                on', u'url': 
u'http://test.action.com', u'notes': u'Test source action desc', 
u'source_type': u'unknown', u'frequency': 
u'MANUAL',                                 u'config': 
u'{"custom_option": ["a", "b"]}', u'id': 
u'3b7e44fa-7621-4714-b09b-f1a6da6dbac0'}
.ckanext.harvest.logic.action.update: INFO: Updating harvest source: 
{u'name': u'test-source-action', u'title': u'Test source 
acti                                on', u'url': 
u'http://test.action.com', u'notes': u'Test source action desc', 
u'source_type': u'test-for-action', u'frequency': 
u'                                MANUAL', u'config': u'not_json', 
u'id': u'3b7e44fa-7621-4714-b09b-f1a6da6dbac0'}
ckanext.harvest.logic.validators: ERROR: Wrong JSON provided in config, 
skipping
ckanext.harvest.logic.action.update: INFO: Updating harvest source: 
{u'name': u'test-source-action', u'title': u'Test source 
actio                                n', u'url': 
u'http://test.action.com', u'notes': u'Test source action desc', 
u'source_type': u'test-for-action', u'frequency': 
u'M                                ANUAL', u'config': 
u'{"custom_option": "not_a_list"}', u'id': 
u'3b7e44fa-7621-4714-b09b-f1a6da6dbac0'}
.ckanext.harvest.logic.action.update: INFO: Updating harvest source: 
{u'name': u'test-source-action-updated', u'title': u'Test 
sou                                rce action updated', u'url': 
u'http://test.action.updated.com', u'notes': u'Test source action desc 
updated', u'source_type': u'te st', u'frequency': u'MONTHLY', u'config': 
u'{"custom_option": ["c", "d"]}', u'id': 
u'3b7e44fa-7621-4714-b09b-f1a6da6dbac0'}
.SSE
======================================================================
ERROR: 
ckanext.harvest.tests.test_queue.TestHarvestQueue.test_01_basic_harvester
----------------------------------------------------------------------
Traceback (most recent call last):
   File 
"/usr/local/mtester.ies.jrc.it/pyenv/lib/python2.6/site-packages/nose/case.py", 
line 197, in runTest
     self.test(*self.arg)
   File 
"/usr/local/mtester.ies.jrc.it/pyenv/src/ckanext-harvest/ckanext/harvest/tests/test_queue.py", 
line 95, in test_01_basic_ha                                rvester
     consumer = queue.get_consumer('ckan.harvest.gather','harvest_job_id')
   File 
"/usr/local/mtester.ies.jrc.it/pyenv/src/ckanext-harvest/ckanext/harvest/queue.py", 
line 174, in get_consumer
     connection = get_connection()
   File 
"/usr/local/mtester.ies.jrc.it/pyenv/src/ckanext-harvest/ckanext/harvest/queue.py", 
line 39, in get_connection
     raise Exception('not a valid queue type %s' % backend)
Exception: not a valid queue type rabbitmq
-------------------- >> begin captured logging << --------------------
ckan.model: INFO: Database table data deleted
ckan.model: INFO: Database initialised
ckan.model: INFO: Database rebuilt
ckan.model: INFO: Database initialised
ckan.model: INFO: Database initialised
ckan.model: INFO: Database initialised
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 11 tests in 7.877s

FAILED (SKIP=2, errors=1)

Thanks

Hilde
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20130701/c1209fdb/attachment.html>