[ckan-discuss] Various issues with CSW Harvesting

Thu Oct 17 16:32:57 BST 2013

As it seems, the GeoSpatial extension installs OWSlib 0.8.2, whereas 
pywcs 1.6.1 prefers an older version, not compatible with OWSlib and 
thereas with the csw_harvester.
But even with switching to pycsw master branch I'm not goin very far:

paster --plugin=ckanext-harvest harvester sources 
--config=/etc/ckan/default/production.ini
2013-10-17 17:29:33,819 DEBUG [ckanext.harvest.model] Harvest tables 
defined in memory
2013-10-17 17:29:33,852 DEBUG [ckanext.harvest.model] Harvest tables 
already exist
2013-10-17 17:29:34,074 DEBUG [ckanext.spatial.model.package_extent] 
Spatial tables defined in memory
2013-10-17 17:29:34,114 DEBUG [ckanext.spatial.model.package_extent] 
Spatial tables already exist

Source id: 45ef5816-f321-4979-a2df-119db51ef27f
       url: 
http://sdi.provincia.bz.it:8080/geonetwork/srv/de/csw?SERVICE=CSW&VERSION=2.0.2
      type: csw
    active: True
frequency: ALWAYS
      jobs: 1

There is 1 active harvest source

Then:

paster --plugin=ckanext-harvest harvester gather_consumer 
--config=/etc/ckan/default/production.ini &
paster --plugin=ckanext-harvest harvester fetch_consumer 
--config=/etc/ckan/default/production.ini &
paster --plugin=ckanext-harvest harvester run 
--config=/etc/ckan/default/production.ini

yields me a:

2013-10-17 17:30:03,286 INFO  [ckanext.harvest.logic.action.update] 
Harvest job run: {}
2013-10-17 17:30:03,299 INFO  [ckanext.harvest.logic.action.create] 
Harvest job create: {'source_id': u'45ef5816-f321-4979-a2df-119db51ef27f'}
2013-10-17 17:30:03,323 WARNI [ckanext.harvest.logic.action.create] 
There is already an unrun job True for this source 
45ef5816-f321-4979-a2df-119db51ef27f
2013-10-17 17:30:03,323 INFO  [ckanext.harvest.logic.action.update] 
Trying to rerun job for 45ef5816-f321-4979-a2df-119db51ef27f skipping
2013-10-17 17:30:03,511 DEBUG [ckanext.harvest.queue] Received harvest 
job id: c0cbb92e-ff59-45d4-9c74-0ab1a2e4762a
2013-10-17 17:30:03,511 INFO  [ckanext.harvest.logic.action.update] Sent 
job c0cbb92e-ff59-45d4-9c74-0ab1a2e4762a to the gather queue
2013-10-17 17:30:03,537 DEBUG 
[ckanext.spatial.harvesters.csw.CSW.gather] CswHarvester gather_stage 
for job: <HarvestJob id=c0cbb92e-ff59-45d4-9c74-0ab1a2e4762a 
created=2013-10-17 15:28:58.882935 gather_started=2013-10-17 
15:30:03.537460 gather_finished=None finished=None 
source_id=45ef5816-f321-4979-a2df-119db51ef27f status=Running>
2013-10-17 17:30:03,563 ERROR [ckanext.harvest.harvesters.base] Error 
contacting the CSW server: Needs an Implementation
2013-10-17 17:30:03,580 ERROR [ckanext.harvest.queue] Gather stage failed

What is it, that I'm screwing up?

Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-discuss/attachments/20131017/79914125/attachment.htm>