[ckan-dev] harvesting ... Error 404 ... returns incorrect source URL as part of error

Mon Nov 4 23:19:08 UTC 2013

I'm experimenting with the harvester and trying to harvest across this test data-set

data.gov.uk/api/2/rest/package/collections-database

I use the command line paster to create a job ... and then the sequence of gather_consumer, fetch_consumer and run.

It fails on the gathering part ... and as part of the error message it returns a URL with '/api/2/rest/package' appended a second time to the end of the correct URL. I'm trying to figure out what that is happening and if that is ultimately the source of my error.

Thanks,
Colum

2013-11-04 22:10:09,274 DEBUG [ckanext.harvest.queue] Received harvest job id: 1ccbb0dc-3642-4870-becd-d80bb3815b6b
2013-11-04 22:10:09,305 DEBUG [ckanext.harvest.harvesters.ckanharvester] In CKANHarvester gather_stage (http://data.gov.uk/api/2/rest/package/collections-database)
2013-11-04 22:10:10,016 ERROR [ckanext.harvest.harvesters.base] Unable to get content for URL: http://data.gov.uk/api/2/rest/package/collections-database/api/2/rest/package: HTTP Error 404: Not Found
2013-11-04 22:10:10,022 ERROR [ckanext.harvest.queue] Gather stage failed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20131104/e7fb26d3/attachment.html>