[ckan-dev] delete a harvesting resource
Elena Camossi
elena.camossi at ext.jrc.ec.europa.eu
Tue Nov 12 15:50:56 UTC 2013
Hi everyone,
we have found out how to partially delete a harvest source.
While command
paster --plugin=ckanext-harvest harvester rmsource [harvest source
id]
just disactivates the harvest source, this still exists as dataset.
So harvest sources can be deleted using command
paster dataset purge [harvest source id]
which removes dataset from db. The "deleted" harvest source still exists
somewhere, because it is still visible when running
paster --plugin=ckanext-harvest harvester harvester sources
but at least it can be re-inserted from the web interface as a new source,
which wasn't the case after running just rmsource.
However, even if the new source is visible from http://myckansite/harvest,
and a new harvesting request can be inserted in the queue, when launching
harvester run we have again an error of Harvest source not found
(pyenv)[ckan at drdsi-data ckan]$ paster --plugin=ckanext-harvest harvester run
-c /etc/ckan/production.ini
Traceback (most recent call last):
File "/usr/local/ckan/pyenv/bin/paster", line 9, in <module>
load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
File
"/usr/local/ckan/pyenv/lib/python2.6/site-packages/paste/script/command.py",
line 104, in run
invoke(command, command_name, options, args[1:])
File
"/usr/local/ckan/pyenv/lib/python2.6/site-packages/paste/script/command.py",
line 143, in invoke
exit_code = runner.run(args)
File
"/usr/local/ckan/pyenv/lib/python2.6/site-packages/paste/script/command.py",
line 238, in run
result = self.command()
File
"/usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/commands/harveste
r.py",
line 113, in command
self.run_harvester()
File
"/usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/commands/harveste
r.py",
line 278, in run_harvester
jobs = get_action('harvest_jobs_run')(context,{})
File "/usr/local/ckan/pyenv/src/ckan/ckan/logic/__init__.py", line 329,
in wrapped
return _action(context, data_dict, **kw)
File
"/usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/logic/action/upda
te.py",
line 291, in harvest_jobs_run
jobs =
harvest_job_list(context,{'source_id':source_id,'status':u'Running'})
File "/usr/local/ckan/pyenv/src/ckan/ckan/logic/__init__.py", line 386,
in wrapper
return action(context, data_dict)
File
"/usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/logic/action/get.
py",
line 226, in harvest_job_list
check_access('harvest_job_list',context,data_dict)
File "/usr/local/ckan/pyenv/src/ckan/ckan/logic/__init__.py", line 207,
in check_access
logic_authorization = new_authz.is_authorized(action, context,
data_dict)
File "/usr/local/ckan/pyenv/src/ckan/ckan/new_authz.py", line 80, in
is_authorized
return auth_function(context, data_dict)
File
"/usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/logic/auth/get.py
",
line 85, in harvest_job_list
{'id': source_id})
File "/usr/local/ckan/pyenv/src/ckan/ckan/logic/__init__.py", line 207,
in check_access
logic_authorization = new_authz.is_authorized(action, context,
data_dict)
File "/usr/local/ckan/pyenv/src/ckan/ckan/new_authz.py", line 80, in
is_authorized
return auth_function(context, data_dict)
File
"/usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/logic/auth/update
.py",
line 19, in harvest_source_update
raise pt.ObjectNotFound(pt._('Harvest source not found'))
ckan.logic.NotFound: Harvest source not found (pyenv)[ckan at drdsi-data ckan]$
gedit /usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/
Does someone have any idea on what went wrong?
Thanks.
Regards,
-Elena
> -----Original Message-----
> From: ckan-dev-bounces at lists.okfn.org [mailto:ckan-dev-
> bounces at lists.okfn.org] On Behalf Of Hildegard Gerlach
> Sent: lunedì 11 novembre 2013 14:29
> To: CKAN Development Discussions
> Subject: [ckan-dev] delete a harvesting resource
>
> Hello,
>
> we have installed CKAN 2.1 on Redhat6 and enabled the datastore
> extension. This caused errors when trying to use the harvester.
>
> IntegrityError: (IntegrityError) duplicate key value violates unique
constraint
> "pg_type_typname_nsp_index", refere jrc.it/harvest [Mon Nov 11 13:58:14
> 2013] [error] [client 139.191.16.105] DETAIL: Key (typname,
> typnamespace)=(_foo, 2200) already exists., referer:
> http://drdsi-data.ies.jrc.it/harvest
> [Mon Nov 11 13:58:14 2013] [error] [client 139.191.16.105] 'CREATE TABLE
> _foo ()' {}, referer: http://drdsi-data.ies.jrc.it/harvest
> [Mon Nov 11 13:58:14 2013] [error] [client 139.191.16.105]
> cursor.execute(statement, parameters)
>
> I disabled the datastore again and have no harvesting sources visible in
the
> interface (hostname/harvest) but they are still there.
>
> (pyenv)[ckan at drdsi-data ckan]$ paster --plugin=ckanext-harvest harvester
> sources all -c /etc/ckan/production.ini
>
> Source id: 92ff0ade-0892-4d84-9e3b-ed8bd2612aff
> url:
> http://rdsi-portal.jrc.it:8080/geonetwork/srv/eng/csw-
> drdsi?request=GetCapabilities&service=CSW&version=2.0.2
> type: csw
> active: False
> frequency: MANUAL
> jobs: 0
>
> Source id: 58f12a89-1525-43d7-a822-cdc07d648101
> url:
> http://drdsi-
> portal.ies.jrc.it/GeoportalProxyWebServices/resources/OGCCSW202?service
> =CSW&version=2.0.2&request=GetCapabilities
> type: csw
> active: False
> frequency: MANUAL
> jobs: 0
>
> Source id: 4a0b4385-4483-4bc3-9a1c-c0d238b90808
> url:
> http://cat.ccss.cz/csw/?request=GetCapabilities&service=CSW&version=2.0.
> 2
> type: csw
> active: False
> frequency: MANUAL
> jobs: 0
>
> There are 3 harvest sources
>
> How can I delete them ? I tried with user ckan in the virtual environment
but
> the user ckan doesn't seem to be authorized.
>
> (pyenv)[ckan at drdsi-data ckan]$ paster --plugin=ckanext-harvest harvester
> rmsource 92ff0ade-0892-4d84-9e3b-ed8bd2612aff -c
> /etc/ckan/production.ini
>
> Traceback (most recent call last):
> File "/usr/local/ckan/pyenv/bin/paster", line 9, in <module>
> load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
> File
> "/usr/local/ckan/pyenv/lib/python2.6/site-
> packages/paste/script/command.py",
> line 104, in run
> invoke(command, command_name, options, args[1:])
> File
> "/usr/local/ckan/pyenv/lib/python2.6/site-
> packages/paste/script/command.py",
> line 143, in invoke
> exit_code = runner.run(args)
> File
> "/usr/local/ckan/pyenv/lib/python2.6/site-
> packages/paste/script/command.py",
> line 238, in run
> result = self.command()
> File
> "/usr/local/ckan/pyenv/src/ckanext-
> harvest/ckanext/harvest/commands/harvester.py",
> line 105, in command
> self.remove_harvest_source()
> File
> "/usr/local/ckan/pyenv/src/ckanext-
> harvest/ckanext/harvest/commands/harvester.py",
> line 239, in remove_harvest_source
> get_action('harvest_source_delete')(context,{'id':source_id})
> File "/usr/local/ckan/pyenv/src/ckan/ckan/logic/__init__.py", line 329,
in
> wrapped
> return _action(context, data_dict, **kw)
> File
> "/usr/local/ckan/pyenv/src/ckanext-
> harvest/ckanext/harvest/logic/action/delete.py",
> line 25, in harvest_source_delete
> p.toolkit.check_access('harvest_source_delete', context, data_dict)
> File "/usr/local/ckan/pyenv/src/ckan/ckan/logic/__init__.py", line 210,
in
> check_access
> raise NotAuthorized(msg)
> ckan.logic.NotAuthorized: User not authorized to delete harvest source
> 92ff0ade-0892-4d84-9e3b-ed8bd2612aff
>
> Thanks
>
> Hilde
>
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
More information about the ckan-dev
mailing list