[ckan-discuss] Harvesting harvesters

Thu Oct 31 16:03:54 GMT 2013

What's the best way to import harvest sources from another instance of CKAN?

I see that it's easy to make an API call that filters for harvesters in
particular organizations, eg
http://catalog.data.gov/api/3/action/package_search?q=type:harvest%20AND%20organization:usgs-gov

but I also see that the stock CKAN harvester filters out harvest sources:

            if package_dict.get('type') == 'harvest':
                log.warn('Remote dataset is a harvest source, ignoring...')
                return True

https://github.com/okfn/ckanext-harvest/blob/master/ckanext/harvest/harvesters/ckanharvester.py#L248

Filtering out harvest sources on an import sounds like a necessary and
obvious default, but if I actually just wanted to import the harvest
sources would it make sense to create a copy of the main CKAN harvester
extension that instead filters only for harvesters rather than filters them
out?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-discuss/attachments/20131031/262c35b0/attachment.htm>