[ckan-dev] Remote harvesting

[Matthew Fullerton]

Hi Andrea,
It depends on what harvester you use, but if you are using the standard
CKAN harvester (https://github.com/ckan/ckanext-harvest#the-ckan-harvester)
then it will import resources as a link (
https://github.com/ckan/ckanext-harvest/blob/master/ckanext/harvest/harvesters/ckanharvester.py#L536).
And when you add resources, then compatible resources will be downloaded
and added to the datastore. There was just another email on the list that
was asking about the repercussions of this (
https://lists.okfn.org/pipermail/ckan-dev/2017-January/010754.html).

My personal feeling is its better to set up a harvester to just link to the
original resources, or to add something that saves a text record of the
data for searching, but it depends on how much searchability and data
explorability (!) you want your CKAN to have over the harvested data.

Best,
Matt

--
Matt Fullerton
Software development & data science for open data and mobility
matt at crowdsalad.de
+49 89 7456 0495 / +49 176 5789 6501
http://www.crowdsalad.de



On 27 January 2017 at 10:02, Andrea Stella <andrea.stella at yacme.com> wrote:

> Hi all,
> I have a doubt: when I harvest from an harvest source, are the data stored
> in the datastore of the latter replicated in CKAN that makes the harvest or
> are they aimed only as an endpoint?
> Thanks in advance.
>
> Best regards,
> a.
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20170130/f34df2d3/attachment-0003.html>