[ckan-dev] Remote harvesting

Thu Feb 2 12:33:59 UTC 2017

I don't think its possible with the configuration options, so you would
have to take the CKAN Harvester and make your own, slightly altered version

By the way, of course as Adrià pointed out I missed the point on Datapusher
being necessary. Big omission, sorry!

@Adrià: Is there any way to turn the "auto" Datapushing off? What I mean is
to still have the option under Manage to "Push" to datastore, but for that
not to happen automatically when a compatible file is uploaded?

Best,
Matt

On 31 January 2017 at 15:38, Andrea Stella <andrea.stella at yacme.com> wrote:

> Hi,
> @Adrià @Matthew: first of all thank you for your answers.
>
> I have some resources that I load through script (using datastore_upsert
> api) then the url in remote machine is of the type
> /datastore/dump/{resource-id}. On the instance that makes the harvest will
> remain the same url, but it will not load something because the data are
> not (and do not want) loaded in the locale datastore. Is there some
> directive that constructs the url as https://{remote-ckan-url}/datastore/dump/{resource-id},
> or should I change mandatorily the harvest code?
>
> Thanks,
> a.
>
>
> Il 31/01/2017 14:31, Adrià Mercader ha scritto:
>
> Hi,
>
> Just a small clarification on what Matthew said,
>
> On 30 January 2017 at 08:50, Matthew Fullerton <matt.fullerton at gmail.com>
> wrote:
>
>> And when you add resources, then compatible resources will be downloaded
>> and added to the datastore.
>
>
> The harvester per se won't download any remote DataStore resources. But if
> you happen to have the DataPusher enabled on the local instance (the one
> that is harvesting), when the CSV, Excel resources are created by the
> harvester the DataPusher will get triggered and data uploaded to your local
> DataStore.
>
> Adrià
>
>
>
>
> _______________________________________________
> ckan-dev mailing listckan-dev at lists.okfn.orghttps://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
> --
> Andrea Stella
> YACME S.r.l. - Gruppo MakeIT
> Via Majani 2 - 40122 Bologna
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20170202/4468c871/attachment-0002.html>