[ckan-dev] Import datasets from remote CKAN with ckanext-harvest and Datastore

Wed Nov 22 17:56:34 UTC 2017

Thanks to all! I will try both options.

El mié., 22 nov. 2017 17:34, Steven De Costa <
steven.decosta at linkdigital.com.au> escribió:

> For the larger resources people should check out the express loader  :)
>
> https://ckan.org/2017/11/20/express-loader-for-ckan-datastore/
>
> It’s fresh!
>
> On Wed, Nov 22, 2017 at 10:48 AM Stefan Oderbolz <stefan.oderbolz at liip.ch>
> wrote:
>
>> Hi Rafa,
>>
>> we have the same use case and use DataPusher for this purpose (
>> http://docs.ckan.org/projects/datapusher/en/latest/).
>>
>> 1. We harvest the datasets using a ckanext-harvest from the source to the
>> target instance
>> 2. DataPusher is installed on the target CKAN and starts importing the
>> datasets to the DataStore
>>
>> Depending on the size of the ressources and the amount of ressources this
>> can lead to performance problems. We currently haven't figured out a stable
>> setup to replicate this for larger ressources (> 50MB), therefore we limit
>> datapusher to only import smaller ressources.
>>
>> - Stefan
>>
>> On Mon, Nov 20, 2017 at 12:16 PM, Rafa <rafsalrob at gmail.com> wrote:
>>
>>> Hey Everyone,
>>>
>>> I'm involved in a project in which I need to import datasets from a
>>> remote CKAN which uses Datastore for store the data. I have configured
>>> DataStore and Haverst extensions in the target CKAN but when I run the
>>> harvest job, the process only copy the resources metadata (title, url,
>>> name, format) and the datastore tables are not created. Is there any way to
>>> copy the files from the FileStore and the data from the DataStore (tables)?
>>>
>>> Cheers,
>>> Rafa.
>>>
>>> --
>>> --------------------------------------------------
>>> Rafael Salas Robledo
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org
>>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>>
>>>
>>
>>
>> --
>> Liip AG  // Limmatstrasse 183 //  CH-8005 Zürich
>> Tel +41 43 500 39 80 // GnuPG 0x7B588C67 // www.liip.ch
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>
> --
> Please note: I’m based in Berlin up to 27 November and my email response
> times will reflect that timezone
>
> *STEVEN DE COSTA *|
> *EXECUTIVE DIRECTOR*www.linkdigital.com.au
>
> Want to be updated with news from Link Digital? Sign up
> <http://linkdigital.com.au/content/newsletter?utm_source=email_signature&utm_medium=email&utm_campaign=newsletter_signup> to
> receive our Newsletter!
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20171122/55e8b69b/attachment-0003.html>