[ckan-dev] Import datasets from remote CKAN with ckanext-harvest and Datastore

Wed Nov 22 09:48:15 UTC 2017

Hi Rafa,

we have the same use case and use DataPusher for this purpose (
http://docs.ckan.org/projects/datapusher/en/latest/).

1. We harvest the datasets using a ckanext-harvest from the source to the
target instance
2. DataPusher is installed on the target CKAN and starts importing the
datasets to the DataStore

Depending on the size of the ressources and the amount of ressources this
can lead to performance problems. We currently haven't figured out a stable
setup to replicate this for larger ressources (> 50MB), therefore we limit
datapusher to only import smaller ressources.

- Stefan

On Mon, Nov 20, 2017 at 12:16 PM, Rafa <rafsalrob at gmail.com> wrote:

> Hey Everyone,
>
> I'm involved in a project in which I need to import datasets from a remote
> CKAN which uses Datastore for store the data. I have configured DataStore
> and Haverst extensions in the target CKAN but when I run the harvest job,
> the process only copy the resources metadata (title, url, name, format) and
> the datastore tables are not created. Is there any way to copy the files
> from the FileStore and the data from the DataStore (tables)?
>
> Cheers,
> Rafa.
>
> --
> --------------------------------------------------
> Rafael Salas Robledo
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>

-- 
Liip AG  // Limmatstrasse 183 //  CH-8005 Zürich
Tel +41 43 500 39 80 // GnuPG 0x7B588C67 // www.liip.ch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20171122/95a04f73/attachment-0003.html>