[ckan-dev] Automate DataStore -> File

Wed Jan 8 17:02:07 UTC 2014

Hi Henrik,

I would suggest using an external script for this rather than CKAN if
you are handling this volume of data. Postgres's command COPY should
do the trick.
You can of course create a CKAN resource later on that points to your
generated files.

Adrià

http://www.postgresql.org/docs/current/static/sql-copy.html

On 7 January 2014 08:51, Henrik Aagaard Sørensen <BU1G at tmf.kk.dk> wrote:
> As it is possible to automate CSV -> DataStore via the DataStorer-extension,
> is it somehow possible to do it the other way around?
>
> I have realtime data being added to a DataStore, via the API. This results
> in datasets with 5-10 mio. records every year.
>
> Downloading via /dump/ is not that good because of the size of the dataset.
>
>
>
> Is it possible to have CKAN automatically create/update a .csv-file matching
> the DataStore for use when downloading the entire dataset?
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>