[ckan-dev] Automate DataStore -> File

[Henrik Aagaard Sørensen]

Hi Adrià

That is actually a good solution for now. Somehow I did not think of that, so thank you :)


-----Oprindelig meddelelse-----
Fra: ckan-dev [mailto:ckan-dev-bounces at lists.okfn.org] På vegne af Adrià Mercader
Sendt: 8. januar 2014 18:02
Til: CKAN Development Discussions
Emne: Re: [ckan-dev] Automate DataStore -> File

Hi Henrik,

I would suggest using an external script for this rather than CKAN if you are handling this volume of data. Postgres's command COPY should do the trick.
You can of course create a CKAN resource later on that points to your generated files.

Adrià


http://www.postgresql.org/docs/current/static/sql-copy.html

On 7 January 2014 08:51, Henrik Aagaard Sørensen <BU1G at tmf.kk.dk> wrote:
> As it is possible to automate CSV -> DataStore via the 
> DataStorer-extension, is it somehow possible to do it the other way around?
>
> I have realtime data being added to a DataStore, via the API. This 
> results in datasets with 5-10 mio. records every year.
>
> Downloading via /dump/ is not that good because of the size of the dataset.
>
>
>
> Is it possible to have CKAN automatically create/update a .csv-file 
> matching the DataStore for use when downloading the entire dataset?
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
_______________________________________________
ckan-dev mailing list
ckan-dev at lists.okfn.org
https://lists.okfn.org/mailman/listinfo/ckan-dev
Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev