[ckan-dev] Automate DataStore -> File

Wed Jan 8 17:13:45 UTC 2014

What's happened to the Filestore-Datastore Consolidation? Is it
happening? Does it solve thi problem?

https://github.com/okfn/ckan/wiki/Spec:-DataStore-and-FileStore-Consolidation

Mark

On 08/01/2014, Adrià Mercader <adria.mercader at okfn.org> wrote:
> Hi Henrik,
>
> I would suggest using an external script for this rather than CKAN if
> you are handling this volume of data. Postgres's command COPY should
> do the trick.
> You can of course create a CKAN resource later on that points to your
> generated files.
>
> Adrià
>
>
> http://www.postgresql.org/docs/current/static/sql-copy.html
>
> On 7 January 2014 08:51, Henrik Aagaard Sørensen <BU1G at tmf.kk.dk> wrote:
>> As it is possible to automate CSV -> DataStore via the
>> DataStorer-extension,
>> is it somehow possible to do it the other way around?
>>
>> I have realtime data being added to a DataStore, via the API. This
>> results
>> in datasets with 5-10 mio. records every year.
>>
>> Downloading via /dump/ is not that good because of the size of the
>> dataset.
>>
>>
>>
>> Is it possible to have CKAN automatically create/update a .csv-file
>> matching
>> the DataStore for use when downloading the entire dataset?
>>
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>

-- 
Business development and user engagement manager
The Open Knowledge Foundation
Empowering through Open Knowledge
http://okfn.org/  |  @okfn  |  http://ckan.org  |  @CKANproject