[ckan-dev] Automate DataStore -> File

Thu Jan 9 06:28:57 UTC 2014

I think having the new filestore in 2.2 is the first step towards that.
Definitely needs to be followed up for 2.3.

Nigel Babu

Developer  |  @nigelbabu <https://twitter.com/nigelbabu>

The Open Knowledge Foundation <http://okfn.org/>

Empowering through Open Knowledge

http://okfn.org/  |  @okfn <http://twitter.com/OKFN>  |  OKF on
Facebook<https://www.facebook.com/OKFNetwork> |
Blog <http://blog.okfn.org/>  |  Newsletter<http://okfn.org/about/newsletter>

 CKAN | http://ckan.org/ | @CKANproject
<http://twitter.com/CKANproject> |the world’s leading open-source data
portal platform

On 8 January 2014 22:43, Mark Wainwright <mark.wainwright at okfn.org> wrote:

> What's happened to the Filestore-Datastore Consolidation? Is it
> happening? Does it solve thi problem?
>
>
> https://github.com/okfn/ckan/wiki/Spec:-DataStore-and-FileStore-Consolidation
>
> Mark
>
>
> On 08/01/2014, Adrià Mercader <adria.mercader at okfn.org> wrote:
> > Hi Henrik,
> >
> > I would suggest using an external script for this rather than CKAN if
> > you are handling this volume of data. Postgres's command COPY should
> > do the trick.
> > You can of course create a CKAN resource later on that points to your
> > generated files.
> >
> > Adrià
> >
> >
> > http://www.postgresql.org/docs/current/static/sql-copy.html
> >
> > On 7 January 2014 08:51, Henrik Aagaard Sørensen <BU1G at tmf.kk.dk> wrote:
> >> As it is possible to automate CSV -> DataStore via the
> >> DataStorer-extension,
> >> is it somehow possible to do it the other way around?
> >>
> >> I have realtime data being added to a DataStore, via the API. This
> >> results
> >> in datasets with 5-10 mio. records every year.
> >>
> >> Downloading via /dump/ is not that good because of the size of the
> >> dataset.
> >>
> >>
> >>
> >> Is it possible to have CKAN automatically create/update a .csv-file
> >> matching
> >> the DataStore for use when downloading the entire dataset?
> >>
> >>
> >> _______________________________________________
> >> ckan-dev mailing list
> >> ckan-dev at lists.okfn.org
> >> https://lists.okfn.org/mailman/listinfo/ckan-dev
> >> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> >>
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > https://lists.okfn.org/mailman/listinfo/ckan-dev
> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> >
>
>
> --
> Business development and user engagement manager
> The Open Knowledge Foundation
> Empowering through Open Knowledge
> http://okfn.org/  |  @okfn  |  http://ckan.org  |  @CKANproject
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20140109/44687f8f/attachment-0003.html>