[ckan-dev] What's the future of DataPusher?

[Steven De Costa]

Cool :)

On Tue, Jan 30, 2018 at 9:09 PM <Florian.Brucker at it.karlsruhe.de> wrote:

> > Florian, have you taken a look at the x loader extension?
> >
> > https://ckan.org/2017/11/20/express-loader-for-ckan-datastore/
>
> I had, but somehow it seems I've forgotten about it. Thanks for the
> reminder, Steven! This does look exactly like what I had in mind, so I will
> give it a try.
>
>
>
> Best regards
> Florian
>
>
> "ckan-dev" <ckan-dev-bounces at lists.okfn.org> schrieb am 30.01.2018
> 11:02:17:
>
> > Von: Steven De Costa <steven.decosta at linkdigital.com.au>
> > An: CKAN Development Discussions <ckan-dev at lists.okfn.org>,
> > Datum: 30.01.2018 11:02
> > Betreff: Re: [ckan-dev] What's the future of DataPusher?
> > Gesendet von: "ckan-dev" <ckan-dev-bounces at lists.okfn.org>
> >
> > Florian, have you taken a look at the x loader extension?
> >
> > https://ckan.org/2017/11/20/express-loader-for-ckan-datastore/
> >
> > On Tue, Jan 30, 2018 at 7:39 PM Matthew Fullerton <
> matt.fullerton at gmail.com
> > > wrote:
> > Hi Florian,
> >
> > Yes, Python scripts :-) The data is being incrementally updated so
> > Datapusher doesn't fit. It is occasionally useful when a CSV file
> > arrives though.
> >
> > Best,
> > Matt
> >
> > On 30 January 2018 at 08:54, <Florian.Brucker at it.karlsruhe.de> wrote:
> > Hi Matthew,
> >
> > thanks for your input!
> >
> > > I have used Datastore a lot without the datapusher because it gives
> > > me a nice HTTP API over the data table entries.
> >
> > So I guess you have some (probably custom) software that you use to
> > upload data to the DataStore via its API in place of DataPusher?
> >
> >
> > Best regards
> > Florian
> >
> >
> > "ckan-dev" <ckan-dev-bounces at lists.okfn.org> schrieb am 30.01.2018
> 08:32:09:
> >
> > > Von: Matthew Fullerton <matt.fullerton at gmail.com>
> > > An: CKAN Development Discussions <ckan-dev at lists.okfn.org>,
> > > Datum: 30.01.2018 08:32
> > > Betreff: Re: [ckan-dev] What's the future of DataPusher?
> > > Gesendet von: "ckan-dev" <ckan-dev-bounces at lists.okfn.org>
> > >
> > > Hey Florian,
> > >
> > > I have used Datastore a lot without the datapusher because it gives
> > > me a nice HTTP API over the data table entries.
> > >
> > > Although the new feature of correcting column types via the data
> > > dictionary was in my opinion a major improvement, it's also a bit
> > > cumbersome and feels like a plaster.
> > >
> > > Best,
> > > Matt
> > >
> > > On 30 January 2018 at 08:28, <Florian.Brucker at it.karlsruhe.de> wrote:
> > > Hi everyone,
> > >
> > > after running into yet another problem* with DataPusher I was
> > > wondering if there are any plans to replace it with a more modern
> > > solution? In my experience, one reason why working with DataPusher
> > > is often cumbersome is its implementation as a separate service. As
> > > far as I understand, the same functionality could nowadays be
> > > implemented easily as a CKAN extension based on background jobs. In
> > > my opinion, this functionality should even be incorporated directly
> > > into the DataStore extension (honest question: are there many people
> > > using DataStore without the DataPusher, and if so, why?).
> > >
> > >
> > > Best regards,
> > > Florian
> > >
> > >
> > > * after adding a resource with an uploaded CSV file to an existing
> > > dataset, DataPusher not only failed to push the data into DataStore
> > > automatically but also somehow managed to break the DataStore-
> > > entries for the dataset's existing resources so that all of them had
> > > to be re-uploaded to the DataStore "manually" by clicking the
> > buttonin the UI.
> > >
> > > _______________________________________________
> > > ckan-dev mailing list
> > > ckan-dev at lists.okfn.org
> > > https://lists.okfn.org/mailman/listinfo/ckan-dev
> >
> > > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> >
> > > _______________________________________________
> > > ckan-dev mailing list
> > > ckan-dev at lists.okfn.org
> > > https://lists.okfn.org/mailman/listinfo/ckan-dev
> > > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> >
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > https://lists.okfn.org/mailman/listinfo/ckan-dev
> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
> >
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > https://lists.okfn.org/mailman/listinfo/ckan-dev
> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> > --
> >
> > STEVEN DE COSTA | EXECUTIVE DIRECTOR
> > www.linkdigital.com.au
> >
> > Want to be updated with news from Link Digital? Sign up to receive
> > our Newsletter!
> >    [Bild entfernt] _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > https://lists.okfn.org/mailman/listinfo/ckan-dev
> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
-- 


*STEVEN DE COSTA *|
*EXECUTIVE DIRECTOR*www.linkdigital.com.au

Want to be updated with news from Link Digital? Sign up
<http://linkdigital.com.au/content/newsletter?utm_source=email_signature&utm_medium=email&utm_campaign=newsletter_signup>
to
receive our Newsletter!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20180130/e8372e7b/attachment-0003.html>