[ckan-dev] What's the future of DataPusher?

Tue Jan 30 10:09:01 UTC 2018

> Florian, have you taken a look at the x loader extension?
> 
> https://ckan.org/2017/11/20/express-loader-for-ckan-datastore/

I had, but somehow it seems I've forgotten about it. Thanks for the 
reminder, Steven! This does look exactly like what I had in mind, so I 
will give it a try.

Best regards
Florian

"ckan-dev" <ckan-dev-bounces at lists.okfn.org> schrieb am 30.01.2018 
11:02:17:

> Von: Steven De Costa <steven.decosta at linkdigital.com.au>
> An: CKAN Development Discussions <ckan-dev at lists.okfn.org>, 
> Datum: 30.01.2018 11:02
> Betreff: Re: [ckan-dev] What's the future of DataPusher?
> Gesendet von: "ckan-dev" <ckan-dev-bounces at lists.okfn.org>
> 
> Florian, have you taken a look at the x loader extension?
> 
> https://ckan.org/2017/11/20/express-loader-for-ckan-datastore/
> 
> On Tue, Jan 30, 2018 at 7:39 PM Matthew Fullerton 
<matt.fullerton at gmail.com
> > wrote:
> Hi Florian,
> 
> Yes, Python scripts :-) The data is being incrementally updated so 
> Datapusher doesn't fit. It is occasionally useful when a CSV file 
> arrives though.
> 
> Best,
> Matt
> 
> On 30 January 2018 at 08:54, <Florian.Brucker at it.karlsruhe.de> wrote:
> Hi Matthew, 
> 
> thanks for your input! 
> 
> > I have used Datastore a lot without the datapusher because it gives 
> > me a nice HTTP API over the data table entries. 
> 
> So I guess you have some (probably custom) software that you use to 
> upload data to the DataStore via its API in place of DataPusher? 
> 
> 
> Best regards 
> Florian 
> 
> 
> "ckan-dev" <ckan-dev-bounces at lists.okfn.org> schrieb am 30.01.2018 
08:32:09:
> 
> > Von: Matthew Fullerton <matt.fullerton at gmail.com> 
> > An: CKAN Development Discussions <ckan-dev at lists.okfn.org>, 
> > Datum: 30.01.2018 08:32 
> > Betreff: Re: [ckan-dev] What's the future of DataPusher? 
> > Gesendet von: "ckan-dev" <ckan-dev-bounces at lists.okfn.org> 
> > 
> > Hey Florian, 
> > 
> > I have used Datastore a lot without the datapusher because it gives 
> > me a nice HTTP API over the data table entries. 
> > 
> > Although the new feature of correcting column types via the data 
> > dictionary was in my opinion a major improvement, it's also a bit 
> > cumbersome and feels like a plaster. 
> > 
> > Best, 
> > Matt 
> > 
> > On 30 January 2018 at 08:28, <Florian.Brucker at it.karlsruhe.de> wrote: 
> > Hi everyone, 
> > 
> > after running into yet another problem* with DataPusher I was 
> > wondering if there are any plans to replace it with a more modern 
> > solution? In my experience, one reason why working with DataPusher 
> > is often cumbersome is its implementation as a separate service. As 
> > far as I understand, the same functionality could nowadays be 
> > implemented easily as a CKAN extension based on background jobs. In 
> > my opinion, this functionality should even be incorporated directly 
> > into the DataStore extension (honest question: are there many people
> > using DataStore without the DataPusher, and if so, why?). 
> > 
> > 
> > Best regards, 
> > Florian 
> > 
> > 
> > * after adding a resource with an uploaded CSV file to an existing 
> > dataset, DataPusher not only failed to push the data into DataStore 
> > automatically but also somehow managed to break the DataStore-
> > entries for the dataset's existing resources so that all of them had
> > to be re-uploaded to the DataStore "manually" by clicking the 
> buttonin the UI.
> > 
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > https://lists.okfn.org/mailman/listinfo/ckan-dev
> 
> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> 
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > https://lists.okfn.org/mailman/listinfo/ckan-dev
> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> 
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev

> 
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> -- 
> 
> STEVEN DE COSTA | EXECUTIVE DIRECTOR
> www.linkdigital.com.au
> 
> Want to be updated with news from Link Digital? Sign up to receive 
> our Newsletter!
>    [Bild entfernt] _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20180130/c67e1f4a/attachment-0003.html>