[ckan-dev] What's the future of DataPusher?

Tue Jan 30 10:02:17 UTC 2018

Florian, have you taken a look at the x loader extension?

https://ckan.org/2017/11/20/express-loader-for-ckan-datastore/

On Tue, Jan 30, 2018 at 7:39 PM Matthew Fullerton <matt.fullerton at gmail.com>
wrote:

> Hi Florian,
>
> Yes, Python scripts :-) The data is being incrementally updated so
> Datapusher doesn't fit. It is occasionally useful when a CSV file arrives
> though.
>
> Best,
> Matt
>
> On 30 January 2018 at 08:54, <Florian.Brucker at it.karlsruhe.de> wrote:
>
>> Hi Matthew,
>>
>> thanks for your input!
>>
>> > I have used Datastore a lot without the datapusher because it gives
>> > me a nice HTTP API over the data table entries.
>>
>> So I guess you have some (probably custom) software that you use to
>> upload data to the DataStore via its API in place of DataPusher?
>>
>>
>> Best regards
>> Florian
>>
>>
>> "ckan-dev" <ckan-dev-bounces at lists.okfn.org> schrieb am 30.01.2018
>> 08:32:09:
>>
>> > Von: Matthew Fullerton <matt.fullerton at gmail.com>
>> > An: CKAN Development Discussions <ckan-dev at lists.okfn.org>,
>> > Datum: 30.01.2018 08:32
>> > Betreff: Re: [ckan-dev] What's the future of DataPusher?
>> > Gesendet von: "ckan-dev" <ckan-dev-bounces at lists.okfn.org>
>> >
>> > Hey Florian,
>> >
>> > I have used Datastore a lot without the datapusher because it gives
>> > me a nice HTTP API over the data table entries.
>> >
>> > Although the new feature of correcting column types via the data
>> > dictionary was in my opinion a major improvement, it's also a bit
>> > cumbersome and feels like a plaster.
>> >
>> > Best,
>> > Matt
>> >
>> > On 30 January 2018 at 08:28, <Florian.Brucker at it.karlsruhe.de> wrote:
>> > Hi everyone,
>> >
>> > after running into yet another problem* with DataPusher I was
>> > wondering if there are any plans to replace it with a more modern
>> > solution? In my experience, one reason why working with DataPusher
>> > is often cumbersome is its implementation as a separate service. As
>> > far as I understand, the same functionality could nowadays be
>> > implemented easily as a CKAN extension based on background jobs. In
>> > my opinion, this functionality should even be incorporated directly
>> > into the DataStore extension (honest question: are there many people
>> > using DataStore without the DataPusher, and if so, why?).
>> >
>> >
>> > Best regards,
>> > Florian
>> >
>> >
>> > * after adding a resource with an uploaded CSV file to an existing
>> > dataset, DataPusher not only failed to push the data into DataStore
>> > automatically but also somehow managed to break the DataStore-
>> > entries for the dataset's existing resources so that all of them had
>> > to be re-uploaded to the DataStore "manually" by clicking the buttonin
>> the UI.
>> >
>> > _______________________________________________
>> > ckan-dev mailing list
>> > ckan-dev at lists.okfn.org
>> > https://lists.okfn.org/mailman/listinfo/ckan-dev
>>
>> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>
>> > _______________________________________________
>> > ckan-dev mailing list
>> > ckan-dev at lists.okfn.org
>> > https://lists.okfn.org/mailman/listinfo/ckan-dev
>> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>
>>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
-- 

*STEVEN DE COSTA *|
*EXECUTIVE DIRECTOR*www.linkdigital.com.au

Want to be updated with news from Link Digital? Sign up
<http://linkdigital.com.au/content/newsletter?utm_source=email_signature&utm_medium=email&utm_campaign=newsletter_signup>
to
receive our Newsletter!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20180130/efafeb9e/attachment-0003.html>