[ckan-dev] Thoughts on realtime data

[Darwin Peltan]

Hi Lasse,

Very interesting to hear this. Would be great to hear more about ideas for
extending support for real time data. If you have time it would be great if
you could jot down your thoughts on the CKAN wiki so that others can
feedback.

Best,

Darwin


On 27 October 2013 12:55, Lasse Vestergaard <ibbernik at gmail.com> wrote:

> Hi all.
>
> We are actually working quite a lot with real time data in relation to
> CKAN... Actually, it's more or less the only thing we use it for at the
> moment.
>
> @Henrik: The municipality of Aarhus (Denmark) is already putting real time
> data on their platform - I did it. They are a little in front of you guys
> in Copenhagen (on that part at least) :-)
>
> We are also working with real time data in a project called Gatesense
> (where CKAN is the core). We have already had quite some luck in
> registering accelerometer data from smart phones (through html5), feeding
> the data to the ckan datastore, and extracting it to a web page (if any of
> you come to the IoT world forum in Barcelona you can see a running
> prototype - a simple game). At the moment we are experiencing a few seconds
> of delay, but that is sufficient in most cases. So we consider CKAN being
> usable for real time data (not in the hardcore milli second understanding,
> but more on the average end user perception), but not the optimal solution
> (ex. Xively is way better for that part, at the moment). We have a couple
> of ideas, on the drawing board, on how to optimise CKAN for having better
> real time performance. Unfortunately, we haven't time for developing these
> features right now. If anyone is interested, then let's try and do some
> collaboration on if, how and why real time data streaming (in and out)
> should be implemented.
>
> Best regards
>
> Lasse Vestergaard
>
>
> 2013/10/21 Mark Wainwright <mark.wainwright at okfn.org>
>
>> Hi Henrik,
>>
>> Others may have more informed advice, but some thoughts:
>>
>> 1. The Orbital project [1] did some work with an engineering project
>> with a dataset of sensor data, using the Datastore API to add rows at
>> particular time intervals.
>>
>> 2. This might be a better bet than adding a new CSV resource every 5
>> minutes, which will mean the URL for the data keeps changing.
>>
>> I don't think much work has been done on using CKAN with real-time
>> data (perhaps others will correct me) so it would be great if you
>> updated the list with your thoughts/findings.
>>
>> Regards,
>>
>> Mark
>>
>>
>> [1] orbital.blogs.lincoln.ac.uk
>>
>>
>>
>> On 16/10/2013, Henrik Aagaard Sørensen <BU1G at tmf.kk.dk> wrote:
>> > I'm trying to implement realtime data in CKAN from a SQL server.
>> >
>> > I've thought of various strategies, such as creating a CSV-file every 5
>> > minute or so and upload it to CKAN or perhaps using the DataStore to
>> add new
>> > records every 5 minutes.
>> >
>> > Does anyone else have experience with such "challenge" as every thought,
>> > idea etc. would be very welcomed.
>> >
>> >
>>
>>
>> --
>> Business development and user engagement manager
>> The Open Knowledge Foundation
>> Empowering through Open Knowledge
>> http://okfn.org/  |  @okfn  |  http://ckan.org  |  @CKANproject
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20131028/0f3d25a4/attachment-0001.html>