[ckan-dev] Thoughts on realtime data

[Lasse Vestergaard]

I will definitely do that when ever I get the time. In the meantime, I'm
also very keen on knowing if any of you guys have explicit experience on
this matter?

Regards


2013/10/28 Darwin Peltan <darwin.peltan at okfn.org>

> Hi Lasse,
>
> Very interesting to hear this. Would be great to hear more about ideas for
> extending support for real time data. If you have time it would be great if
> you could jot down your thoughts on the CKAN wiki so that others can
> feedback.
>
> Best,
>
> Darwin
>
>
>
> On 27 October 2013 12:55, Lasse Vestergaard <ibbernik at gmail.com> wrote:
>
>> Hi all.
>>
>> We are actually working quite a lot with real time data in relation to
>> CKAN... Actually, it's more or less the only thing we use it for at the
>> moment.
>>
>> @Henrik: The municipality of Aarhus (Denmark) is already putting real
>> time data on their platform - I did it. They are a little in front of you
>> guys in Copenhagen (on that part at least) :-)
>>
>> We are also working with real time data in a project called Gatesense
>> (where CKAN is the core). We have already had quite some luck in
>> registering accelerometer data from smart phones (through html5), feeding
>> the data to the ckan datastore, and extracting it to a web page (if any of
>> you come to the IoT world forum in Barcelona you can see a running
>> prototype - a simple game). At the moment we are experiencing a few seconds
>> of delay, but that is sufficient in most cases. So we consider CKAN being
>> usable for real time data (not in the hardcore milli second understanding,
>> but more on the average end user perception), but not the optimal solution
>> (ex. Xively is way better for that part, at the moment). We have a couple
>> of ideas, on the drawing board, on how to optimise CKAN for having better
>> real time performance. Unfortunately, we haven't time for developing these
>> features right now. If anyone is interested, then let's try and do some
>> collaboration on if, how and why real time data streaming (in and out)
>> should be implemented.
>>
>> Best regards
>>
>> Lasse Vestergaard
>>
>>
>> 2013/10/21 Mark Wainwright <mark.wainwright at okfn.org>
>>
>>> Hi Henrik,
>>>
>>> Others may have more informed advice, but some thoughts:
>>>
>>> 1. The Orbital project [1] did some work with an engineering project
>>> with a dataset of sensor data, using the Datastore API to add rows at
>>> particular time intervals.
>>>
>>> 2. This might be a better bet than adding a new CSV resource every 5
>>> minutes, which will mean the URL for the data keeps changing.
>>>
>>> I don't think much work has been done on using CKAN with real-time
>>> data (perhaps others will correct me) so it would be great if you
>>> updated the list with your thoughts/findings.
>>>
>>> Regards,
>>>
>>> Mark
>>>
>>>
>>> [1] orbital.blogs.lincoln.ac.uk
>>>
>>>
>>>
>>> On 16/10/2013, Henrik Aagaard Sørensen <BU1G at tmf.kk.dk> wrote:
>>> > I'm trying to implement realtime data in CKAN from a SQL server.
>>> >
>>> > I've thought of various strategies, such as creating a CSV-file every 5
>>> > minute or so and upload it to CKAN or perhaps using the DataStore to
>>> add new
>>> > records every 5 minutes.
>>> >
>>> > Does anyone else have experience with such "challenge" as every
>>> thought,
>>> > idea etc. would be very welcomed.
>>> >
>>> >
>>>
>>>
>>> --
>>> Business development and user engagement manager
>>> The Open Knowledge Foundation
>>> Empowering through Open Knowledge
>>> http://okfn.org/  |  @okfn  |  http://ckan.org  |  @CKANproject
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>>
>>
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>
>>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20131102/6ba80444/attachment.html>