[ckan-dev] Datastreaming possible?

[Ross Jones]

Hi Stephan,

If you find that the datastore can't handle the speed with which you will
be inserting data it might be worth checking out a time-series DB built for
the purpose and storing the other data (dataset etc) in CKAN. There's
probably no reason that the datastore can't process the data, but it is
unlikely to provide a lot of the 'slicing and dicing' that you'd want for
charting.

I've been looking at Seriesly (
http://dustin.github.com/2012/09/09/seriesly.html and
https://github.com/dustin/seriesly) recently which sounds like it might do
what you're after.  It's pretty fast and the API is reasonable, but would
probably mean either a new recline backend, or as Dominik suggests
self-built charting.

There are more details on the performance of seriesly at
http://dustin.github.com/2012/09/13/inside-seriesly.html and the figures
there are pretty impressive. I'm intending to use this for personal
projects integrated with CKAN at some point as I can think of a few
use-cases for this type of data.

Ross.



On Sun, Mar 24, 2013 at 3:46 PM, Stephan Opfer <opfer at vs.uni-kassel.de>wrote:

> Hi CKANer,
>
> I wonder if it is possible to use CKAN for the same purpose as
> thingspeak.com or cosm.com are made for.
>
> The basic idea is to collect data from several sensor nodes every second
> and integrate them into the datastorage of ckan. The data can contain
> temperature, sonar-based distances, accelerations (x,y,z).
>
> At least I saw, that it is possible to visualize data in a graph.
> Especially, it is possible to combine several data sources in one graph, in
> order to compare them. This, for example, is not possible with
> thingspeak.com.
>
> A crucial question is, how fast can new data be integrated automatically.
>
> Best Regards,
>   Stephan
> --
> Distributed Systems Research Group
> Stephan Opfer T. +49 561 804-6283 F. +49 561 804-6277
> Univ. Kassel, FB 16, Wilhelmshöher Allee 73, D-34121 Kassel
> WWW: http://www.vs.uni-kassel.de/**mitarbeiter/wiss-mitarbeiter/**
> stephan-opfer/<http://www.vs.uni-kassel.de/mitarbeiter/wiss-mitarbeiter/stephan-opfer/>
>
> ______________________________**_________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/**listinfo/ckan-dev<http://lists.okfn.org/mailman/listinfo/ckan-dev>
> Unsubscribe: http://lists.okfn.org/mailman/**options/ckan-dev<http://lists.okfn.org/mailman/options/ckan-dev>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20130326/10175341/attachment-0001.html>