[ckan-discuss] Where do city and other governments store their CKAN data?

Thu Mar 20 16:57:19 UTC 2014

Thanks everyone who provided comments on how data storage is currently
managed using CKAN within government agencies. I have summarised some of
the feedback in the article on FI-WARE and have thanked you all for your
help with background material. Please let me know if you require changes to
the mentions of data storage as i have currently described it (in the final
third of this article):

http://blog.programmableweb.com/2014/03/20/fi-ware-opens-up-the-iot-future-to-everyone/

As far as the partnership with CKAN goes, i think the FI-WARE team might
have been discussing this with CKAN before the recent association
announcement. From what i gather, some stakeholders from each are
discussing how to promote CKAN's open source software within FI-WARE's
catalog, and how to integrate data published on CKAN instances with
FI-WARE's open APIs to enable the data to be fed into third-party developer
applications.

Thanks again, and please feel free to reach out if you have news to share
with ProgrammableWeb's API developer audience.
m.

Mark Boyd
mark at mgboyd.com

Web: *http://www.programmableweb.com/profile/MarkBoyd
<http://www.programmableweb.com/profile/MarkBoyd>*
Twitter: @mgboydcom
Skype: mark.boyd.es
Google+: https://plus.google.com/100669793976591887741/posts/p/pub
LinkedIn: http://es.linkedin.com/pub/mark-boyd/36/869/283
Phone: +34 650 527 143

On Thu, Mar 20, 2014 at 10:46 AM, Stefan Oderbolz
<stefan.oderbolz at liip.ch>wrote:

> Hi Mark,
>
> for the Swiss Open Data Portal (http://opendata.admin.ch), there are two
> answers to your question, depending on the government agency:
>
> 1. Store all primary data in their existing environment. This is
> especially true for agencies that are already used to store huge amounts of
> data. They only deliver metadata for the portal, i.e. links to files or
> APIs in their infrastracture
> 2. Store all primary data on Amazon S3. It's very easy to store data there
> (either they upload it manually or a harvester reads it from an internal
> system and uploads it to S3 for them).
>
> But we also heard from customers that they are obliged to use their own
> infrastracture (no matter what).
>
> Regards Stefan
>
>
> On Thu, Mar 20, 2014 at 9:43 AM, Antoine Logean <
> antoine.logean at opendata.ch> wrote:
>
>> Hi Mark,
>>
>> Very good point. I also would be very interested to know better what are
>> the best alternatives to cover this operativ part of CKAN. In the company
>> (~10000 employees) where I work I have startet a pilot where we would like
>> to evaluate how we can use CKAN as intern data catalog. For us is it clear
>> that CKAN will ONLY contained meta data and so should remain a catalog. The
>> effectiv storage (with all the associated non functional requirements) of
>> the data is clearly done by other dedicated plateforms. For the moment our
>> CKAN instance is running in an intern cloud plateform on a virtual Linux
>> machine.
>>
>> Look forward to the others answers.
>>
>> Kind regards
>>
>> Antoine
>>
>> Le mercredi 19 mars 2014, Mark Boyd <mark at mgboyd.com> a écrit :
>>
>> Hi all
>>>
>>> I hope this is alright to ask this here in the CKAN email discussion
>>> list.
>>>
>>> I am a follower of CKAN and have written about the platform several
>>> times for ProgrammableWeb. I am currently working on an article about the
>>> new FI-Ware open source platform that i understand will partner with CKAN
>>> in the near future. FI-Ware is, in part, aiming to set up data centers to
>>> provide governments (and others) with independent distributed data storage
>>> solutions for their smart cities projects, and for storage of their open
>>> data platforms.
>>>
>>> This leads me to ask existing CKAN users: *How do you manage your CKAN
>>> instances and open data storage now? *Are you using cloud storage
>>> (Amazon?) or internal servers? If open data is to grow and at a (smart-)
>>> city level is to include sensor data in realtime, what storage needs will
>>> you have and what is the current thinking to solve this challenge?
>>>
>>> Again, i apologise if this is inappropriate to ask on this list. Feel
>>> free to email me directly if you prefer.
>>>
>>> Thanks for your help and for making CKAN such a great public resource.
>>>
>>> Mark Boyd
>>> mark at mgboyd.com
>>>
>>> Web: *http://www.programmableweb.com/profile/MarkBoyd
>>> <http://www.programmableweb.com/profile/MarkBoyd>*
>>> Twitter: @mgboydcom
>>> Skype: mark.boyd.es
>>> Google+: https://plus.google.com/100669793976591887741/posts/p/pub
>>> LinkedIn: http://es.linkedin.com/pub/mark-boyd/36/869/283
>>> Phone: +34 650 527 143
>>>
>>
>>
>> --
>>
>> Opendata.ch - Enabling Open Government Data in Switzerland
>> Antoine Logean | Founding Board Member | Community & Communication FR +41 79 3518482 | antoine.logean at opendata.ch | http://twitter.com/ecolix
>>
>>
>>
>> _______________________________________________
>> ckan-discuss mailing list
>> ckan-discuss at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/ckan-discuss
>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-discuss
>>
>>
>
>
> --
> Liip AG  //  Feldstrasse 133 //  CH-8004 Zurich
> Tel +41 43 500 39 80 // GnuPG 0x7B588C67 // www.liip.ch
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/pipermail/ckan-discuss/attachments/20140320/60f20353/attachment.html>