[ckan-discuss] Cloud services for Open Data

[Tim McNamara]

On 28 October 2010 02:57, Neil McEvoy <neil.mcevoy at cloudventures.biz> wrote:

>
> Hey folks
>
> I'm new to this list. By way of introduction I'd like to become more
> involved in the infrastructure piece, in particular I specialize in Cloud
> Computing and would like to help Cloud providers offer the ideal hosting
> environments for Open Data.
>

I don't know if it's possible to have a single set of requirements. I don't
think that the cloud offers much value in this area. Scalability (multi-user
concurrency & tall usage spikes) are far less likely to be an issue.
Governments also re vary wary of the legal considerations of having "their
data" housed in foreign countries, for example.

I can see two main mechanisms to provide data: 1) data dumps, 2) API access.
It's harder to talk about the needs of API access, because requirements will
vary.

Here are some high-level commonalities:

 - optimise for read access
 - throughput is more important than latency

Government departments are also generally in favour of:

 - metrics
 - integrity of data

I'd like to work within a team who can define these requirements and I can
> supply them to the hosting providers to achieve this.
>

Are you able to elaborate on this? Are you intending to create a commercial
service for governments to provide high availability of their data? Asking
for free advice sounds like free riding to me.

I would like to see a system whereby demanders of data are able to
contribute to the production of that data for the next demander, e.g. the
torrent model. This wouldn't work for API access, but would provide a very
cheap, secure and scalable technical solution for governments handling
terrabytes of data.


> For example I've recently been working with a start-up who has a
> Cloud-based distributed storage technology, and I think I saw a
> requirement for this.
>
> Kind regards, Neil.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-discuss/attachments/20101028/b13a6ed9/attachment.htm>