[okfn-discuss] [okfn-coord] question from the Twitter gallery about Open Data Grid

[Rufus Pollock]

On 3 October 2010 14:36, Jo Walsh <jo at frot.org> wrote:
> Rufus, this is all good to hear; i took coord out of the cc.
> As it clearly is of quite general interest!
>
>>> * mongodb + gridfs:
>>> <http://lists.okfn.org/pipermail/okfn-help/2010-June/000662.html>
>>> * riak<http://www.basho.com/Riak.html>  - could be used similar to
>>> mongodb
>>> * Eucalytus
>>> Walrus:<http://open.eucalyptus.com/wiki/EucalyptusStorage_v1.4>
>>>
>>> I think investigating these other alternatives would be a better use
>>> of people's time right now [than any more work on/with Tahoe].
>>
>>> <http://wiki.okfn.org/p/Distributed_Storage/>
>
> I still wonder if there are other similar looking efforts that OKF could
> plug into, that may be further along. I'd like to know (and put random
> people who might know, into the CC.)

Totally agree. I know there is biotorrents people (pinged them at one
point) and now defunct geotorrents. Many projects/tools are listed on:
<http://wiki.okfn.org/p/Distributed_Storage/Research> - though this
more tools to build grids with than grid efforts themselves.

> Experiments are fun and are why we are all here. But!
> There's a lot of money to be saved and benefit to be had in putting this
> together (e.g. at level of public administrations, mapping agencies, etc).

+1 and I'd love to know of a friendly institutions that would make
storage space available for open data ...

> And others are making business investments in this sort of thing (SimpleGeo
> come to mind). I wonder how much illumination can come via OKF from a few
> hours evaluation, vs a bit more focus on testing things that come from
> others, affiliations, etc.

Completely agree and that's why (a) it is essential we collaborate
with others here wherever possible (b) we've never sought to 'write'
anything like our own 'distributed storage' system but rather to build
on existing tools plugging them together tools from others into a
functioning 'service'.

Obviously one question to ask here is: given the existence of S3,
Google Storage, etc does one need to run one's own grid?

IMO I think there is at least some reason to do so, both from the open
software service (http://opendefinition.org/ossd/) end of things, and
from the practical point of view that storage nodes may be one thing
that it is easier to get contribution to and storage remains a
practical problem for the open data community.

Rufus
-- 
Open Knowledge Foundation
Promoting Open Knowledge in a Digital Age
http://www.okfn.org/ - http://blog.okfn.org/