[okfn-discuss] Distributed Storage: Suggestions?

Thu Apr 23 11:43:35 BST 2009

2009/4/22 Gael Varoquaux <gael.varoquaux at normalesup.org>:
> On Wed, Apr 22, 2009 at 12:27:44PM +0100, Rufus Pollock wrote:
>> Anyone out there have ideas or suggestions about distributed storage.
>> We have an ever growing need to store sizeable amounts of open
>> material in a replicated and distributed fashion.
>
> I here a lot about Tahoe:
> http://allmydata.org/~warner/pycon-tahoe.html

Thanks for the pointer. I'd played around with this a while ago when
it had just gone 1.0. My two concerns were:

1. It seemed heavily focused on encryption of stuff (because it was
intended for backing up private data). We don't need that at all.

2. (perhaps for similar reasons) The user/api interface was hard to
get a handle on (how did you browse what the system contained, how did
you manage permissions, how did you automatically locate chunks and
download).

Looking at their recent 1.4 release it looks like a lot of this has
been improved so this really might be worth taking a look again.
Anyone's experience setting up tahoe nodes, creating an interface etc
much appreciated.

Rufus

PS: I've created a wiki page to aggregate this kind of info:
<http://wiki.okfn.org/projects/Distributed_Storage/Research>