[okfn-discuss] Collaborative Development of Data

Mon Mar 26 18:34:06 UTC 2007

Apologies for the long lag in replying to this.

Erik Moeller wrote:
> On 2/19/07, Rufus Pollock <rufus.pollock at okfn.org> wrote:
>> Then it will include a lot of stuff including the examples you mention.
>> By the way what would happen if you allow editing by other means (e.g.
>> local modifications with some commit mechanism) in addition to/instead
>> of by the web.
> 
> I would define access through a web as a minimum requirement, all
> other means of access is optional (though desirable).

Interesting requirement. Obviously this excludes most source code 
systems as while providing read access they don't provide write access 
through the web.

>> Similarly, musicbrainz, which definitely has some kind of versioning and
>> a web interface,  would also be a wiki.
> 
> Open to changes & reasonably easy? Fully reversible? User-visible
> changelogging? I don't know -- I'm only superficially familiar with
> their system.

I think they have most of these. See for example:

http://musicbrainz.org/doc/EditType

you can also get an idea from:

http://bugs.musicbrainz.org/browser/mb_server/trunk/admin/sql/CreateTables.sql

>> Of course this is no bad thing -- but it would seem to be a very broad
>> definition and it might be clearer just to say: this is a system which
>> supports versioning and liberal write access.
> 
> The term is useful if one applies the definition rigorously; access
> restrictions have a habit of creeping into even the most open systems
> (there are alternatives, such as reducing the impact of writes by
> delaying version pointer changes or reviewing changes before
> committing them to a "branch"). For example: Wikipedia still has
> traditional page protection, and images only recently became
> undeletable. The drive to make that changes, and the resistance to
> requiring logins, were both closely associated with "wiki culture."

I now get a clearer idea of what you mean and think this is a nice way 
of putting it -- it means that if I use a wiki system such as moinmoin 
of mediawiki to run a normal website (in which most pages are not world 
editable) I am *not* running a wiki. Similarly most systems be they for 
data, code or content are not 'wikis' because they have some form of 
access restrictions (in open source projects it is typical to have quite 
a restricted cadre of 'committers').

> I do expect that versioning systems, wikis and databases will
> converge; this, then, can lead to a fully read/write web, where code,
> data and content can all be maintained collaboratively, with
> fine-grained security models making way for socially learning trust
> systems, and traditional RDBMS gradually being replaced by
> self-optimizing generic data stores.

Wow! Quite a vision. I certainly think we will see convergence though 
perhaps I am a little more pessimistic about how rapidly we will be able 
to do away with authorization and permission systems and move to 
something that much more sophisticated as a generic trust system.

>> Out of interest can you make changes to multiple objects as part of a 
>> single
>> transaction
> 
> Yes. An import of 80,000 objects could be represented as a single
> transaction that can be rolled back as one (but also selectively).

That sound neat.

>> and how do you implement many to many relationships?
> 
> Since we are using an RDBMS, many-to-many relationships in a join
> table are simply versioned rows like any other, with a known lifespan
> determined by the associated transaction IDs for each table row.

Yes since asking the original question it became clear to me that this 
was the way one wanted to proceed.

> Feel free to play with the code:
> http://www.omegawiki.org/Development

I'd already taken a look around and have looked again since. I have to 
say I did find it hard to work out where to get started -- something you 
are aware of as you warn:

<quote>
Documentation sucks right now. Help is especially needed in this 
department! Contact Erik (erik AT wikimedia DOT org) and he'll give you 
a whirlwind tour through the main things you understand.
</quote>

I assume the best place to look is the db creation scripts:

<http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/Wikidata/Database%20scripts/Incremental/>

In the mean time I've been working to polish up the versioned domain 
model code I mentioned earlier and now have a fully functional 0.1 version:

http://p.knowledgeforge.net/ckan/svn/vdm/trunk/

~rufus