[okfn-help] vdm versus Clio

[Rufus Pollock]

On 3 August 2010 18:35, Martijn Faassen <faassen at startifact.com> wrote:
> Hi there,

Hi, and thanks for writing!

> [I previously mailed this to the info@ address. Jonathan Gray wrote
> back I'd probably get more feedback here on okfn-help, so here I am]
>
> I've been looking at the vdm library to see whether it would be
> applicable to my projects. I created a library called Clio:
>
> http://packages.python.org/clio/
>
> This tries to do some of the same things as vdm does, except that I
> think vdm probably uses better architecture. So I'm considering
> whether I should dump Clio and shift to vdm for my projects.
>
> In the use cases for vdm, I'm particularly interested in use case 9
> listed, support for pending updates. This is what Clio's been designed
> to do. Recovering historical revisions is interesting too, especially
> the auditing aspect of it, but the primary use case is to let someone
> edit a database (through a web UI typically), adding, removing and
> editing records, and then this person, or someone else, can then later
> publish these changes, so that they become available for the consumers
> of this database.

Yes, this is something we thought about too.

> From the use cases for vdm however I fear that this hasn't been
> implemented yet, do I read this correctly? I see a few references to a
> PENDING state in the code, but not a lot. If it isn't there, how hard
> would it be to add this functionality? What is vdm being used for

We changed in last release to allow for 'state' to be unicodetext. One
reason for doing this was to easily allow for the addition of new
'states', most obviously 'pending'.

Obviously just adding the 'pending' state does not automatically get
it integrated into the structure -- i.e. make stuff 'pending-aware'.

At the moment, in CKAN, the biggest project of ours using vdm, we
explicitly filter on state=active in queries. We like this
explicitness but it would be truly trivial to have a new method like:
"filter_by_active" added to 'revisioned' objects which would
automatically add the relevant restriction to a given query.

There are clearly some slightly tricky issues here: what happens if
you have further changes on top of pending changes? Are they
automatically marked as pending, what happens if someone approves
later pending changes before previous pending changes etc?

> primarily right now?

We use vdm in several of our own projects, most significantly in CKAN:
<http://knowledgeforge.net/ckan/> which v1.0 and is used heavily in
production environments. It is probably being used elsewhere --
various people have emailed with questions about it over the last 2
years!

I would say that vdm is pretty powerful, in particular it handles
versioning of m2m and m21 relationships pretty well, has built in
diffing, tools for purging revisions (i.e. deleting a given revision
and revert changes), and has been *heavily* field-tested -- it's the
bugs that always cost the time :)

Looking at clio it looks like vdm would give you much of the basic
structure and would provide archiving for you. Pending at a simple
state could be added easily and then one could add specific domain
logic around pending in a separate plugin module (some people make not
require pending ...).

> In general I'm a bit at a loss to figure out what vdm's API is and how
> it is to be used. I've tried to read the code, but any hints would be
> appreciated.

That's a clear indication docs could be better! I'll try and pull
something together quickly today and upload it to packages.python.org
as a starting point.

> If vdm turns out to be useful to my purposes, I'd be happy to help
> contribute to it, including improved documentation, which I think it
> needs. :)

Agreed. We clearly need some sphinx docs and would *really* welcome
your contribution if you were interested ...

Rufus