[okfn-help] vdm: pending state

[Rufus Pollock]

On 11 October 2010 15:59, Martijn Faassen <faassen at startifact.com> wrote:
> Hi there,
>
> On Mon, Oct 4, 2010 at 6:25 PM, Rufus Pollock <rufus.pollock at okfn.org> wrote:
>> Apologies for the delay Martijn!
>
> I'm late with responding too, but expect to spend some more time on
> this the coming weeks..
>
>>> * certain can publish changes (add/update/removals made)
>>
>> I assume you mean 'certain people'
>
> Yes.
>
>>> I understand vdm maintains the database in the normal tables, which
>>> you'd access as if the content wasn't
>>> revisioned. It then retains information about this and other revisions
>>> in the revision tables.
>>
>> Right, but also adds a 'state' field and revision_id field to the
>> 'normal' table.
>
> Doesn't this break uniqueness of primary keys? If you have more than
> one record with id '1' with different states, what then? Though I
> think you avoid this in the sketch below.

Continuity table keeps it normal primary keys. 'Object
revision/version' table takes continuity table primary key and adds to
that primary key the revision_id column.

>>> In the pending use case, should the normal tables contain the pending
>>> information or the published information?
>>
>> This is an interesting question. Suggestion:
>
>> 1. Case where object is entirely new. Write it in normal way (to both
>> normal table with copy to revision table) and set state to 'pending'
>>
>> 2. When writing a 'pending' change to an already 'active' object. In
>> this case, I suggest we just write a new revision to the 'revision'
>> table with state set to pending and make no change to the 'normal'
>> table. Then when the change is approved we will write that change to
>> the active table (and, I guess, update the revision table to set the
>> state on that revision to active -- to avoid issues if we revert).
>> This will require a bit of a change to vdm to support this
>> functionality:
>>
>>  * Change the Revisioner MapperExtension so that it checks the state
>> field when making the write (before_update method i imagine) and does
>> not write to continuity object fields if state is 'pending'.
>>  * Have some support for marking 'pending' revisions as approved
>> which triggers write to continuity object (normal table)
>
> How would this work for records that are pending-removed?

Good question. Two options:

1. Ditch above and add a new explicit 'workflow' or 'pending' column
-- only needed on 'Revision' objects.
2. Have a special 'pending-deleted' state that moves the object to
deleted state on application.

The latter is more inline with the above but clearly somewhat hackish ...

> So your suggestion is to write the published info to the normal tables
> (the "continuity", right?), but also have pending new records in it?

Not really. Completely new objects which are 'pending' could either go
in continuity or not (given my proposal above would actually make more
sense to not put them in continuity -- however, if all you need was
the ability to have things as unpublished until some given point then
the simple use of 'pending' state would suffice).

Normal 'pending' changes would not go in continuity (but would only be
in revision table).

> Doesn't that make two common use cases more difficult?
>
> * query & manipulate "pending" data - the normal tables don't quite
> represent this as records may be updated invisibly in the revision
> table instead.

But how would you put pending items in the main table? You are going
to have to ignore them somehow ...

> * query & manipulate published data - the normal tables don't quite
> represent this as new pending records may be present.

True but I'm not sure how you would get round this. Do you have ideas?

> One important use case is that people may want to query the published
> data using raw SQL - it should be as close to unversioned data as
> possible. It's less important that the pending state can be queried in
> this way - though there should of course be some way to query a
> consistent state through the ORM so that a UI can be presented.
>
> I almost think there should be two continuities: one pending and one published.
>
> When a pending change is published, all other pending changes that
> point to it with a foreign key relation should also be published.

What is 'it' here. Also what happens if you want to cherrypick pending
changes to publish. I guess the question to ask here is:

"What exactly are the use cases"?

> I'm trying to figure how this relates to the concept of Revisions,
> which are global per database. It doesn't appear to be enough to be
> able to point to a revision and declare it the most recently published
> revision, because pending objects would be included as well. The
> analogy to a version control system is that "pending" changes are
> modifications to a checkout but haven't been committed yet.
>
> If we follow the analogy with a version control system, you could say
> the pending continuity is the checkout, and the normal continuity is
> the trunk. When a commit is made, parts (or the whole) of the checkout
> is committed into the trunk. This can all be done in a single
> revision. The checkout itself does not have individual revisions, only
> the trunk does. We can also assume that only a single checkout ever
> exists - that's a difference with a software version control system,
> so that updating the checkout to synchronize changes should never be
> needed.

But what happens if you want a) pending revisions to different objects
b) multiple pending revisions to a given object (or is that not
allowed?)

Rufus