[okfn-help] vdm: improvement thoughts

[Rufus Pollock]

On 25 August 2010 12:36, Martijn Faassen <faassen at startifact.com> wrote:
> Hi there,
>
> Now that I've looked a bit at vdm, here are some thoughts about
> improving it I'd like to run by you all. If I end up using vdm for my
> project (and I am hoping to), I can spend some time helping out with
> these.
>
> * vdm has support for both sqlalchemy and sqlobject. Is the sqlobject
> support being maintained at all? Is it being used? If we can, I'd like
> to throw out the sqlobject support altogether. We could then get rid

We should. It is clearly marked as deprecated in the docs.

> of some nesting and put the actual code in 'vdm' proper, retaining
> 'sqlalchemy' for backwards compatibility only. I think that would

I think that is a bad idea. there is no harm with the extra nesting
and it means we could add couchdb or mongodb packages if we wanted :)

> clean up the namespace and make the imports nicer. Unless we
> anticipate vdm working in the future with another ORM altogether, but
> I don't see the reason to keep the code bundled into one package
> anyway - such a new vdm would presumably have quite a different design
> and its use would be different.

My feeling is the cost is minimal of keeping it how it is and changing
it will be a major pain (and deprecating the sqlalchemy subpackage
will be very difficult giving usage by client libraries)

> * pending revision support - the published database in one state, but
> a set of newer pending revisions are available as well. Perhaps this
> is simply an implementation pattern we can document, or perhaps this
> needs new code.

I think first pass would be to write some client code doing this. If
it 'just works' we can document this as a pattern -- o/w we can add
new code.

> * constructing a vdm compliant set of tables and models is rather
> involved: set up a revision table for your database, make your tables
> stateful, set up revision tables for each table, inherit from a whole
> bunch of mixins in the models, create the Revision model, use the
> Revisioner extension in the mapper, map the revision models. I hope we
> can cut down on the amount of stuff you need to write by automating
> some of this.

I agree, though it is a one-off cost and more you automate the more
magic there is!

> * MySQL support; I've already done work on this.
>
> * test infrastructure for testing with MySQL and PostgreSQL - lots of
> discussion about MySQL support already
>
> * the documentation needs work. there's no discussion on how to set up
> revisioned tables, or how to set up revisioned models (what mixins are
> for what?), some of the documentation is actually not correct in
> combination with sqlalchemy (there's no 'revision.commit()' for
> instance), and it'd be nice to have an explanation of how you are
> supposed to set a revision on a session (direct assignment or using
> the set_revision infrastructure?). Explanation of how to do queries
> within revisions would also be useful (I'm not exactly sure how myself
> right now).

All good points. Please feel free to start modifying documentation and
I'll pull the patches.

> I don't mean to step on anyone's toes with these ideas of course, and
> it'll take time to do all of them, but I'd like to start at least a
> discussion on it.

No this is great the suggestions are all good. The toughest thing is
where we introduce backwards incompatible changes!

Rufus