[okfn-help] vdm: improvement thoughts

[Martijn Faassen]

Hi there,

On Tue, Aug 31, 2010 at 8:14 PM, Rufus Pollock <rufus.pollock at okfn.org> wrote:

>> of some nesting and put the actual code in 'vdm' proper, retaining
>> 'sqlalchemy' for backwards compatibility only. I think that would
>
> I think that is a bad idea. there is no harm with the extra nesting
> and it means we could add couchdb or mongodb packages if we wanted :)

Already SQLObject and SQLAlchemy have significant API differences
(making the SA docs as published broken!) and at least they're both
about an ORM. Wouldn' t it be extremely likely that versioning with
couchdb or mongodb is going to be very different? Would it then make
sense to name it 'vdm'? What benefit would there be to a user that
it's under 'vdm'? What benefit to a developer of vdm?

I can see drawbacks - vdm would have a widely different set of
possible dependencies depending on how it is to be used, and it'll be
hard work for someone to run all the tests. And how to document it? I
can't see a cost in just starting a new package for a new underlying
database - in fact if that had been done with the sqlalchemy support
there'd been no reason to get rid of the sqlobject code now, and
someone might've ported over the docs. :)

I'll tell you why the extra nesting as it is now bothers me. Normally
I'm not bothered much by extra nesting as I can do:

from foo import bar

and then refer to the individual names in the code using 'bar.qux()"
and such. This works if 'foo' is a namespace package, like 'vdm' is.

it balances things nicely, not too verbose (foo.bar.qux() tends to
become that quickly) and not too obscure where 'bar' comes from when
you're reading the code.

but doing:

from vdm import sqlalchemy

gives me the name sqlalchemy, which is useless as there already is a
'sqlalchemy'. I could add in my own 'as vdm', but that would make my
code different from any other user's, and the job of a good library is
to make it clear how to use it in a clean way.

If we just had the code in 'vdm', we'd be able to write:

import vdm

and then use methods from that. Minor tweaks to an API can make quite
a difference in how it is perceived.

>> clean up the namespace and make the imports nicer. Unless we
>> anticipate vdm working in the future with another ORM altogether, but
>> I don't see the reason to keep the code bundled into one package
>> anyway - such a new vdm would presumably have quite a different design
>> and its use would be different.
>
> My feeling is the cost is minimal of keeping it how it is and changing
> it will be a major pain (and deprecating the sqlalchemy subpackage
> will be very difficult giving usage by client libraries)

Changing it is easy; you just move the code from 'sqlalchemy' into
'vdm' and then have 'sqlalchemy' import from vdm to retain backwards
compatibility. You can emit deprecation warnings (or just do
documentation) telling people to fix their imports and eventually
eliminate the sub package. It's easy to maintain the deprecated
sub-package as it'd just be a bunch of imports (which you could test
for).

>> * pending revision support - the published database in one state, but
>> a set of newer pending revisions are available as well. Perhaps this
>> is simply an implementation pattern we can document, or perhaps this
>> needs new code.
>
> I think first pass would be to write some client code doing this. If
> it 'just works' we can document this as a pattern -- o/w we can add
> new code.

I don't know how it is supposed to work and I need help. Would the
continuation be the pending revision or the published revision, for
example? How does one access the non-continuation revision in queries?
I don't know, and I need some insight on how this could be done. I
need to write this kind of code, sooner rather than later

>> * constructing a vdm compliant set of tables and models is rather
>> involved: set up a revision table for your database, make your tables
>> stateful, set up revision tables for each table, inherit from a whole
>> bunch of mixins in the models, create the Revision model, use the
>> Revisioner extension in the mapper, map the revision models. I hope we
>> can cut down on the amount of stuff you need to write by automating
>> some of this.
>
> I agree, though it is a one-off cost and more you automate the more
> magic there is!

It's quite an additional burden over writing a SQLAlchemy model and it
takes a beginner quite a bit of time to figure out what is going on
(especially with no documentation about it, only example code).
Calling a simplification of this 'magic' seems to imply you think such
a simplification cannot be done in a clean way?

Regards,

Martijn