[ckan-dev] sqlalchemy migrate, dependency issues.

Thu Jan 27 09:08:34 UTC 2011

On 27 January 2011 00:37, David Raznick <kindly at gmail.com> wrote:
> Hello
>
> The correct way to do this will involve, based on my understanding and your
> comments.
>
>  * changing the setup.py dependencies for sqlalchemy-migrate and sqlalchemy
> to their latest versions.
>  * running the migrate script on the repository (or changing the repository
> scripts manually).
>  * checking in both these changes at once.

Yes exactly. Just to emphasize there is no need to block on this
change. We just update the requirement to CKAN to sqlalchemy >= 0.5
(or whatever is needed for latest migrate) as there is no need to
support 0.4 going forward. As you say we should also upgrade the
migrate scripts at the same time.

> Along side this we should make one of these is in place.  In order of my
> preference.
>
> 1. We have a script on app launch that automatically updates the database to
> the latest version of the migrate-repository it has got.

+1 (can always raise error on failure). Perhaps a slightly nicer model
here is to show a window (a la wordpress) saying: "Your database is
out of date. Click here to upgrade." and then showing an upgrade
success window.

> 2. We should make sure the app will not run (i.e raise an exception) until
> the db update is done and it is in sync with the latest migrate-repository
> version it has checked out.
> 3. We add something to the fabfile to run update db every ckan update, and
> we always update using the fabfile.
> 4. We make it an absolute process that after any upgrade we do an update db.

That's what we do at the moment anyway (at least if you follow the UPGRADE.txt.

> At the moment I imagine we have got a slightly less strict version of 4
> going on?  am I right?

Yes.

> The issue I see with 1 is that if the application is started as an apache
> wsgi script with multiple processes there could be a lot of pain (we do not
> want to upgrade a database from 2 processes) and this would be hard to
> debug.  We would need some kind of update db lock.
>
> Basically every instances database HAS to be in sync with the latest version
> of the repository it has got.  If this is not assured we are not in a good
> place.
>
> I am happy to do the first 3 points (as they are easy) but only if we are
> confident that one of the above 4 points are in place.

Happy to follow your lead on this here. Sounds like this should be
ticketed whatever we do.

Rufus