[ckan-dev] sqlalchemy migrate, dependency issues.

Thu Jan 27 09:11:22 UTC 2011

On 27 January 2011 00:37, David Raznick <kindly at gmail.com> wrote:
> Hello
>
> The correct way to do this will involve, based on my understanding and your
> comments.
>
>  * changing the setup.py dependencies for sqlalchemy-migrate and sqlalchemy
> to their latest versions.
>  * running the migrate script on the repository (or changing the repository
> scripts manually).
>  * checking in both these changes at once.
>
> Along side this we should make one of these is in place.  In order of my
> preference.
>
> 1. We have a script on app launch that automatically updates the database to
> the latest version of the migrate-repository it has got.
> 2. We should make sure the app will not run (i.e raise an exception) until
> the db update is done and it is in sync with the latest migrate-repository
> version it has checked out.
> 3. We add something to the fabfile to run update db every ckan update, and
> we always update using the fabfile.
> 4. We make it an absolute process that after any upgrade we do an update db.
>
> At the moment I imagine we have got a slightly less strict version of 4
> going on?  am I right?
>
> The issue I see with 1 is that if the application is started as an apache
> wsgi script with multiple processes there could be a lot of pain (we do not
> want to upgrade a database from 2 processes) and this would be hard to
> debug.  We would need some kind of update db lock.

I would vote for both (2) and (3).

For (2), in addition to your point above, you can never be sure that
something odd on the version you're upgrading means it needs admin
attention.  A process that is kicked off by hand would be more visible
to the naive user.

For (3), the update should indeed happen as part of our standard
deploy process, which means it should be scripted.  Obviously a failed
migration should cause a rollback to the previous version.

Thanks,

Seb

> Basically every instances database HAS to be in sync with the latest version
> of the repository it has got.  If this is not assured we are not in a good
> place.
>
> I am happy to do the first 3 points (as they are easy) but only if we are
> confident that one of the above 4 points are in place.

> On Wed, Jan 26, 2011 at 8:30 PM, David Read <david.read at okfn.org> wrote:
>>
>> Well it's only an issue when we upgrade CKAN, so let's focus on those
>> that we do need to migrate. I'm needing on gvt servers later this week
>> so I'll have a play if David doesn't get there first. I suggest we
>> check-in a change that changes the requirement in setup.py to
>> sqlalchemy>=0.6 and a migration script that prints a warning showing
>> the need to run this migrate tool (assuming the migrate script can't
>> run it itself?).
>>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
>
>

-- 
skype: seb.bacon
mobile: 07790 939224
land: 0207 183 9618
web: http://baconconsulting.co.uk