[ckan-discuss] buildbot success in Open Knowledge Foundation on buildbot-test

[Rufus Pollock]

cc'ing ckan-discuss ... (Background: we've deployed a buildbot for
daily automated testing of ckan codebase)

On 18 March 2010 19:46, John Bywater
<john.bywater at appropriatesoftware.net> wrote:
> Phew! So, 47 builds later we've got a green bar. :-)
>
> http://buildbot.okfn.org/waterfall

Good work. By the way how does one get the ckan only set of results
(for the case where we're testing code other than ckan in the buildbot
at some future point ...). E.g. can i do:

http://buildbot.okfn.org/ckan/waterfall

> In earlier builds, there were tonnes of errors, mostly to do with version
> incompatibilities (and a few issues with using the AGPL3 license in the
> tests, which is in the "all_alphabetical" list but not in the
> "ckan_original" list that is now used by default).
>
> 1. The very latest available FormAlchemy package has a method signature
> incompatibility with CKAN. So I narrowed the version range in setup.py.

Great.

> 2. I did the same thing for Routes, because the very latest version isn't
> able to resolve "/tags" (Rufus: is this the dict()/{} issue again, as it was
> with KForge?).

I have already encountered significant (1h+) pain with this myself
over the last week (and should have mentioned it!) when deploying
no.ckan.net (and de.ckan.net). Basically restrict to <= 1.11.99.

> I would think those version numbers can be increased in setup.py once CKAN
> has been made to work with these newer versions.
>
> In general, I don't think it's a very good idea to set the required version
> of a dependency to be "anything later than x". I would propose we limit the
> version numbers to those versions of dependencies that are actually known to
> work.

I agree.

> In other words, if we are going to test only one combination of versions of
> dependencies (as we are with the current buildbot setup) it would be
> technically superior to require a specific version of each dependency. Since
> we install CKAN to virtualenv, there appears to be zero upside having
> variability of versions. The downside is non-zero. Inevitably there is

The risk is an older version is no longer available from e.g. pypi and
the install fails. That is a real risk and has happened to me.

> uncertainty which follows from the guaranteed variability of successive
> versions. There wouldn't be a new version if nothing had changed, and if
> something has changed there might be an incompatibility, however small the
> difference. So the cautious approach would be to fix the version numbers in
> any given revision, monitor dependecies for new releases (I don't yet know
> how to do that automatically), and integrate new versions in a deliberately
> gradual manner.

OK. I agree we don't want >= alone but i'm willing to take a
reasonable range of versions e.g.:

Routes>=1.9,<1.11.99

That said I can see the benefits at least in the "stable" requirements
of going with a very specific version.

> Anyway, now it's working, I've switched off the "force rebuild" option in
> the Web interface (because anybody could do that). Now it's set to run once
> per day at 0400. Let's see what's in our Inboxes in the morning....

That's great. I think we probably want a team at ckan.net address here
(or maybe, to be more precise, team at ckanproject.org ...)

Rufus