[ckan-dev] test performance and sqlalchemy complete

Thu Jan 6 11:19:45 UTC 2011

On Thu, Jan 6, 2011 at 10:53 AM, Seb Bacon <seb.bacon at gmail.com> wrote:

> It's just been pointed out to me that you also need to apply
> sqlalchemy.patch (in the root of the ckan src) to get everything to
> pass.
>
> I guess the best way to deal with that for now is to monkeypatch it on
> startup.  Thoughts?
>
> I think a monkeypatch is fine, especially as its just for testing.
However, it will only work for sqlalchemy 0.6.

Seb
>
> On 5 January 2011 19:21, Seb Bacon <seb.bacon at gmail.com> wrote:
> > Hi,
> >
> > I've finished the saga that was upgrading sqlalchemy and also getting
> > tests to run against sqlite.  Huge thanks to David for doing a code
> > review and suggesting further changes. For the record, here's the
> > speed improvements on my box for each incremental change (I'd be
> > interested to know speed improvements on other platforms):
> >
> > 1) benchmark to start with: 41m (this is on Lucid, so the worst case)
> > 2) changes and optimisations of the test suite, plus turning off
> > postgres durability settings: 21m
> > 3) upgrade to sqlalchemy 0.6: 16m
> > 4) using sqlite in-memory database: 5m
> >
> > The work closes tickets #876 and #576, but unfortunately I seem to
> > have forgotten my password, and the password reminder function is
> > broken (can anyone help?!)
> >
> > Below is an extract from the updated README.txt to fill you in on
> > getting tests to run faster on your box.
> >
> > This is all in my fork at https://bitbucket.org/sebbacon/ckan, which
> > hopefully someone will pull to the main repo soon.  Note that for all
> > the tests to pass, you also need an updated VDM (currently in my
> > bitbucket account, also needs to be pulled).  The test suite passes
> > against sqlalchemy 0.4.8 and 0.6.5, but as we don't have 100% coverage
> > we should be really careful to user test it thoroughly before
> > deploying.
> >
> > Seb
> >
> > --------
> > from README.txt:
> >
> > The test suite takes a very long time to run against a standard
> > postgres.  You can make postgres run faster by turning off durability
> > as described at
> > <http://www.postgresql.org/docs/9.0/static/non-durability.html>.
> >
> > Even with durability turned off, the full test suite can take around
> > 15 minutes to run.  If you test against an in-memory sqlite database,
> > this can drop to as low as 5 minutes.  To do this, uncomment the
> > following line in test.ini:
> >
> > ::
> >
> >  sqlalchemy.url = sqlite:///
> >
> > .. caution ::
> >
> >   Note that (a) some search-related tests are currently skipped due
> >   to postgres-specific code; and (b) only postgres is currently
> >   supported in production anyway.  Therefore, you should treat the
> >   sqlite support as a convenience during development, and always run
> >   the tests against postgres as a final check.
> >
>
>
>
> --
> skype: seb.bacon
> mobile: 07790 939224
> land: 0207 183 9618
> web: http://baconconsulting.co.uk
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20110106/a8486e7a/attachment-0002.html>