[ckan-dev] SQLite removal?

[Sean Hammond]

> >> Having Travis run the tests for us has really reduced the amount of
> > times devs need to run them on their laptops.
> >>
> >> You still need to sometimes, of course
> >
> > Not running tests often is the traditional programming way. However
> > many people these days write and run tests frequently. But of course
> > it's OKF's perogative whether or not they support that or not.
> 
> I think what Sean is saying is that we run the tests via travis more
> often than locally.

Right, I'll often run the specific test modules that the code I'm
working on affects frequently as I'm developing, but only run the whole
test suite at the end before merging to master (if at all, Travis will
often have run it for me).

If we had thorough *unit* tests for every CKAN module as a coding
standards thing, this approach would work better. The idea is that the
unit tests test the interface/behaviour that a given module provides to
other modules, so in theory if you don't break a module's unit tests you
should be able to modify it and expect CKAN in general to keep working.
And the unit tests for a single module should run very fast. You just
need to run the whole test suite, including unit tests for other modules
and cross-module integration/functional tests, before merging to make
sure.

Unfortunately CKAN's tests are far too slow (either with postgres or
sqlite) to do some sort of local continuous testing thing where my
laptop is running the whole test suite every time I modify the code.

> The issue is that the core developers rarely/never run the sqlite
> tests and these are not run by travis

This has long been a problem, I think the time saving gain of sqlite is
a bit dubious because it sort of introduces two test suites. Since
everyone knows the postgres tests are the most thorough ones (skip less
tests, more similar to production by using the same database) they run
those ones and ignore the sqlite ones. As a result, stuff gets committed
that fails with sqlite but not with postgres. Diagnosing and fixing
these problems just wastes the time of any developer trying to save time
by using the sqlite tests.

If you really want to be thorough, you have to run the sqlite tests and
the postgres ones to make sure your code doesn't break any of them, and
then sqlite actually costs you time doesn't save it.

And as I said, I think sqlite provides sort of a crutch that lets us run
the tests faster without having to face up to efficiency problems in the
test code and in CKAN itself.