[ckan-dev] SQLite removal?

[David Read]

> Right, I'll often run the specific test modules that the code I'm
working on affects frequently as I'm developing, but only run the whole
test suite at the end before merging to master (if at all, Travis will
often have run it for me).

Great, you're finally addressing the situation that I'm talking about
(I'm in agreement about not running the full tests - leave it to
Travis).

As per my example, postgres takes 12s versus sqlite 3s for running a
couple of simple tests - the ones relevant to the code currently being
written. As you add a couple of lines of code and add a new assert to
the test in parallel with the feature, you'll find yourself running
the relevant tests several times a minute. Waiting 12s for them to run
each time and you simply break your flow.

Dave


On 14 June 2013 12:05, Sean Hammond <sean.hammond at okfn.org> wrote:
>> >> Having Travis run the tests for us has really reduced the amount of
>> > times devs need to run them on their laptops.
>> >>
>> >> You still need to sometimes, of course
>> >
>> > Not running tests often is the traditional programming way. However
>> > many people these days write and run tests frequently. But of course
>> > it's OKF's perogative whether or not they support that or not.
>>
>> I think what Sean is saying is that we run the tests via travis more
>> often than locally.
>
> Right, I'll often run the specific test modules that the code I'm
> working on affects frequently as I'm developing, but only run the whole
> test suite at the end before merging to master (if at all, Travis will
> often have run it for me).
>
> If we had thorough *unit* tests for every CKAN module as a coding
> standards thing, this approach would work better. The idea is that the
> unit tests test the interface/behaviour that a given module provides to
> other modules, so in theory if you don't break a module's unit tests you
> should be able to modify it and expect CKAN in general to keep working.
> And the unit tests for a single module should run very fast. You just
> need to run the whole test suite, including unit tests for other modules
> and cross-module integration/functional tests, before merging to make
> sure.
>
> Unfortunately CKAN's tests are far too slow (either with postgres or
> sqlite) to do some sort of local continuous testing thing where my
> laptop is running the whole test suite every time I modify the code.
>
>> The issue is that the core developers rarely/never run the sqlite
>> tests and these are not run by travis
>
> This has long been a problem, I think the time saving gain of sqlite is
> a bit dubious because it sort of introduces two test suites. Since
> everyone knows the postgres tests are the most thorough ones (skip less
> tests, more similar to production by using the same database) they run
> those ones and ignore the sqlite ones. As a result, stuff gets committed
> that fails with sqlite but not with postgres. Diagnosing and fixing
> these problems just wastes the time of any developer trying to save time
> by using the sqlite tests.
>
> If you really want to be thorough, you have to run the sqlite tests and
> the postgres ones to make sure your code doesn't break any of them, and
> then sqlite actually costs you time doesn't save it.
>
> And as I said, I think sqlite provides sort of a crutch that lets us run
> the tests faster without having to face up to efficiency problems in the
> test code and in CKAN itself.
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev