[ckan-dev] SQLite removal?

[David Read]

Ha, well it's just one line in the occasional test...


On 14 June 2013 11:15, Toby Dacre <toby.okfn at gmail.com> wrote:
> On 14 June 2013 10:30, David Read <david.read at hackneyworkshop.com> wrote:
>>> Having Travis run the tests for us has really reduced the amount of
>> times devs need to run them on their laptops.
>>>
>>> You still need to sometimes, of course
>>
>> Not running tests often is the traditional programming way. However
>> many people these days write and run tests frequently. But of course
>> it's OKF's perogative whether or not they support that or not.
>
> I think what Sean is saying is that we run the tests via travis more
> often than locally.
> The issue is that the core developers rarely/never run the sqlite
> tests and these are not run by travis
>
> they add extra complexity to the code and dubious benefit to the core
> developers who tend to run the full postgres test exclusively.
>
> To be honest we are looking at a total review of the tests as they are
> showing their vintage and have turned to a spaghetti mess over time
> and really need rationalising.
>
> It sounds like the core devs are in favour of removing sqlite and
> apart from the speed issue (which does need addressing) I do not hear
> any convincing argument for keeping it.
>
>>
>> Where tests rely on postgres features then we have skips for running in sqlite.
>>
>> Dave
>>
>>
>> On 13 June 2013 23:33, Sean Hammond <sean.hammond at okfn.org> wrote:
>>> Having Travis run the tests for us has really reduced the amount of
>>> times devs need to run them on their laptops.
>>>
>>> You still need to sometimes, of course.
>>>
>>> I'd argue that the real fix for this is to write a better and faster
>>> test suite. Have more unit tests and not as many functional tests, unit
>>> tests are faster because they just import a module and tests its
>>> functions directly, instead of testing the whole application stack.
>>> Avoid unnecessarily hitting the disk, network or database in tests.
>>> And let each test just run the setup code that it needs, instead of
>>> having large, generic text fixtures (create_test_data) that get run
>>> again and again.
>>>
>>> Also, making CKAN faster will make the tests faster.
>>>
>>> Of course this is a (very) long-term approach, sqlite is a quick gain.
>>>
>>> I think the problem with sqlite is we're starting to get maintenance
>>> costs. People are introducing code that works with postgres but fails
>>> with sqlite (and not realizing because they don't run the sqlite tests).
>>> People are adding code that uses postgres features that sqlite doesn't
>>> support.
>>>
>>> We'd have to get Travis to run the sqlite tests as well, if we want to
>>> make sure they keep working. And Travis already runs the tests four
>>> times for every commit with different versions of Python and Postgres.
>>>
>>> On Wed, Jun 12, 2013 at 02:53:17PM +0100, David Read wrote:
>>>> sqlite tests were introduced purely for speed. Running all tests was
>>>> taking over 10 minutes, putting developers off running them. We even
>>>> had people starting to ruthlessly disable reams of key tests to make
>>>> them faster! With a tuned sqlite we got it down to 1 minute for all
>>>> tests - about 400 at the time I think.
>>>>
>>>> I use sqlite still for CKAN tests all the time. I think most of the
>>>> new developers simply haven't given it a try, so here's my pitch:
>>>>
>>>> sqlite is perfect for TDD or anything approaching it, since the
>>>> code-test turn-around time of a few tests is far quicker. In the
>>>> example below, running 3 tiny tests takes 3s with sqlite and 12s with
>>>> postgres.
>>>>
>>>> Or maybe people have found another way to run a test in a couple of seconds?
>>>>
>>>> David
>>>>
>>>> (pyenv-ckan)dread at dthink:~/gitroot/ckan$ time nosetests --ckan
>>>> --with-pylons=test-core.ini ckan/tests/lib/test_munge.py
>>>> ...
>>>> ----------------------------------------------------------------------
>>>> Ran 3 tests in 8.940s
>>>>
>>>> OK
>>>>
>>>> real 0m11.963s
>>>> user 0m6.060s
>>>> sys 0m0.720s
>>>> (pyenv-ckan)dread at dthink:~/gitroot/ckan$ time nosetests --ckan
>>>> --with-pylons=test.ini ckan/tests/lib/test_munge.py
>>>> ...
>>>> ----------------------------------------------------------------------
>>>> Ran 3 tests in 0.365s
>>>>
>>>> OK
>>>>
>>>> real 0m3.282s
>>>> user 0m2.852s
>>>> sys 0m0.384s
>>>> (pyenv-ckan)dread at dthink:~/gitroot/ckan$
>>>>
>>>> On 12 June 2013 08:53, Hendrik Bunke <bunke.hendrik at gmail.com> wrote:
>>>> > Personally I love using SQLite for developing and testing
>>>> > purposes. Postgresql is kind of overkill for such cases. So I'd
>>>> > vote for leaving it in.
>>>> >
>>>> > regards
>>>> > hendrik
>>>> >
>>>> > --On 2013-06-11 14:19, Toby Dacre wrote:
>>>> >> As I understand it we can use SQLite for the tests (for some
>>>> >> performance benefits)
>>>> >>
>>>> >> There are also some issues regarding SQLite being broken
>>>> >>
>>>> >> Is this something that we should continue to support?  I am not sure
>>>> >> the extra maintenance is worth it - what do people think?
>>>> >>
>>>> >> If so shall we do this in 2.2
>>>> >>
>>>> >
>>>> > --
>>>> > Dr. Hendrik Bunke
>>>> > http://gplus.to/hbunke
>>>> > http://twitter.com/hbunke
>>>> > http://www.hbxt.org
>>>> >
>>>> > _______________________________________________
>>>> > ckan-dev mailing list
>>>> > ckan-dev at lists.okfn.org
>>>> > http://lists.okfn.org/mailman/listinfo/ckan-dev
>>>> > Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>>>
>>>> _______________________________________________
>>>> ckan-dev mailing list
>>>> ckan-dev at lists.okfn.org
>>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
>
>
> --
> Toby Dacre
>
> The Open Knowledge Foundation
>
> Empowering through Open Knowledge
> http://okfn.org/  |  @okfn
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev