[kforge-dev] using nosetests (pros/cons)

[John Bywater]

Rufus Pollock wrote:
> On 05/05/08 20:36, John Bywater wrote:
>> Rufus Pollock wrote:
>>>> We aren't diverging on tools, are we?
>>>
>>> I don't think so though I would *really* like to start using a 
>>> standard testrunner such as nosetest rather than our own 
>>> hand-crafted one if that were possible.
>>>
>>> I know this is something I've brought up before and we've stayed 
>>> with our hand-crafted solution as a results of your arguments but 
>>> with the development of e.g. nosetests over the last few years and 
>>> our usage of it on other projects it seems like it might really make 
>>> sense to start migrating to this ... (at least on kforge).
>>
>> Let's stick to running KForge tests with kforge-test for now. That 
>> code could make better use of the unittest test runner code. I never 
>> straightened it out. But let's keep things simple and leave this 
>> aspect alone for now.
>
> But why bother to continue to write our own bespoke wrapper of the 
> unittest runner where nosetests already exists?
>
>> Of course in time we can change anything. But let's be disciplined 
>> and remain with the current state for now.
>>
>>
>>>> My view #1: we don't need to change anything here because nothing 
>>>> is broken.
>>>> My view #2: we both need to be a force for convergence.
>>>>
>>>> Don't you want to use the unittest package when testing KForge? 
>>>> What's happening?
>>>
>>> nosetests runs unittest-based tests as well as non-unittest based 
>>> tests ...
>>
>>
>> Yes, but you don't know that it does everything in the same way, and 
>> we need to run the tests in the same way.
>>
>> Let's just run KForge tests with kforge-test. It isn't broken and we 
>> don't really need to spend time discussing it. Can't we both just run 
>> kforge-test?
>
> The thing is I really find nosetests a lot more convenient:
>
>   * No need to write special 'suite' methods
>   * Easier ways to run/not run tests from command line
>   * Better reporting from succeeding/failing tests (using -d switch)
>   * Possibility to use nosetest plugins etc
>
> Thus, at least on KForge I'd be a big +1 for moving to this 
> particularly since I don't think we actually need to change any of our 
> tests to have them run by nose ...

One problem I noticed with automatic test case detection (or whatever 
its called) is that it tries to run base test cases.

So where we have...

WidgetTestCase
 - lots of common widget testing behaviour

TestWidgetMode1
 - tests a slender aspect of behaviour

TestWidgetMode2
 - tests another slender aspect of behaviour

etc.

...then the WidgetTestCase would be picked up, which is wrong. I've had 
to avoid this with Microfacts/CKAN but it's "incorrect". I don't like 
the behaviour, it smells. Also, it's not very well patterned.

If you don't see any value in the test suite, then look again at the 
composite pattern: the test suite objects work to compose the test 
method containers (test cases).

I'm sure you would agree, the relative cost of writing the name of a 
test case into the test suite's list is negligible in comparison with 
the cost of writing the test. In my view, the benefit of the added 
stability that arises from the explicit statement of the test suite's 
contents indicates having a test suite is a worthwhile thing to do. 
Having things explicit is more Pythonic. Additionally, using nosetests 
with KForge adds another dependency.

We aren't continuing to write our own test runner code. We can properly 
invoke the unittest test runner code.

The broad view is that there are two separate concerns: doing the same 
thing; developing what we do. We firstly need to do the same thing. Then 
we need to change together. But I don't want to move to using nosetests 
with KForge, at least at the moment. Nosetests is "an alternative test 
discovery and running process for unittest". But I don't need to 
discover my test cases. When I'm being good, the first thing I do when 
writing a new test case is add the name into the suite and check to see 
the suite fails!

J.