[ckan-dev] Testing update

[Sean Hammond]

I'm off on holiday next week. Here's an update on the testing pull
request:

https://github.com/okfn/ckan/pull/1117

The pull request is ready for review now. There's a list of TODO items
for ongoing work on the tests at the bottom of this email.

It'd be good if other people could start taking these on now. Even when
I'm back after next week, I have other stuff to do (*lots* of work on
the docs, plus lots of pull requests and issues assigned to me..). Also
it's just probably good if more of the dev team get their hands on this
test stuff and have an influence on it. I still intend to keep adding
new tests but I don't think I'll be able to spend anything like the same
kind of time on this.

Current status
--------------

This pull request #1117 is blocking any work on the new tests, since it sets
out the guidelines and examples for writing new-style tests, we can't go
ahead and start writing lots of tests until this pull request is
reviewed and merged.

The pull request is ready for review now. Unfortunately there wasn't
time to get it merged and reviewed by the end of the week. If someone
can review it next week, then I can respond to any comments when I get
back the week after, then we can get it merged.

Todo
----

(I'll make Trello cards on the team board for each of these.)

After this pull request has been merged, the main remaining tasks that
come to mind (in rough order of priority) are:

* Write more new-style logic tests.

This PR adds some new-style tests for user_update but doesn't even cover
all of user_update. We should pick the most important action functions,
e.g. user_*, package_* and organization_*, write tests for them first.
These should test the action, auth and converters/validators. Then
expand to the rest of the action functions.

* Add in some unicode.

One important thing that the tests I've written so far don't do is throw
any unicode at CKAN, this should be added in.

* Make the tests faster.

We've paid a fair bit of attention to this already but someone should
maybe profile it, something related to the db is making these tests
taking 30s to run when they should take 1s.

* Start writing controllers tests.

There's no tests for CKAN's frontend which sucks.

* Start writing model tests.

As with the logic tests, concentrate on the most important models first.

* Start writing lib tests.

Here certain things that are used e.g. by templates and extensions seem
especially important, e.g. helpers.py, and anything imported by the plugins
toolkit.

* Extension tests.

There's ongoing work on improving the docs for writing extensions,
writing example extensions, writing tests for example extensions, as
well as testing the plugin interfaces themselves directly.