[okfn-help] Post mortem: Is It Open launch

[John Bywater]

Having spent some slightly uncomfortable time yesterday discussing 
errors in the recently launched Is It Open System with Peter, and as I 
have just a minute, I feel it shouldn't be left uncommented that the 
errors which Peter experienced, and which he raised on the main public 
discussion list (with some implications for reputation of the various 
parties involved) arose from an *untested* section of code (that which 
receives response emails).

This aspect of the isitopen code is basically a piece of user interface 
("data handler sends an enquiry response to the system by email") and 
could have been tested for a modest range of input cases, by asserting 
each input case is presentable. It would not have been more than a few 
minutes before the "broken HTML in email" case showed up.

Therefore I don't think this was just a case of bad luck, but rather it 
was an error of process ("deviation from test-driven development"): it 
was decided (collectively) that this section of code would not be tested 
any more as it was considered to be working already.

It matters only that we understand this was not some stochastic event (a 
piece of "bad luck") but determined by not cooking up the system so that 
it is bounded to work only in ways that are intended (through a 
test-first approach).

Of course, this isn't to demand the impossible ("all releases must 
always be totally error free") but that in this case a little more time 
spent testing something that was known to be untested would have avoided 
much more time being spent discussing a broken service with users. In 
other words, producing a higher quality system would have involved less 
total time overall.

Best wishes,

John.


-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: Attached Message Part
URL: <http://lists.okfn.org/pipermail/okfn-help/attachments/20100221/8d153048/attachment.txt>