[annotator-dev] Automated test results

Tue Aug 5 21:01:48 UTC 2014

Hi Everyone,

Based on what we discussed on the call a couple of weeks back, we went back to using grunt-saucelabs to run our test suite.  As I mentioned previously, SauceLabs gives us detailed error reporting for each failed test case as well as providing easily-embeddable results that we can publish on the GitHub repo or elsewhere.That's been working well for the most part, and we're starting to get a pretty good picture of what's working and what isn't.  Here's a link to the results so far:  

https://saucelabs.com/u/krues8dr-annotator

The current results have a few "Unknown" values that are the result of the test not getting run properly - I have open tickets for some of these issues with SL.  There are also simply some browsers for which we can't run automated tests due to lack of support, but in most cases we should be able to run the tests manually through their service.  Among these is IE8, which appears to not be supported by Selenium itself without some extra work from SL - which I have not gotten them to acknowledge yet.

I've made a few minor changes to support this work on our fork over here while it's still in-progress:

https://github.com/opengovfoundation/annotator/tree/grunt-test

Among these changes is replacing chai with assertive-chai as the former doesn't support IE8 either.  Assertive-chai is just a stripped down version that basically only uses the assert() paradigm of chai - which works perfectly for how we're using it.

Our next steps are to try and get the last few missing browsers from that list up and running, and working down the list of browsers that aren't working to assess and fix issues, starting with IE8.

Cheers,
-Bill

Bill Hunt
Senior Developer
OpenGov Foundation
http://opengovfoundation.org/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/annotator-dev/attachments/20140805/b8cb0f3a/attachment-0003.html>