[annotator-dev] Automated Browser Testing

Mon Jun 23 17:58:58 UTC 2014

I spent the morning working with BrowserStack, and it doesn't appear to be quite as mature as SauceLabs.

You can definitely still run tests in a cloud-based browser as all of the other services offer, but like Browserling, the failures are not logged and reported in the dashboard.  Or, at least, it doesn't appear to do this by default and they don't really have very deep documentation on how to do this.   It still looks like those errors can probably be reported back to the script running them, but I haven't had time to dig in very deep on that and again, it doesn't look as easy as what SauceLabs already has in place.  If anyone on the list has other experience they can contribute, please chime in!

Aron - to your question, grunt-saucelabs gives us the seamless integration with the SauceLabs API and dashboard.  We can run tests automatically and the results are published to the SauceLabs dashboard.  Sounds simple, but apparently not everyone is doing this.

I've found only one Gulp-SauceLabs module so far, but it's only got two commits several months old; I have not actually had the time to try and get it to run yet though.  Again, Gulp is still really new, and we probably aren't going to find wide support yet here.  Obligatory wheel-reinventing-yadda-yadda.

Cheers,
-Bill

Bill Hunt
Senior Developer
OpenGov Foundation
http://opengovfoundation.org/

Ph: 20-BILL-HUNT
       202 455 4868
bill at opengovfoundation.org

On Jun 23, 2014, at 10:53 AM, Aron Carroll <aron.carroll.lists at gmail.com> wrote:

> On 20 Jun 2014, at 23:08, Bill Hunt <bill at krues8dr.com> wrote:
> 
>> The next most popular option for Saucelabs integration is grunt-saucelabs which works perfectly, but requires grunt.  It's pretty coupled to grunt tasks too, so it'd require a bit of rewrite to get it to work without grunt.
> 
> Hey Bill,
> 
> I’m not particularly fond of Grunt either, and much prefer the Makefile approach. I may have missed some of the prior discussion but what particular functionality is grunt-saucelabs providing?
> 
> Also looking at the README it seems that the project only requires grunt to run itself. And then just requires a custom Mocha runner to be used by the test suite. So worst comes to worst, we can probably continue as we are and just use Grunt to talk to Sauce Labs.
> 
> I’d also be interested in us vetting other options such as BrowserStack first though.
> 
> Cheers,
> Aron

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/annotator-dev/attachments/20140623/3c53afab/attachment-0004.html>