Ben EmmonsWeb & Mobile Services, UITSAutomated Testing

10/28/2014

Why automation?

10/28/2014

“Any time you have a human at a keyboard doing

ANYTHING, that process will not scale.”

- Chad Smith, Global Knowledge

10/28/2014

One Site

10/28/2014

Easy Patching

10/28/2014

One Multisite

10/28/2014

Medium Patching

10/28/2014

Multiple Environments

10/28/2014

Arduous Patching

10/28/2014

340+ Sites

10/28/2014

Sisyphean Patching

10/28/2014

Solution?

10/28/2014

Automate Security Audits

10/28/2014

Automate Patching

10/28/2014

Automate Syncing

…and clearing cache, and renewing SLAs,

and security scan reminders,and customer notifications,

and…

10/28/2014

Automated Testing

10/28/2014

Why is that so hard?

10/28/2014

First, pick engine(s)

Gotchas:• IE can only run reliably on a Windows OS

Ramification: testing solution requires Windows• Latest Safari will only run on Mac hardware

Ramification: no AWS option and good luck hosting a Mac in the UITS server room…

10/28/2014

Next, pick driver

Gotchas:• IE Compatibility Mode is not reliable: http://stackoverflow.com/a/11418046

• TrifleJS is “very much in beta” and cannot emulate versions on Windows Server 2012

• WebDriver is slow• Node.js solutions target a specific layout engine• Very little Sahi documentation and small community

10/28/2014

Unit testing?

10/28/2014

… How about Functional?

10/28/2014

… Ready to run the tests?

10/28/2014

… Don’t forget your CSS!

10/28/2014

Why bother?

- Jackson Murtha

http://iterate.blendinteractive.com/preventing-upgrade-headaches-with-selenium-webdriver/

10/28/2014

Making the right choice• Avoid choice paralysis http://addyosmani.com/blog/front-end-choice-paralysis/

“The challenge is that when presented with an abundance of choice, you can feel increasingly unsure of whether yours is the right one. Instead of being happy with our decisions, we’re filled with anxiety. Did I pick the wrong one? Would I have been better off with something else? What would my friends have used? ... If you’re an experienced developer speaking to someone stuck on a decision, try to help them simplify their options.” - Addy Osmani

• Unit vs. Functional http://www.helpscout.net/blog/functional-testing-casperjs/ “No one cares that your Unit tests are passing if your application doesn’t function. Unit testing makes sure you are using quality ingredients. Functional testing makes sure your application doesn't taste like crap. ... [In] the real world, most developers barely have time for one kind of automated test, let alone two. If that describes you, I would strongly encourage you to go with Functional testing.” - Brett Jones

10/28/2014

Our Stack• Amazon Web Services: https://aws.amazon.com/

• Selenium Grid: https://code.google.com/p/selenium/wiki/Grid2

• IE, Ff, Ch: https://github.com/conceptsandtraining/modernie_selenium

• Safari: UITS Mac colo or Sauce Labs free• Mobile: http://appium.io/

• WebdriverIO: http://webdriver.io/

• Unit Testing: Buster.jshttps://github.com/webdriverio/webdriverio/blob/master/examples/webdriverjs.with.buster.js

• WebdriverCSS: https://github.com/webdriverio/webdrivercss

• WebdriverCSS Adminpanel: https://github.com/webdriverio/webdrivercss-adminpanel

10/28/2014

Eas(ier) Button

• Getting a test environment up and running is half the battle. Do yourself a favor and just pay out.• If getting budgetary approval is akin to an act of

Congress, at the very least use our AWS CloudFormation template: https://bitbucket.org/uitsweb/selenium-grid-ec2

10/28/2014

Demo

10/28/2014

Demo

10/28/2014

Demo

10/28/2014

Demo

10/28/2014

Demo

http://www.whatsmybrowser.org/ node wdio.js

10/28/2014

Demo

http://coolepochcountdown.com/ node wdcss.js

10/28/2014

Demo

http://coolepochcountdown.com/ node wdcss.js

10/28/2014

Next Steps• Update Selenium Grid template to use NAT private

subnet• Firewall exception to allow AWS Selenium Grid to

access test sites (or use SSH reverse proxy)• Update Selenium Grid template to use Modern.IE VMs• Drush script for dynamically obtaining top menu links• Update our Drupal automation tool to launch (and

respond to) automated testing of top site pages• Set up Appium on Mac OS X server and arrange colo• Profit!

10/28/2014

Questions?