OUTFITTERY QATEST AUTOMATION ChallengesSolutionsThe tools we are going to useHow we are going to use the toolsIntroduction to Zephyr for Test case management

CHALLENGES

Micro services ArchitectureHeterogeneous ComponentsShifting PrioritiesLack of Resources

SOLUTIONS

Leverage existing tool sets Implement changes incrementally Start NOW

AUTOMATION TOOLS

jMETER Jenkins The Intern OWASP ZAP Logstash NewRelic JIRA Browserstack

JMETER

Can interface with multiple components from a single interface reducing the cost of context switching

Modules can be used independently by developers, can be run headless in IDE

Tests scale for load testing using BLAZEMETER Integrates with Jenkins Exercises out RESTful layer Automated testing components of Task Manager

JMETERTEST

JENKINS

Jenkins has plugins that support all of the tools we plan on using

Enabling code coverage tools begin at the JENKINS interface Is the locus of control for all automation If it can be called from command line, Jenkins can use it Will require developer assistance initially

JENKINS

The plugins we plan on using Code Coverage: JaCoCo or Cobertura Blazemeter plugin for jMETER Selenium Grid plugin JIRA plugin Zephyr Plugin Zapper Plugin

THE INTERN

The best JavaScript automation framework Integrates with Selenium grid and Browser Stack Integrates with Jenkins Requires developer assistance to complete configuration for

QA

OWASP ZAP

Automated security and penetration testing Scriptable, extensible Integrates with Jenkins plugin We need to understand the vulnerability of our systems Last run exposed 4 SQL injection vulnerabilities

LOGSTASH

Centralizes the processing of all logs to be monitored Facilitates synthetic monitoring Jenkins plug available On site expertise available for assistance in configuring

NEWRELIC

Additional monitoring tool for aggregating performance data Supports Synthetic monitoring Already deployed, ready to extend

JIRA

Our enterprises ALM tool Supports our SDLC Multiple plugins to reduce context switching

BROWSERSTACK

The solution for cross platform/browser testing Scripts created in INTERN will be deployed to Browsestack This is how we will obtain test coverage against the plethora of

mobile devices we currently don’t test against

HOW WE ARE GOING TO AUTOMATE

First step is to configure Jenkins to use a code coverage plugin We will generate instrumented build to evaluate the

effectiveness of our test coverage. This will enhance the effectiveness of the manual testing by

identifying sparse code execution Data will assist in deciding which paths are the best

candidates for the next level of automation

HOW WE ARE GOING TO AUTOMATE

Configure Jenkins to use jMETER scripts Make repository of all modular jMETErRtests available to all

development team Use existing jMETER tests created for Task Manager as the

first set. Continually add modular tests as created Preparation for load, stress and performance testing when

ready

HOW WE ARE GOING TO AUTOMATE

Configure Jenkins to execute Intern tests via the selenium grid driver

First test suite is a simple funnel test Next test is exercising the create order page Continually add tests once the simple framework is verified to

be working as expected Extend tests to support Synthetic Monitoring

HOW WE ARE GOING TO AUTOMATE

After simple funnel tests are verified to work wit the Jenkins setup the tests will be passed to Browserstack

Browserstack will execute the tests against selected environment currently not tested against

Browser stack will be included with all manual tests

HOW WE ARE GOING TO AUTOMATE

Jenkins will be configured for ZAPPER based upon ZAP scripts modified on the desktop

ZAPPER test will run on a schedule that includes new build and a fixed schedule e.g. for new deployed builds and on Sunday night

Jenkins will be configured to use SLACK top provide granular real time notifications of automation status

HOW WE ARE GOING TO AUTOMATE

When all of the preceding configurations are running as expected we will add to the test suites by automating regression tests based upon bugs or specific feature tickets

Zephyr is a key component of this strategic process

ZEPHYR

Plugin for Atlassian Cloud Replacement for Test Rails Test Case management while in the JIRA ecosystem, reduces

context switching Test case creation accessible to all JIRA users Works in synch with current dev processes Supports refinements to our development processes Extensible, can be configured to execute automation from

inside of JIRA

ZEPHYR

Supports test case reuse Links to tickets Can be created based upon any existing issue in JIRA Can plan test execution cycles. Facilitated distribution of test cases to other team members Adds an element of transparency to our projects Dashboard widgets easy to configure

ZEPHYR DEMO

Demonstrate creation of a test case Creation from a clone Creation of a test cycle Reuse

QUESTIONS?

I appreciate any questions I’m sure I’m missing something so please feel free to comment This is a group effort and will not succeed without your

assistance

LINKS

jmeter http://jmeter.apache.org/ http://jmeter-plugins.org/

The Intern https://github.com/theintern/intern https://theintern.github.io/

OWASP ZAP https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project

logstash https://www.elastic.co/products/logstash

new relic http://newrelic.com/

LINKS

jira https://www.atlassian.com/software/jira?_mid=d3728664bd63df52b9d9e1e74df5f38d&gclid=COrRgfCSvMYCFQXJtAodOu0Pig

browserstack https://www.browserstack.com

jenkins plugins

The ZAPPER for OWASP ZAP https://wiki.jenkins-ci.org/display/JENKINS/Zapper+Plugin

jacoco https://wiki.jenkins-ci.org/display/JENKINS/JaCoCo+Plugin

Cobertura https://wiki.jenkins-ci.org/display/JENKINS/Cobertura+Plugin

Zephyr https://wiki.jenkins-ci.org/display/JENKINS/Zephyr+Enterprise+Test+Management+Plugin

Jira https://wiki.jenkins-ci.org/display/JENKINS/JIRA+Plugin https://wiki.jenkins-ci.org/display/JENKINS/Jira+Issue+Updater+Plugin

LINKS

blazemeter https://wiki.jenkins-ci.org/display/JENKINS/BlazeMeter+Plugin

logstash https://wiki.jenkins-ci.org/display/JENKINS/Logstash+Plugin

newrelic https://wiki.jenkins-ci.org/display/JENKINS/New+Relic+Deployment+Notifier+Plugin

selenium https://wiki.jenkins-ci.org/display/JENKINS/TestingBot+Selenium+Plugin https://wiki.jenkins-ci.org/display/JENKINS/Selenium+Plugin