xl testview overview

Introducing XL TestView

Test Analytics and Decision Support across all testing tools in your environment

August 2015

2 Copyright 2015. Confidential – Distribution prohibited without permission

Agenda

Key Test Automation Challenges and Questions

Positioning XL TestView

Demo


SPECIFY INTEGRATE RELEASEREGRESSION USER ACCEPTANCEDESIGN BUILD TEST

VALUE CHAINCONCEPT CASH

TestEffort

Transformation to Test Automation

Too many (testing) hours are spent late in the process, and there are too many repetitive manual tasks.

This screams for automation!


SPECIFY INTEGRATE RELEASEREGRESSION USER ACCEPTANCEDESIGN BUILD TEST

TestEffort

AcceptanceDriven Testing

“I add value by sharpening the

acceptance criteria of requested features”

Development = TestTest = Development

“Testing is transforming to an automation mindset and skill instead of

a separate activity”

Automate ALL

“Test automation serves as the safety net for my new functionality: I focus on

running the appropriate tests continuously during the iterations”

USER ACCEPTANCE

VALUE CHAINCONCEPT CASH

Transformation to Test Automation


An Example Project

At the end, the code produced by all three teams need to be tested together in order to make a go-live decision

A project responsible for developing a new banking system, with three teams working on the application

One front-end team responsible for developing

front-end web site

One application team responsible for developing

business logic (e.g. ‘savings’)

One back-end team responsible for developing database / storage layer

Each team will also test the parts of the banking system they are developing


Key Test (Automation) Challenges - Details

Many test tools exist for each of the test levels, but not a single answer to:

Vanity Check

TTA

ALM stack AppDynamicsCode Tester for Oracle

Arachnid

“Good enough to go live?”


Key Test (Automation) Challenges - Details

Requirements coverage is

not available“Did we test everything?”

Minimize the mean time to repair

“Support for failure analysis” Quality control of the tests

themselves and maintenance of testware

Running all tests all the time takes too long, feedback is

too late

Thousands of tests makes

test sets hard to manage“Where is my subset?”

“What tests add most value, what tests are superfluous?”

“When should we run which tests?”


Key Test Automation Questions

How do you combine, visualize and analyze all relevant test results and other data to support fast, accurate go/no-go decisions?

How do you actively manage rapidly growingtest sets to ensure maximum relevance, efficiency and value for money?

How do you flexibly manage, orchestrate and invoke an ever-changing set of test tools to support changing definitions of quality and risk?

1

2

3


Test Analytics & Decision Support Across All Test Tools

Introducing…

The first test results management and analysis tool that allows you to define and execute tests across your full

spectrum of test tools, then aggregate, and visualize results to provide insight into your application’s quality.


Introducing XL TestView

XL TestView serves as

The central quality nervous system within your continuous delivery initiative

Quality assurance hub across your test tools, including history and trend reports

XL TestView provides

Integrated test results across your (open source and enterprise) test tools

History and ‘flakiness’: what tests failed today (but passed yesterday, and failed the day before)?

Duration: is my test set run is getting slower, then I may need to take measures…

Expert advice: what tests add most value?


Back to Our Example Project

At the end, the results of all three teams need to be tested together in order to make a go-live decision

A project responsible for developing a new banking system, with three teams working on the application

One front-end team responsible for developing

front-end web site

One application team responsible for developing

business logic (e.g. ‘savings’)

One back-end team responsible for developing database / storage layer

Each team will also test the parts of the banking system they are developing


Check-in Build Unit TestsDeploy to Test

EnvFunctional Tests

CD Pipeline for the Application with Test JobsDo the low-level functions

work as intended?Check the business

functionality

• 375 unit tests in JUnit

• 100 unit tests in DelphiUnit

e.g.: • 459 FitNesse tests

• 321 Cucumber tests (savings component)

e.g.:


CD Pipeline for the Application with Test Jobs (2)

Deploy to PERF Env

Performance Tests

Deploy to ACC Env

End 2 End Acceptance Tests

Deploy to PROD Env

Deploy database to ACC Env

Deploy frontend to ACC Env

How does the system behave when 1000’s of users access the

functionality simultaneously?

Does everything work well together? Run the most common user journeys

for the banking application.

• 10 Gatling scenarios each performing 10 simulations

• 5 JMeter plans

• 23 HP LoadRunner scripts

e.g.: • 30 Cucumber tests

e.g.:


Summary of Test Specificationsfor Our Example Project


Key Features of XL TestView

Visualize and analyze test results,

trends, and correlation in real-

time

Collect results from – and control – all major test tools

Automate go/no-go decisions

Optimize testing for value and speed

Install and configure in minutes


Save time evaluating test resultsand release status

Key Benefits of XL TestView

Clearly see software quality trends; does it get better or worse?

Don’t fly blind: Data-driven decisions lead tohigher quality code


Key Benefits of XL TestView (2)

Automate your go/no-go decisionsbased on quantifiable data and set criteria

Pull the risk out of your software lifecycle/testing: know what you are putting into production

Optimize and reduce your testing efforts by allowing you to focus resources where they are most needed

XL TestView Demo


Test Results in Perspective


More Test Results! ;-)


Aligning Results for Go-Live Recommendations


Integrating XL TestView with Jenkins

It’s as easy as one – two – three…

1. Create placeholder test specifications in XL TestView

2. Add the post-build activity ‘Send results to XL TestView’ and select the appropriate test specification

3. Make sense of all test results in XL TestView through dashboards and qualifications


Never Change a (Winning) Jenkins Setup


Pricing XL TestView

Enterprise

Price per project Request quote

Dashboards per project Unlimited

Jenkins integration Yes

LDAP integration Yes

Support 5x8 or 24x7

User interface Yes

HTTP API Yes

Standard reports Yes

Flakiness report Yes

Diff report Yes

OSS Test Tool Plugins Yes

Enterprise Test Tool Plugins Yes, rate sheet


Be responsive to customer needs and out-deliver competition...

XL TestView helps you deliver features that work

to your customers – faster.


See for yourself…

Learn more: https://xebialabs.com/products/xl-testview/

Free trial: https://xebialabs.com/download/xl-testview/

https://xebialabs.com/products/xl-testview/

https://xebialabs.com/products/xl-testview/

https://xebialabs.com/download/xl-test/




xl testview overview

Software

test levels

growing test sets

xltestview test analytics

relevant test results

confidential distribution

automation mindset

appropriate tests

testing tools