fueling devops with a testing trifecta: how the new world of testing is driving agile development

Fueling DevOps with a Testing Trifecta: How the New World of Testing is Driving Agile Development

DevOps: Continuous Delivery

@Scott_D_Edwards

#CAWorld

Scott Edwards

CA Technologies

DevTest Portfolio

Session Number

2 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

© 2015 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.

The content provided in this CA World 2015 presentation is intended for informational purposes only and does not form any type of

warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA.

For Informational Purposes Only

Terms of this Presentation


Abstract

In this session, see how CA’s DevTest portfolio arm your testing teams with three testing solutions built for Agile teams:

1. Test automation to create test cases right from requirements

2. Generating synthetic test data to be used on demand

3. Eliminating testing constraints by virtualizing every environment you need access to

Make testing the accelerator, not the hurdle, to fast application delivery with high quality.

Scott Edwards

CA Technologies

Director of Product Marketing


Agenda

TESTING IS BROKEN

CUSTOMER SATISFACTION AT THE HEART OF DIGITAL INITIATIVES

WHY QUALITY IS CRITIAL

THE NEED FOR AN AGILE TESTING TRIFECTA

HOW TO ACT NOW

1

2

3

4

5


What is More of a Threat to You?

External Breach Regular Software Update


outage was caused by a failure during a daily system refresh“


KNIGHT TRADING … a “software update accessed outdated code that suddenly made more than

$440 MILLIONin bad trades in

30 MINUTES

BNY MELLONproblem arose after bank and 3rd party software provider pushed out an upgrade…

Affecting over

1,200 MUTUAL FUNDS


Any Plan that Defers Testing To The End of a Release …

is BROKEN!

In an Agile / Continuous

Delivery World …

© 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD


For Embarking on Digital InitiativesFreeForm Study, “Exploiting the Software Advantage” 2015Top Drivers

Improved Customer Satisfaction

Better Workforce Productivity

Improved Customer Experience

Improved Competitive Differentiation

Better Agility/ Speed to Market

Opening up New Markets

49%

44%

41%

38%

37%

35%

TESTING CANNOT BE AN AFTERTHOUGHT!



Gatepoint Research Sept 2015147 IT Executives/Directors/Managers responded


87% Of respondents rate loss of customer satisfaction as potentially the most critical issue stemming from flubbed application delivery


What is Your Delivery Experience? What is Your Customer Experience?

12 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

Any Focus On Speed…Without QualityWill Result in

CONTINUOUS DELIVERY SUICIDE


Ignite Continuous DeliveryEliminate Your Development and Testing Barriers

Focus Area #1

DEVELOPER SPEED

Focus Area #2

AGILE TESTING TRIFECTA

Focus Area #3

IGNITING DEVOPS

Director of Development

KEY LAUNCH ASPECTS Service

Virtualization on Demand in Azure Marketplace

SV for Developers (vapi.ca.com)

KEY LAUNCH ASPECTS1. Test automation 2. Test data

management 3. Test constraint

removal

KEY LAUNCH ASPECTS DevOps Simulation

Game Hero Kit Benefit/ROI

Calculators

© 2015 CA. ALL RIGHTS RESERVED. FOR INTERNAL USE ONLY.

Dir. Of QA / Testing The Influencer


Agile Testing Trifecta … Critical for Continuous Delivery

1Test Automation

2

Test Data Management

3Test Constraint

Removal

TODAY’S TRADITIONAL TESTING METHODS INHIBIT DEVELOPMENT SPEED AND

IMPACT QUALITY

A new testing methodology,

combined with advanced

technology, is needed



Test Automation Solutions from CA1


“WHY DO SOFTWARE PROJECTS FAIL?”

Changing or poorly documented requirements*

#1 Reason

48%

*Source: IDC White Paper, sponsored by Application Developers Alliance, Application Developers Alliance Developer Insights Report: A Global Survey of Today's Developer, August 2015.

“…software developers would do well to invest more time and effort into understanding the requirements of the systems they plan to build”

Application Developers Alliance in partnership with IDC

Al HilwaIDC Program Director, Application Development Software


http://www.appdevelopersalliance.org/developer-insights-report-2015


CA Test Case OptimizerAutomatically Create Test Cases… Right from Requirements

REAL WORLD RESULTS

Cover 100% of requirements in the smallest number of tests

80% increase in test coverage

Reduce test cycles by 30% by automatically detecting any

redundant, outdated and duplicate test cases

Define & manage changing Software Requirements, mapping them to unambiguous visual flow charts

56%Build Quality into your Software from the Start

of software defects can be traced back to ambiguities in requirements1

Financial Services CompanyReduced Test cases from

326 to just 17!

1Test Automation


CA Application Test w/ Mobile Test AutomationComplete test coverage for Agile Practices … Mobile to Mainframe

Beneath the UI … lies

the complexity to be tackled

MORE THAN GUI TESTING! Invoke and verify the behavior of each layer that makes up an application

Automate Unit, Functional, Load, Regression & Performance Testing

KEY BENEFITS

Use for API, Rest API, Mobile, Swift, and SAP testing

Presentation layer test automation for Web & Mobile UIs

Reporting and collaboration capabilities that allow team

members to execute, monitor and analyze test results

Financial Service Company Run over 400 tests on a nightly basis

QA runs 460 tests whenever development releases a build

83% reduction in defects

1Test Automation


Test Data Management Solutions from CA2


Why Test Data

Anonymised live data may only cover around 30% of a bank’s test coverage requirements

£800 million worth of fines to businesses breaching financial principles in 2015 (Financial Conduct Authority)

Fines and regulations by the EU's General Data Protection Regulation starting in 2016

Businesses will have to notify customers – in plain language – what information about them is collected and how it is used

GDPR fines from the Information Commissioner’s Office (ICO) currently planned for 2 per cent of a business’s global revenue (capped at €100million)

FCA planning unlimited fines


CA Test Data ManagerGenerate synthetic test data for maximum coverage

REAL WORLD RESULTS Create millions of rows of high quality data in

just minutes

Reduce infrastructure costs by up to $50k per database

Reduce the time needed to generate synthetic test data by as much as 50%

1Bender RBT

63% of data breaches caused by human error in exposing sensitive records$5.4 Million = average cost of data breach incidents

2Test Data Mgmt.

Automatic creation of realistic test data sets required for testing

Designed to Eliminate the Risk of Data Breach

Large Credit Card Company 70% reduction in time and effort for getting access to

Test Data (Months to days)

80% reduction in time spent manually searching for the ‘right data’

70% reusability of Test Data Conditions for releases


New Test Data & Test Automation Integrations

TCO w/ Rally

TCO w/ HP ALM

TDM w/ Service Virtualization

TDM w/ Mainframe

Automatically convert user stories into the smallest set of test cases with 100% coverage, linked to the right data and expected results

Import existing requirements and test cases to an unambiguous, active flowchart, de-duplicate them, and reduce test cycles by up to 30%

Generate realistic virtual data which covers the full range of possible scenarios for effective service virtualization

High Performance Masking and Test Data Generation for IMS and DB2 using native utilities and operations, and automated data profiling.

2Test Data Mgmt.



Test Constraint Removal Solutions from CA3


Why Virtual Services?Today’s Hurdle to Innovation? Constraints across the SDLC

Incomplete Developmen

t

Data Constraints

System Constraints

CostConstraints

DevelopmentConstraints

80%Only

56%

79%

of teams experience delays in development and QA due to unavailable dependencies

of critical dependencies are unavailable when dev and test teams need them

of teams face prohibitive restrictions, time limits or access fees on needed 3rd party services

Source: voke Market Snapshot TM Report: Service Virtualization – January 2015

3Remove Testing

Constraints


Service Virtualization

the Wind tunnel of Software

Accelerate Application

Delivery

Simulate Dependent Systems

Provide 24/7 Access

Increase Developer Productivity

1 2 3

Data Constraints

System Constraints

CostConstraints

DevelopmentConstraints

How Service Virtualization Eliminates Testing Constraints

CA ServiceVirtualization

3Remove Testing

Constraints


“Shift-left” Testing

Without SV

WITH SV

Dev1

Dev2

Dev3

System Test

Integration

Performance Test

UAT

SystemDev1

Dev2

Dev1

Dev3

Dev2

Dev3 Integration Performance UAT…

Uncertain delivery schedule – defects persist until UAT

Typical composite app today waits for whole assembly to begin

With CA Service Virtualization quality effort moved earlier in the lifecycle

Reduction in SDLCFaster Delivery

“Went from finding 40% of our defects in the 3rd and final stage of testing to often finding ZERO. Service Virtualization turned this final test phase into a dress rehearsal for production.”

- AutoTrader http://bit.ly/AutoTrader-SV

3Remove Testing

Constraints

http://bit.ly/AutoTrader-SV


CA Continuous Application InsightRegression & API Testing … Reinvented

The CAI Difference Provides system of record behind the UI or API

Identifies dependencies between application

servers, databases and 3rd-party services

Turbocharge Your Service Virtualization

Global Technology Company Automated 3500 regression tests

Large Insurance Company Automatically generated 300-600 test cases

a day … w/ 90% coverage

Deep Insight to Diagnose Right Issues

Automatic Creation of Test &

Virtual Services

End-to-EndTransaction Discovery

3Remove Testing

Constraints


Test Automation from Requirements to Production

Development

Pre

-Pro

du

ctio

nQA/Test – Goal > 80% AutomationFunctional Regression Integration Build Mobile

CA Test Case OptimizerTest Less, Cover More

Test cases optimized Deduplicate manual tests Quantify costs and time for changes

CA Service VirtualizationUnconstrained Development and Test

Simulate missing data, functions, and services. Simulate APIs and third-party systems Turn contracts into assets

Requirements

CA Application Test with Mobile Any Test, Anywhere

API functional and non-functional testing Create the test once and then run across all targets Build level reporting for simplified analysis Support for Selenium and Appium

PerformanceLoad Testing

CA Test Data ManagerThe Right Data, in the Right Place at the Right Time

Eliminate manual data creation and masking Minimize security risks by not exposing PII Get data to testers and outsourcers fast

Shift Left

Unit Testing

CA Continuous Application InsightAutomate the Automation

Automatically generate 1000s of test cases Provides system of record for what is behind the UI or API Follow the transaction across all systems and get detailed error reporting Logging and production monitoring

Test cases created and validated against requirements.

Review mobile testing needs.

Mobile and load testing via cloud providers.


Your Next Steps


New Benefits Calculators


DevOps Simulation Game Experience…

Experience it @ CA World • See the Demo in Theater 3 area

Experience ½ Day Simulation Game! • Santa Clara, CA - December 3• Boulder, CO - December 10• Plano, TX - January 12• Chicago, IL – January 19• Atlanta, GA – January 26• New York, NY – February 2

Register Today at http://bit.ly/DevOpsSimXP_RS


Q & A


Recommended Sessions

SESSION # TITLE DATE/TIME

DO3T21S

Case Study: Nordstrom Succeeds in E-Commerce

with Innovative Performance Testing to Reduce

Cycle Times and Improve Quality

DEVOPS THEATER 3

Thursday at 11:30 am

DO4T17S

Case Study: T-Mobile’s DevOps and Continuous

Delivery Journey – Building a Foundation for a

Future Built for Agility

DEVOPS THEATER 4

Thursday at 1:00 Pm

DO3T24S

Business Transformation: Accelerate DevTest with

Microsoft AZURE™ and Service Virtualization on

Demand

DEVOPS THEATER 3

Thursday at 3:00 pm


For More Information

To learn more, please visit:

http://cainc.to/Nv2VOe

CA World ’15

http://cainc.to/Nv2VOe

http://bit.ly/1wbjjqX