devops: are you pushing bugs to your clients faster?

Parasoft Proprietary and Confidential 1

2015-10-16

DevOps: Are You Pushing Bugs

to Your Clients Faster?

Wayne AriolaChief Strategy Officer - Parasoft

Parasoft Proprietary and Confidential 2Parasoft Proprietary and Confidential 2

Re-Evaluate the Cost of Software Quality

http://alm.parasoft.com/continuoustestingbook

http://alm.parasoft.com/continuoustestingbook


2015 Non-Functional Requirements Survey

Architect15%

Business Analyst/Product

Owner6%

Developer/Engineer32%IT Ops

4%

Senior Management

17%

Tester26%

OF 780 RespondentsWhat best describes your role in your organization?


Does your organization measure or monitor compliance to NON-functional requirements?

No56%

Yes44%




59%

53%

49%

43%

34%

0% 20% 40% 60% 80% 100%

Waterfall

Hybrid

Iterative

Agile

Agile-ish

‘Yes,’ My Organization Monitors Compliance to NON-Functional Requirements (NFRs)


From Automated to Continuous Testing


Testing is a Bottleneck

Basic Tasks

Skilled Tasks

Process Oriented

Business Views

Fully Automated Provisioning

End-to-End Tasks

Data Analytics

Organic Systems

Automation is Incremental

Effi

cien

cy

Software Quality Processes


Fixed Fixed

Impact of Test in the Evolving SDLC

Plan Develop Build Test Release

Test as a time-boxed event is the prevailing method

“Are we done testing?”


Impact of Test in the Evolving SDLC

Plan Develop

Bu

ildTe

st

Release

Bu

ildTe

st

Bu

ildTe

st

Bu

ildTe

st

Bu

ildTe

st

Bu

ildTe

st

Bu

ildTe

st

Bu

ildTe

st

Bu

ildTe

st

Bu

ildTe

st

Bu

ildTe

st

“Does the release candidate have an acceptable level of risk?”


Software Drives Innovation

The Cost of

Quality

associated with

software has

shifted

dramatically

Software has

shifted from

process enabler to

business

differentiator

SDLC Speed will

be the difference

between a first

mover and a

follower

Switching costs

associated with

software are

dramatically lower

Parasoft Proprietary and Confidential 11

Software Failures = Headlines

Financial Airlines/Aero Government Media Internet/Tel

Automotive Technology Retail

http://aws.amazon.com/

http://aws.amazon.com/


Software Failures = Headlines 2014

-$2.3 Bn-3.75%

The day of the announcement companies lost an average

of shareholder value

Software failures make headline news—eroding customer confidence, shareholder value and brand equity


Escalating Cost of Failure

-3.12%-3.37%

-3.75%-4.06%y = -0.0032x - 0.0278

-4.50%

-4.00%

-3.50%

-3.00%

-2.50%

-2.00%

-1.50%

-1.00%

-0.50%

0.00%

2012 2013 2014 2015 (est.)

Impact on Market Capitalization


Software Failures = Headlines

821,0001,080,000

451,000

1,400,000

896,000

1,550,000

5,500,000

650,000

3,490,000

2,240,000

0

1,000,000

2,000,000

3,000,000

4,000,000

5,000,000

6,000,000

Cloud Services Airline Bank Software Retail

1st Event 2nd Event

Market does not forget— news about failure increase and average of 167%


Building a stronger American - Sometimes

-5.70%-1.9B


United Airlines ‘Glitch’

"Airlines are flying computers," said

industry analyst Henry Harteveldt.

"Increased reliance on technology has

enabled (airlines) to become a much

more successful and efficient

business, and that also creates an

exposure."

-3.61%-$750M


Let’s Reverse the Game…Spot the Failure

-7.4B-15%


The Cost of Software Failure - Sony

http://www.reuters.com/article/2011/04/26/us-sony-stoldendata-idUSTRE73P6WB20110426

http://www.digitaltrends.com/gaming/sony-fined-almost-400000-for-2011-playstation-security-breach/

Mark

et

Valu

e

Event 15 Days 30 Days

-22% -33% -30%-19% -11% -12%

Cumulative Loss = $18B

http://www.reuters.com/article/2011/04/26/us-sony-stoldendata-idUSTRE73P6WB20110426

http://www.digitaltrends.com/gaming/sony-fined-almost-400000-for-2011-playstation-security-breach/


We must Re-invent Test

The penalty for faulty software is increasing

Brand erosion

All industries at risk for total transformation

Technology is faster and more distributed

DevOps

Containers

Microservices


Next Generation Testing


Getting from “Automated” to “Continuous”

There is a big leap moving from “Automated” to “Continuous”

Business expectations clearly defined Business risks identified

Per application, team, release

Defects automatically prioritized versus business drivers No defect left behind

Mitigate business risks

Testing in ‘complete’ test environments continuously Leveraging simulation – Service Virtualization

Protecting the user experience “top-down”

Feedback loop for defect prevention Pattern definition

Defect prevention practices enhanced


Service Virtualization delivers a simulated test environment

to help development and testerswith earlier, faster

and more complete testing

Service Virtualization


So, What are you Virtualizing?

• Mainframe

• SOA / Web services

• Java

• .NET languages

• XML

• WSDL

• WADL

• UDDI

• WSIL

• SOAP

• PoX (Plain XML)

• REST

• JSON

• BPEL

• Web Applications

• Mobile Interfaces

• AJAX

• JSP

• JavaScript

• HTML

• CSS

• WS-*

• Standards

• MTOM(XOP) / MIME / DIME

• OAuth

• TCP/IP

• webMethods Broker

• webMethods IS

• HL7

• FTP

• HTTP 1.0

• HTTP 1.1

• HTTPS

• JMS

• IBM WebSphere MQ

• Sonic MQ

• RMI

• EJB

• SMTP

• Tibco Rendezvous

• .NET WCF

• ISO 8583

• EDI

• SAP

• custom

AUT


Access to Dev / Test Systems

Test Access

Simple Difficult

Configura

tion

Complex

Internal

Service

Internal

Application 3rd Party

Service

ERP

Mainframe

External

Application

External

Database

Internal

Database

Cloud

App

Message

Queue

ESB

Web

Server


Service Virtualization & Dependencies


Virtual Lab / Server

Virtualization

Configura

tion

Access

Access Dependent applications difficult Scheduling conflicts High access fees Geo-political boundaries 3rd party or partner applications

Configuration Complex to configure No control Limited variability Consumes test time

DifficultSimple

Ea

sy

Com

ple

x


The Total Cost of System Access

Service Virtualization dramatically reduces the cost of delivering

a dev / test environment

Hardware Savings

Access EfficiencyStaged w/ Virtual Test Lab

Staged Application Instance

System Behavior

Cost Only manage the functionality

you need Only manage data you need Low-Risk “disposable instance”

that can be accessed at anytime Dramatically reduce test

environment setup and management



Test Environment Management - Provisioning and Elastic Cloud Scale

Test Optimization

Environment Simulation (SV)

Auto Test Construction – Synthetic

The Next Generation Software Quality ‘System’

Continuous Assessment of RiskContinuous Verification of Requirements

Development Operations

APM – Real Time Data

Development (E1)

ProductionTest(E2)

Integration(E3)

Pre-Production(E4 – En)

Release Candidate

Development Tests


Transforming the Identification of Risk

From To

Causal Observations Probabilistic

Defect Documentation Most Likely Causes

Only Structured Data Structured and Unstructured

Filters Business Policies

Ad Hoc Automated

Tool Dependent Tool Independent

Business

Objectives


Parasoft Service Virtualization

Wayne Ariola

Chief Strategy Officer

Parasoft Corporation

[email protected]

Questions?

mailto:[email protected]