virtualization to improve speed and increase quality
TRANSCRIPT
T20
Test Techniques
5/7/2015 3:00:00 PM
Virtualization to Improve Speed
and Increase Quality
Presented by:
Clint Sprauve
HP
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
Clint Sprauve
HP
Clint Sprauve is the senior product marketing manager for HP Software Functional Testing.
Previously, Clint was the director of product strategy for the Borland Solutions division of Micro
Focus, where he created Borland’s mobile strategy for functional test automation. With more
than twenty years of experience in the software development and quality assurance industry,
Clint has served as the senior product marketing manager for the Silk Testing Solutions at
Borland Software and Segue Software, and as a senior technical sales engineer for both
companies. He has been an independent consultant, specializing in test management and test
automation.
4/30/2015
1
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
VirtualizationImprove Speed & Increase Quality
Clint Sprauve – Product Marketing Manager, Functional Testing
Todd DeCapua – Thought Leadership, Service & Network Virtualization
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2
Agenda
Welcome!
- The IT world we live in today
- The Lifecycle Virtualization Story
- Network and Services
- A real customer case-study
- Q+A
4/30/2015
2
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The IT world we live in today�
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.4
It’s a mobile world
+
4without
compromising
time to market
4that actually
scale and perform
4and truly leverage
the power of big data
+
Test Across Multiple Devices Deliver Beautiful Experiences Accelerate Application Delivery7 000 000 000 mobile subscriptions WW (2014)
40 000 000 000+app downloads on Apple alone
$1.3T Mobile technology spend by 2015 (35% of the technology economy)
4/30/2015
3
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Essential evolutionHigh quality application performance is essential to success
Poorly performing applications and
software updates that anger and
alienate are not an option.
It’s time for an aggressive approach.
Virtualization of users, services,
and networks to create realistic test
conditions, and accurate results.
D.I.Y.
TESTING
FREEWAR
E
SIMULATIO
N
HP SERVICE AND NETWORK
VIRTUALIZATION
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Networks
A constantly challenging courseMany challenges impede the delivery of high-quality and stable
applicationsServices
Agile
Data center
moves
Composite
appsERP
Security
Cloud
E2E
testing
Mobile
4/30/2015
4
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The cost of poor app performanceYour business can perform no better than its applications
Lost revenue
Lost brand
reputation
Lost
customer
s
Lost
competitive
advantage
61%of users are unlikely to return to a
site if they’d had trouble viewing it
on a mobile device
70%of mobile transaction
response time stems
from the network
37%of consumers will
shop elsewhere if a
mobile site or app
fails to load in 3
secs.
1 sec.Is how long mobile
devices have to respond to
user input, in order to keep
the user engaged.
Poor performance drives
customers away:
© Copyright 2014 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The essentials for high-quality appsLifecycle Virtualization – a solution offering only available from HP
User Virtualize users and transactions to
execute as real world users would
exercise the systems
ServiceVirtualize infrastructure and
service interactions to simulate
E2E business processes
DataVirtualize production
data and transaction
responses to emulate
application request
and responses
NetworkVirtualize network
conditions to emulate
production architecture and
real user conditions
4/30/2015
5
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The new style of ITProblems lie on both sides of the application delivery lifecycle
* Source: “The Data Explosion in 2014 Minute-by-Minute” by ACI Information Group.
http://aci.info/2014/07/12/the-data-explosion-in-2014-minute-by-minute-infographic/
MODERN APPLICATION DEVELOPMENT
Applications not releasing quickly
enough
Low quality releases
DELIVER
Y
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Deliver amazing user experiences
MODERN APPLICATION DEVELOPMENT
Reduce costs
Increase customer attraction/retention
Increase the value of your brand
Get to market faster
The new style of ITMega-trends create demands on modern applications
A data explosion in 2015�
* Source: “Enterprise Mobile Facts You Need to Know in 2015” by App Data Room.
http://appdataroom.com/enterprise-mobile-facts-need-know-2015/
Mobile
Cloud
45%of mobile enterprise app initiatives
will be delayed or go over budget
due to infrastructure difficulties
25%IT software budget dedicated to
mobile app development,
deployment and management
by 2017
35%of large enterprises will use
mobile app development platforms
1 in 5of these will be specifically
for enterprise use
381 million tabletswill be sold worldwide
Dev Ops
Agile
4/30/2015
6
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
MODERN APPLICATION DEVELOPMENT
The new style of ITMany challenges impede the delivery of high-quality applications
Networks
Data center
moves
ERP
E2E testing
Agile
Composite apps
Security Services
Lost brand Lost brand reputationreputation
Lost customersLost customers
Lost revenueLost revenue
Lost competitive Lost competitive advantageadvantage
Reduce costs
Increase customer attraction/retention
Increase the value of your brand
Get to market fasterComplicated,
unavailable
or expensive testing
Deliver amazing user experiences
Poor quality
release
Poor
performance
Delayed release
schedule
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Hitting the roadblocksConstraints plague application development and testing
Third Party
Services
Mobile
Customer
Corp.
Customer
Mobile
Employee
HQ
Employe
e
Mobile
Partner
Remote Site
EmployeeUser & UI
Enterprise Mobile Reporting Website Warehouse Partner
Application & Services
SOAP AdapterMQ Adapter SOAP
Adapter
RFC /
IDOC
Adapter
Enterprise Service Bus & Integration
Mobile
ComponentsMainframe
Distributed
Services
ERP
(SAP, Peoplesoft, etc.)
Systems & Backend
APPLICATION
LAYER:
TooCostly
Not Available
Not Available
TooCostly
Not Available
How?
Limited
How?
How?
How?
Limited
4/30/2015
7
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The lifecycle virtualization story
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Essentials of Lifecycle VirtualizationThe four foundations of virtualization
DataVirtualize production data and
transaction responses to emulate
application request and
responses
NetworkVirtualize network conditions
to emulate production
architecture and real user
conditions
DEV TEST OPSMODERN APPLICATION
DEVELOPMENT
UserVirtualize users and
transactions to execute as real
world users would exercise the
systems
ServiceVirtualize infrastructure and
service interactions to simulate
E2E business processes
4/30/2015
8
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Without virtualization, testing is incompleteReality is rarely local. Why should your test be?
DATA CENTER
Local Virtual
Users
• Load generators
• Local switch
Application
under test
Hi-Speed
LAN
Hi-Speed
WiFiTablets
Smartphones
LOCAL TEST ENVIRONMENT
Branch office
Residential
Mobile
anywhere
Road warrior
Mobile
workforce
SITES
DEVICES
FIRE
WAL
L
NETWOR
K
APPS
Business
systems
Online
B2C
Mobile
Corporate
Mobile
Consume
r
Mobile
Consume
r
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Service Virtualization (SV)Composite applications need virtualized services
SERVICE
VIRTUALIZATION
WORLD-CLASS APPLICATION
services can support a single composite
application.
With HP Virtualization software, virtual services can be created to replace these targeted
services.
38Up to
4/30/2015
9
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
SERVICE
VIRTUALIZATION
WORLD-CLASS APPLICATION
Service Virtualization (SV)Composite applications need virtualized services
Developers and testers can perform functional
or performance testing even when:
• Real services are not available
• Data access is restricted
• Data is difficult to attain
• Services are not suitable for the particular test
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Service Virtualization: The ImpactThe cost of not using virtualized services
Delayed or lack of real
quality and performance
testing
Delayed production
waiting for access to
services
Delayed development
Delayed delivery times
Expensive test
hardware
and back-end systems
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
4/30/2015
10
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Network Virtualization (NV)
WORLD-CLASS APPLICATION
NETWORK
VIRTUALIZATION
of mobile transaction response time is
spent on the network.
NV recreates mobile, cloud, internet and
corporate network conditions, in order to
test applications under real world network
conditions.
70%
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Network Virtualization: The ImpactWhat happens when you don’t take the network into account
No visibility into the performance that your users will
getFailure in transactions under
actual mobile network
conditions
1.2 1.34.3
1
12.2
5.2 7.42
22.319.2
27.4
2.3
39.2 37
59
2.8
0
10
20
30
40
50
60
70
Launch App Log In View Account Sign Out
Local User (No NV)
WiFi (Remote Branch)
4G
3G
NV can reveal as much as a 10x difference in
response time under mobile conditions
Customer dissatisfaction
Brand damage
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
4/30/2015
11
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Network Virtualization goes beyond the test
Discovery and capture
Identify and record real-world
network performance
conditions
Virtualization and test
Virtualize production network
conditions to accurately emulate
and test app performance
Analysis and Optimization
Analyze test results and
implement optimization strategies
to improve performance
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Lifecycle Virtualization has a role everywhereThe four essential components are involved across the entire app
architecture
Mainframe
Storage
DB
servers
Web services
App
servers
Web
servers
Load
balancers
Mobile components
Major
ISP
3rd party
cloud services
Content
delivery
networks
Mobile
carriers &
ISPsBrowsers
& devices
Customers
Distributed
services
Employee
s
User Virtualization
UV
UV
UV
Service Virtualization
SV
SV
SV SV SV
SV
SV
SV
Network Virtualization
NV
NV
NV
NV
NV
NV
NV
NV
NV
NV
NV
NV
NV
NV
Data Virtualization
DV
DVDV
DV
DV
DV
DV
DV
DV
DV
4/30/2015
12
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Lifecycle Virtualization Benefits are everywhereHelping the product and the players
Development
Test
Operations
Business
Enterprise
System
Architects
IT Security
• Functional
• Performance
• Security
User
• Infrastructure
• Service
• 3rd party component
Service
• Discovery & capture
• Emulate & test
• Analysis &
Optimization
Network
Faster time to market
Higher quality
Applications that delight
Increased brand value
WHO? ROLES WHAT? VIRTUALIZATION &
CAPABILITYHOW? IMPACT
• Front end
• In motion
• Back end
Data
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Wait for SAP
availibility
Coordinate
remote LG for
the test
Pilot release
modules 1 and
2
Field test
Lifecycle Virtualization delivers velocityAccelerating the process
Dev Module 1
Dev Module 2
Test 1 / 2 Perf Test 1 / 2 Release
Wait for Module 1 to be
complete
Wait for Module 1 and 2
to be complete
A typical application delivery schedule involves a lot of delays4
using Module 1 virtual svcs
with
real world conditions
Monitor User
Experience
Virtualization enables faster releases and
higher quality
4/30/2015
13
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Lifecycle Virtualization ResultsClient results are unbeatable
400+%ROI
50%reduction in application
transaction response time
10Ximprovements in mobile
transaction times
90
50-90%reduction in wait time
5Xdecreased effort to test
Less than
30%faster application delivery
45%increase in test coverage
and cycles in same time
EFFICIENCYPERFORMANCE VALUE
day break even
60%cost savings due to
faster simulation of
services
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
MODERN APPLICATION DEVELOPMENT
LV Customers are EverywhereSuccesses span industries and use cases
Successes:� Cost Savings: Reduction and avoidance
� Improved Brand Equity
� Improved services offerings
� Increased revenue
Gov’t / Public Sector
Financial Services
Healthcare
Higher Education
Power and Energy
� Increased performance
� Predictive analysis – performance incidents
� Realistic testing – network conditions
� Reduced delays - server, performance, network, test
Retail
Telcomm
Full end-to-end Testing
Enterprise Systems(e.g.: SAP, Oracle, SFDC, PPLSFT, etc)
Remote site Operations
Ecommerce
and POS
Systems
Data Center
Consolidation/Relocation
Performance CoE /
Performance Incident
Management Process
Release Automation /
Management and DevOps
4/30/2015
14
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
SV+NV= A complete solutionAll the information you need for truly accurate app testing4and more
Lost revenue
Lost brand
reputation
Lost
customer
s
Lost
competitive
advantage
Increased revenue
Strong brand
reputation
More customers
Industry leader
Avoiding the pitfalls of
poor performance4.
4and instead,
capitalizing on the new
style of IT
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Customer Case-Study
4/30/2015
15
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
A Retailer Real-Life ScenarioTracking the course
Custome
r login
Add to
cart
Checkou
t
Shipping
detail
Delivery
tracking
Shopping app
(web or mobile)
Purchas
e
received
A typical online purchase has
predictable landmarks in its
process4But4 the road from beginning to
end is hardly a direct route
QUIT SCENARIO
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
A Retailer Real-Life ScenarioTracking the course
Shopping app
(web or mobile)
Database
activity
Security
validation
Inventory
query
Database
activity
3rd party
ship
service
Logistical
services
Acc’t
info
3rd party
database
3rd party
reporting
Purchas
e
receivedAdd to
cart
Shipping
detail
Delivery
tracking
Custome
r login
Security
validation
3rd party
payment services
Account
authorization
Checkou
t
But4 the road from beginning to
end is hardly a direct route
QUIT SCENARIO
4/30/2015
16
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Database
activity
Security
validation
Inventory
query
Database
activity
3rd party
ship
service
Logistical
services
Acc’t
info
3rd party
database
3rd party
reporting
Security
validation
3rd party
payment services
Account
authorization
Shopping app
(web or mobile)
Purchas
e
receivedAdd to
cart
Shipping
detail
Delivery
tracking
Custome
r loginCheckou
t
A Retailer Real-Life ScenarioUse cases raise numerous questions at every step
Mobile
Performance
Functional
DevOps
Security
Development
How does it look
on the screen?
Is it secure?
What’s happening
with CPU / MEM?
OS’s?
Browsers
?
What
devices?
Do I have
the right
transactions?
Where are my
users coming
from?
What
network
connection?
What
happens
offline or
unavailable
?
Was the
experience within
brand?
USE CASE:
QUIT SCENARIO
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Mobile
Performance
USE CASE:
Database
activity
Security
validation
Inventory
query
Database
activity
3rd party
ship
service
Logistical
services
Acc’t
info
3rd party
database
3rd party
reporting
Security
validation
3rd party
payment services
Account
authorization
Shopping app
(web or mobile)
Purchas
e
receivedAdd to
cart
Shipping
detail
Delivery
tracking
Custome
r loginCheckou
t
A Retailer Real-Life ScenarioUse cases raise numerous questions at every step
User types?(mobile, web, other)
Scenarios
and
transactions
?
Geographies of
users?
Type of network
connections?
Production load
vs. test load?
Environment scale? How
about waiting for
environment?
Services
availability
and cost?
Does it meet future
needs? How much
capacity?
Is it secure
under load?
Open
sessions?
Functional
DevOps
Security
Development
QUIT SCENARIO
4/30/2015
17
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Database
activity
Security
validation
Inventory
query
Database
activity
3rd party
ship
service
Logistical
services
Acc’t
info
3rd party
database
3rd party
reporting
Security
validation
3rd party
payment services
Account
authorization
Shopping app
(web or mobile)
Purchas
e
receivedAdd to
cart
Shipping
detail
Delivery
tracking
Custome
r loginCheckou
t
A Retailer Real-Life ScenarioUse cases raise numerous questions at every step
Geographies of
users?
Flows?
User types?(mobile, web, other)
Services availability
and cost?Single vs.
Multiple?Type of
network
connections?
Environment
availability?
Is it secure?
Does it
function?
Functional
Development
USE CASE:
Performance
DevOps
Security
Mobile
QUIT SCENARIO
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Database
activity
Security
validation
Inventory
query
Database
activity
3rd party
ship
service
Logistical
services
Acc’t
info
3rd party
database
3rd party
reporting
Security
validation
3rd party
payment services
Account
authorization
Shopping app
(web or mobile)
Purchas
e
receivedAdd to
cart
Shipping
detail
Delivery
tracking
Custome
r loginCheckou
t
DevOps
USE CASE:
A Retailer Real-Life ScenarioUse cases raise numerous questions at every step
How many
versions to
support?
Release
rollback
test pass?
Operations break/fix
environment for
Prod incidents?
Capacity
for
login?
Data center config?(on premises, hybrid, cloud, live
live)
System
and
component
resiliency?
Integrations:
secure, tested,
working?
What are we monitoring for
KPI’s
in Prod and Pre-Prod?
What if 3rd party
dependency fails?
Performance
Functional
Security
Development
Mobile
QUIT SCENARIO
4/30/2015
18
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Database
activity
Security
validation
Inventory
query
Database
activity
3rd party
ship
service
Logistical
services
Acc’t
info
3rd party
database
3rd party
reporting
Security
validation
3rd party
payment services
Account
authorization
Shopping app
(web or mobile)
Purchas
e
receivedAdd to
cart
Shipping
detail
Delivery
tracking
Custome
r loginCheckou
t
A Retailer Real-Life ScenarioUse cases raise numerous questions at every step
End-to-end
performance and user
experience:
Too much security?
Regulatory:
Legal, 3rd party,
audit, other
security?
Data security:
Front, Motion,
Back
What if “authorization”
is compromised?
Impact from
network on
Security: XSS,
SQL Injection?
X Factor
authentication?
Internal
security
risks?Risks from
BYOD,
browsers,
etc?
Intrusion
detection?
Security
Performance
Functional
DevOps
Development
Mobile
USE CASE:
QUIT SCENARIO
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Development
Database
activity
Security
validation
Inventory
query
Database
activity
3rd party
ship
service
Logistical
services
Acc’t
info
3rd party
database
3rd party
reporting
Security
validation
3rd party
payment services
Account
authorization
Shopping app
(web or mobile)
Purchas
e
receivedAdd to
cart
Shipping
detail
Delivery
tracking
Custome
r loginCheckou
t
A Retailer Real-Life ScenarioUse cases raise numerous questions at every step
How are we
testing
security?
How to manage as a
service, and have
available?
Continuous Integration,
Development, Release?
How to manage as a
service, and have
available?How does it
perform?Development
data available?
Does it meet
Business Need
and Customer
Experience?
Target end user
device(s), interface(s),
portal(s)
Integrated
Development
Framework?Performance
Functional
DevOps
Security
Mobile
USE CASE:
QUIT SCENARIO
4/30/2015
19
© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank you