secrets to testing & automation at digital speed
TRANSCRIPT
--------------------------------------------------------------
© 2015, All Rights Reserved. www.ness-ses.com
www.ness-ses.com
© 2015, All Rights Reserved.
Secrets to Testing &
Automation
at Digital Speed
--------------------------------------------------------------
© 2015, All Rights Reserved. www.ness-ses.com
About the Presenters
Jean-Paul de VooghtDirector Client Solutions
Ness Software Engineering Services
Jyoti MandhareProgram Director
Ness Software Engineering Services
--------------------------------------------------------------
© 2015, All Rights Reserved. www.ness-ses.com
About Ness Software Engineering Services
OwnershipThe Rohatyn
Group
Global Presence20 countries
60+ client centers
ApproachTransparent.
Responsive.
Collaborative.
HeritageProduct
Engineering in
our DNA
ScaleOver 3,000
connected
engineers
ExecutionSeamless delivery
onsite, offsite,
nearshore and
offshore
VisibilityReal-time
performance
toolset, metrics and
reporting
InnovationDesign &
Innovation Lab.
Rapid prototyping
--------------------------------------------------------------
© 2015, All Rights Reserved. www.ness-ses.com
Recurrent Themes from the Field
Frequent Releases
While Minimizing Costs
Fast Test Cycles
With High Coverage
--------------------------------------------------------------
© 2015, All Rights Reserved. www.ness-ses.com
Test Automation Pyramid
Frequent Quality Releases
Unit /
Component
Tests
Acceptance Tests
(API Layer)
GUI Tests
Manual Tests
Based on test automation pyramid concept developed by Mike Cohn, author of Succeeding with Agile,
November 2009.
Before AfterAutomated
Test Count
Trunk Gating
Solution based on
Git/Gerrit
4 weeks 2 hours 300
Intelligent Transport
System Platform2 months 10 hours 8,000
Digital Marketing
Platform for Auto
Dealers
10 days 1 day 24,000
Learning
Management System45 days 10 hours 4,500
--------------------------------------------------------------
© 2015, All Rights Reserved. www.ness-ses.com
ExploratoryTesting
End to EndGUI Tests
Integration Tests
Workflow Tests
Business LogicAcceptance Tests
Unit Tests
Embarking on a Transformation Journey
End-to-End GUI Tests
Integration Tests
Unit
Tests
80%
15%
5%
Manual Tests
20%
80%
Manual
Tests
--------------------------------------------------------------
© 2015, All Rights Reserved. www.ness-ses.com
Maximizing Your Automation ROI
Transformation: Five Key Elements
Define Test StrategyEstablish Test
Framework
Define Automation Management Process
Integrated Engineering Environment
Team Organization
--------------------------------------------------------------
© 2015, All Rights Reserved. www.ness-ses.com
First: Define the Test Strategy
Transformation Strategy at SaaS Vendor
Phased Approach
• Progressively build up test
base while enhancing/
introducing a feature and
making the product more
testable
• Transition from ‘hand off’ from
Dev to QA to Dev ownership
of validation, QA doing
business process-level
validation & feature
certification
Backlog Churn Responsibility Shift
Dev QA
--------------------------------------------------------------
© 2015, All Rights Reserved. www.ness-ses.com
Single Source of Truth
Achieving In-sprint Automation
Second: Establish Test Framework
Sprint 1 Sprint 2+
Automation
Sprint 1
Sprint 3+
Automation
Sprint 2
Sprint 2+
Automation
Sprint 2
Sprint 3+
Automation
Sprint 3
Sprint 1+
Automation
Sprint 1
Lagged Automation Testing In-Sprint Automation Testing
--------------------------------------------------------------
© 2015, All Rights Reserved. www.ness-ses.com
Assigning Tests to Regression Suites
Third: Define the Automation Process
--------------------------------------------------------------
© 2015, All Rights Reserved. www.ness-ses.com
Fourth & Fifth: Set Engineering Environment &
Team Organization
Integrated Engineering Environment Distributed Scrum Team Formation
--------------------------------------------------------------
© 2015, All Rights Reserved. www.ness-ses.com
Fighting Recurrent Release Pains as “Shifting the
Burden” Systems Archetype
Concluding Remarks
One problem symptom,
two solutions, one
adverse side-effect.
Strengthening the
system’s
ability to solve its
problem
requires understanding
and time.
Based on Peter Senge’s Fifth Discipline, The Art & Practice of Learning
Organizations, 1990.
2011 2012 2013 2014
QA
Productivit
y
35% 40% 50% 55%
Automation
ROI54% 75% 79% 82%
Cost
Saving
over Total
Eng.
Spend
14% 30% 25% 30%
Real ROI Improvement Profile :
1
2
--------------------------------------------------------------
© 2015, All Rights Reserved. www.ness-ses.com
Thank You.
For more information, contact:
Jean-Paul de Vooght
Director Client Solutions
ness-ses.com
--------------------------------------------------------------
© 2015, All Rights Reserved. www.ness-ses.com
תודה
धन्यवादdanke
ďakujem
Thank you
mulțumesc