continuous delivery & testing madrid aftertest
TRANSCRIPT
@petemar5hallhttp://leansoftwareservices.com
Continuous DeliveryTesting challenges
@petemar5hallhttp://leansoftwareservices.com
Continuous Delivery is a software
development discipline where you build
software in such a way that the software
can be released to production at any time.
Its about building Quality in!
@petemar5hallhttp://leansoftwareservices.com
Do I work in a continuous
delivery model?• Your software is deployable
throughout its lifecycle
• Your team prioritises keeping
the software deployable over
working on new features
• Anybody can get fast,
automated feedback on the
production readiness of their
systems any time somebody
makes a change to them
• You can safely release fully
tested software at anytime
@petemar5hallhttp://leansoftwareservices.com
Why should you care about
this delivery model?
Apart from the obvious…
More and more companies are asking for this experience
More and more companies are trying to achieve some aspect of continuous delivery
@petemar5hallhttp://leansoftwareservices.com
What test challenges does
CD give us?
• Speed of delivery - How can I test everything?
• Emphasis on automation over use of manual
procedures - who does all this automation?
• How do I understand and get good test
coverage?
• How do you do regression testing?
@petemar5hallhttp://leansoftwareservices.com
In context
• My experience of shaping testing practices on a
large financial enterprise project trying to achieve
a continuous delivery model
@petemar5hallhttp://leansoftwareservices.com
The scene
A large financial institution
New product - Digital Payment Platform
@petemar5hallhttp://leansoftwareservices.com
Their Goal
Deliver updates to a digital payment platform every
week
- allowing them to compete by adding and changing
features on demand
- allowing them to safely and quickly react to issues
@petemar5hallhttp://leansoftwareservices.com
Initial State
• 9 Month release cycle
• 200+ development team (20+ teams)
• Poor agile implementation
• Lots of issues (communication, politics, etc)
• Numerous sequential test phases (dev, qa,
functional, performance, integration, business
acceptance, operational, pre production)
@petemar5hallhttp://leansoftwareservices.com
Dev
Pro
du
ctio
n
Business
Acceptance
PartnerRelease
Test
Operational
Test
Functional
Test
Platform
Integration
Test
QA
Performance
Test
• Very traditional waterfall model
• Some agile practices
Initial Test & Release Model
9 Months
@petemar5hallhttp://leansoftwareservices.com
Some of the initial problems
• Most tester were from a more traditional testing
background (scripted micro managed testing)
• Very long feedback loops between development and test
• Each test team in each phase had to re analyse
requirements
• Configuration on each test environment was different
• Test management team opposed to change
@petemar5hallhttp://leansoftwareservices.com
Proposed model
Dev Production1 Week
Business
AcceptanceMost test phases &
practices to
happen in
“development”
@petemar5hallhttp://leansoftwareservices.com
Approach
• Reduce the amount of features going through the release
process
• Build as much testing into the build process as possible
• Focus testers away from documentation and creating test
cases, apply a context driven approach
• Incorporate as many of the post development test
phases as possible
@petemar5hallhttp://leansoftwareservices.com
Approach (Cont.)
• Use crowd testing (bug bashes) with the whole
team
• Invest heavily in configuration management and
environment provisioning
• Bring business owners close to the teams
• Training on agile thinking and lean engineering
practices
@petemar5hallhttp://leansoftwareservices.com
“Quality is the responsibility of everybody.”
@petemar5hallhttp://leansoftwareservices.com
TEST
MANAGEMEN
T SAYS NO!
@petemar5hallhttp://leansoftwareservices.com
• How is it possible to do all that testing..
• what will my 20 test teams do?
• the quality will be terrible!
• Where is your independent testing?
@petemar5hallhttp://leansoftwareservices.com
How is it possible to do all
that testing..
• Education
• Demonstrations
• Invite the authors of
books & well known
testers to do workshops
@petemar5hallhttp://leansoftwareservices.com
What will my 20 teams do?
• Reduce the number of teams, you don’t need that
many testers
• Move your remaining testers into the
development teams
• Explain the release test team anti pattern
@petemar5hallhttp://leansoftwareservices.com
The quality will be terrible!
• I say prove it…
• Over 18 months, critical defects reduced from
>100 per release when we started to < 2 per
release at the end
@petemar5hallhttp://leansoftwareservices.com
Where is your independent
testing?
• What does that actually mean?
• What is the value of an independent test team?
@petemar5hallhttp://leansoftwareservices.com
Functional/Platfor
m Integration test
The compromise
Dev Production4 Weeks
Business
AcceptanceMost test phases &
practices to
happen in
development
@petemar5hallhttp://leansoftwareservices.com
To conclude
• We changed the culture
• We changed the perception of what testing means
• Remove scripted testing, and decreased test team
size
• Placed responsibility of quality on everybody
• We operated as a team
@petemar5hallhttp://leansoftwareservices.com
Useful resources
• https://leanpub.com/buildqualityin
• http://codedetective.blogspot.com
• http://continuousdelivery.com