Continuous Delivery with Smarter Testing

Me

Dave Hart – Lead Developer in Test @ IBM

david.hart@uk.ibm.com

@DeveloperInTest

http://developerintest.blogspot.com/

Agenda

• Continuous delivery

• Smarter story level testing

• Smarter product level testing

• Automation

• One team

• Stories +

• Conclusion

• Questions

Continuous Delivery

The Agile ideal

The twelve principles of agile software:

1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

7. Working software is the primary measure of progress.

Where we can fall short

Test in agile teams– Can still be silo’d

– Let test lag behind development

Story focused– Our week by week work is blinkered – strong focus on individual

stories

– Lots of testing not necessarily story related gets neglected• System Quality Testing (Installers, performance, deployment,

compatibility, stress, release hardening, globalisation etc etc)

What can we do?

• Develop a ‘Test Runway’

• Be a team

Smarter TestingStory Level

Smarter testing – story level

• Start testing earlier– Requirements and design– TDD, BDD– Alongside development

Smarter testing – story level

• Minimise test documentation– Guidelines over precise steps– Completed stories and running tests primary

measure of testing performed

• Know your features– Understand the purpose of the story deliverable– Know how it fits in the wider product– Be involved in the design

• Prioritise the scenarios– Attempt to understand how a feature will be used– Break it down into common, occasional and possibly

never scenarios, and prioritise accordingly

Smarter testing – story level

• Reduce test duplication– Know who is testing what– What is tested at the unit level, integration

system etc

• Utilise test metrics– Coverage statistics– Defect route cause analysis– In sprint defects raised

Smarter TestingProduct Level

Smarter testing – product level

• Minimise Test documentation– Approach and strategy over detailed plan– Guidelines for common areas– The backlog state is our measure of progress

• Know your customers– Understand how and why they use your

software– Understand the day to day routes through

your software– Prioritise the bits that are important over the

rarely used (test for the 80%)

Smarter testing – product level

• Have an idea of all the parts of your release

• Know what is tested where– Reduce duplication– Identify bits that fall between teams/stories– Define a test strategy / approach so we have

an idea of the confidence we can derive from the various testing activities

• Choose your test technologies carefully

Smarter testing – an example

Web Service Application

Client

Unit

UnitUnit

Unit

Unit

Unit

Component

Component

Unit

Unit

Unit

Component

Smarter Testing - Automation

Automation

• Regression, regression, regression– Continuous feedback of continued operation– Run often (continuous integration builds ideal)

• Build in testability– At code level– At UI level– Architect and model for code re-use

• Automate what you can

Automation

Diagram from Agile Testing – Lisa Crispin & Janet Gregory

One Team to Rule them All

One team

• Commit as a team, deliver as a team

• Support specialism's

• Develop a quality culture

Stories +

Stories + - infrastructure

Infrastructure is Key!

Build systems– Fast feedback– Track failures back to source easily– Configurable triggers

Stories+ - DevOps

• Automate your deployments– Link it into your build system– Make it available– Consider build frequency

Web ServicesApplication

CustomCode

Web Server

HardwareConfiguration

Environment

CustomerConfiguration

Deployment

Stories + - DevOps

• Enables– Cutting edge software for demonstration– Fast, usable feedback – at all levels

Stories + - DevOps

Testability

CodeCodeTestsCodeTests

Deployment

ConfigurationTesting

PerformanceTesting

User testingAPI testingGlobalisation

Testing

Enables

Smarter Testing – An example

Web Service Application

Client

Unit

UnitUnit

Unit

Unit

Unit

Component

Component

Unit

UnitUnit

Component

DeployedSmoke test

Deploy

PerformanceFully System

UI test

Conclusions

Conclusions

• Start at the design

• Automation and regression

• Testing Runway

– Step back– Inspect– Plan

• Know your product

• Invest

Conclusions

Recommended Reading

• Agile Software Requirements – Dean Leffingwell

• Agile Testing – Lisa Crispin, Janet Gregory

• Continuous Delivery – Jez Humble, David Farley

• Cartoon tester: http://cartoontester.blogspot.com/

Any Questions?