Software Testing

How has agile changed

the game?

Karen Greaves

Why?

• Building the system right

• Compare against a specificationValidation

• Building the right system

• Compare against customer needsVerification

Cost of fixing a defect

Test Early

HOWTO: Test Early

Testers involved in Sprint PlanningTesters work from day 1 of the sprintDone includes testing & fixingDon’t let developers get aheadTeam signs off each story

Cost vs Benefit of Automation

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250

50

100

150

200

250

300

Automated Testing

Manual Testing

Number of Test Executions

Rela

tive C

ost

Automate

HOWTO: AutomationMake automation part of every storyEveryone runs automated testsQuick to run & interpretFlexible to runDon’t automate once off testsAutomate things you do everydayAutomate sanity tests

Defect Yield

High YieldAverage YieldLow Yield

Time Testing

Defe

cts

Fou

nd

Test Smarter

HOWTO: Test Smarter

White or grey box testingUnderstand where bugs could existTrust developersStatic testingUse fundamental techniques

Test Fundamentals

Pair wise / All pairs testingEquivalence partitioningBoundary case analysisDependency maps for regressionSeparate functional testing from load testing

TrackingWhen should you log a bug?How many defects states?Priority and severity?The importance of classifying bugsClean out your defect system regularly

Test Reporting

System status at a glance Test depthDefect trends

Working software over comprehensive documentation

Technical stuff you need to know

Version ControlBuild ProcessInstallers and UpgradesEnvironments

Implications of agile

Build the right systemTest EarlyAutomate TestingTest SmarterQuality is everyone’s concern