agile testing faqs and mythbuster - software testing atlanta conference 2015
TRANSCRIPT
@yuvalyeret#STAC15
AgileTes5ngFAQsandMythbuster
YuvalYeret-AgileSparks#STAC15
1
@yuvalyeret#STAC15
Agile Team2
Agile Team1
Ongoing
Done Features Backlog
Develop
Feature/ Sprint
in Progress
Story-level Test & Fix
Deployment
Done Ongoing Done
Stories Backlog
Ongoing
Specify
Done
A
B
C
G
H
J K
L
M
H.6 H.0
H.1 H.2
H.3
H.4 H.5 D2
D3
T2 T1
D1 P1
H.7
T2
Implement Feature by Stories
How does an agile project/program looks like? (by the book…)
@yuvalyeret#STAC15
Butwhenthetypicaldev/testgroupmeetsAgile…
SeeKentBeck’sideaasdescribedbyMarkusGartnerathKp://www.shino.de/2010/11/04/soQware-g-forces-the-effects-of-accelera5on/
Agile Team2
Agile Team1
Ongoing
Done Features Backlog
Develop
Feature/ Sprint
in Progress
Story-level Test & Fix
Deployment
Done Ongoing Done
Stories Backlog
End of Release Testing
Ongoing Done Ongoing
Specify
Done
D
E
F
A
B
C
G
H
J K
L
M
H.6 H.0
H.1 H.2
H.3
H.4 H.5 D2
D3
T2 T1
D1 P1
H.7
T2
Implement Feature by Stories
UAT Regression
Performance Security
Functional Progression
Exploratory
ATDD
Auto
Long wait for the endgame
Real Life… Not so agile testing...
“Sprint is too short for everything we need to
achieve DONE”
“Let’s leave the serious testing for the release
testing phase”
The 2-level Test Strategy Pyramid – Story + Release-level
Platform Matrix
Real Network
@yuvalyeret#STAC15
The result - only a limited amount of feedback is early and effective
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9101112131415161718192021222324
done
donedone
40% 60%
Iden5fycoveragethathappensoutsidethesprint.Whichisthebiggestqualityriskiffoundlate?
Agile Team2
Agile Team1
Ongoing
Done FeaturesBacklog
Develop
Feature/Sprint
in Progress
Story-level Test & Fix
Deployment
Done Ongoing Done
Stories Backlog
Ready For
Feature Test
Test Feature-
level Ongoing Done
End of Release Testing
Ongoing Done Ongoing
Specify
Done
D
E T F T
T
A
B
C
G
H
J K
L
M
H.6 H.0
H.1 H.2
H.3
H.4 H.5 D2
D3
T2 T1
D1 P1
H.7 T2
Implement Feature by Stories
UAT
Regression
Performance Security
Functional Progression
Exploratory
Try - Slowly shifting left towards Continuous Quality and adding the Feature-level testing stage in the pipeline
Add Feature/Epic level testing
1. Add Feature/Epic/ Iteration level
testing.
2. left-shift more and more testing through automation, enabling teams using
environments/tools/knowhow and more.
@yuvalyeret#STAC15
Roleoftes5ngistofinddefectstoassurequality
ElaborateRequirements
TechnicalDesign
Coding/UnitTes5ng
TestExecu5on+Fixing Done
TestDesign TestAutoma5on
MythorBestPra
c5ce?
@yuvalyeret#STAC158
@yuvalyeret#STAC15
• Mostexpensiveautoma5ontodevelop,run&maintain,sominimize!!!
• MovemajorityofE2Etes5ngcoveragetoService/APIlayer
• QTP/UFT/Selenium/PerfectoMobile/etc.
UI
• “TheWorkhorse”ofenterpriseagiletes5ng
• Createdbytesters&developersonagileteamssupportedbyframeworks/guidancebyAutoma5onCoE
• soapUI,etc.
Acceptance(Service/API)
• LeverageAgileTeamsdevelopertes5ngtoreducecoverageneeds
• Abilitytoautoma5callydetect(throughcoveragetoolsetc.)whatiscovered
UnitTes5ng
Manual
hKp://www.mountaingoatsoQware.com/blog/the-forgoKen-layer-of-the-test-automa5on-pyramid
Ifweacceptthetestautoma5onpyramidwhatdoesitmeanforTesterSkills?
MythorBestPrac5ce?
10
Credit-UdiWeinberg–HPSoQware
@yuvalyeret#STAC15
BestPrac5ce–AgileTestersDRIVEquality
@yuvalyeret#STAC15
ATDD=AcceptanceTestDrivenDevelopment-BuildQualityIntoDesign–preven5ngdefects
ratherthanjustfindingthem
ElaborateRequirements TestDesign Technical
DesignCoding/Unit
Tes5ngTestExecu5on
+Fixing Done
1. ATDDThinkingUsetestscenariostoguidedesign(whichcanthenhappenusingTDD,ornot…)
hKp://www.agilesparks.com/test-first-reading-list
TestAutoma5on
ATDD/BDD-T
heEnterprise/
Real-worldwr
apperforTDD
…
@yuvalyeret#STAC15
Automation is a USEFUL skill for Testers/ Quality
Engineers. NOT their core skill!
It IS a core team
capability for enabling Continuous Quality
@yuvalyeret#STAC15
TheFuture?*
*AgileSparkspredic5on
2005 2010 2015 2020
SWDevelopers
Testers
SWEngineers
Automa5onExperts
TestEngineers/Experts
@yuvalyeret#STAC15
UserStory1578• AcceptanceCriteria:
– Scenario1– Scenario2– Scenario3– Scenario4– Scenario5– Scenario6– Scenario7– Scenario8– Scenario9– Scenario10– Scenario11
15
Whatdoyouthink?Isth
isagoodstory?
Whatdoyoulikeabout
it?
Howwouldyouimprove?
Agile Team2
Agile Team1
Ongoing
Done FeaturesBacklog
Develop
Feature/Sprint
in Progress
Story-level Test & Fix
Deployment
Done Ongoing Done
Stories Backlog
Ready For
Feature Test
Test Feature-
level Ongoing Done
End of Release Testing
Ongoing Done Ongoing
Specify
Done
D
E T F T
T
A
B
C
G
H
J K
L
M
H.6 H.0
H.1 H.2
H.3 H.4
H.5
D2
D3
T2
T1 H.7
T2
Implement Feature by Stories
Specify/Groom using Acceptance Tests
Backlog Grooming / Acceptance-Tests Specification Workshop • Identify acceptance criteria/tests for the next stories in the
backlog. • Use acceptance tests as guidance for slicing stories smaller to
enable more effective agile collaboration
@yuvalyeret#STAC15
Acceptance Tests/Mini-
Stories Stories MMFs Features
Feature
MMF H
Story H1 Test for H1
Mini-story H4
Mini-story H5
Story H2
Story H3
MMF I
MMF J
MMF K
Testers are key to effective
grooming together with the PO
and the Devs!!!
@yuvalyeret#STAC15
Ques5ons?• hKp://www.agilesparks.com/agile-tes5ng-reading-list• hKp://www.agilesparks.com/Services/AgileTes5ng• Bloggingatyuvalyeret.com• Emailmeques5ons/feedback/sugges5onsat
[email protected]• UnlockmyAgileTes5ngWorkshopinyourcity!
• TheseslideswillbeupathKp://www.slideshare.net/yyeretsoon.Myothertalksarealreadythere…
18
@yuvalyeret#STAC15
DevopsTools
Tradi5onalProcesses
IdealBatchw/oCon5nuousQuality
Investment
ReduceBatchSizeFurtherWITH
Greatautoma5onandleanerprocesses
Con5nuousQualityisajourneywithlotsofvalueateachstep