agile testing: solving the agilist\'s dilemma
DESCRIPTION
Star East 2009 - one hour describing why and how to use TDD and ATDD on an Agile teamTRANSCRIPT
Agile Testing:
Solving the Agilist’s
Dilemma
Star East
6 May 2009
11 May 2009 1© Rob Myers 2009
Rob Myers
• Teaching Agile courses since 2002
• XP/Agile coach since 1999
• Using TDD to build software since 1998
• Professional software development since 1986
• Blog: PowersOfTwo.agileInstitute.com
11 May 2009 2© Rob Myers 2009
The Agilist’s Dilemma
11 May 2009 © Rob Myers 2009 3
Debt
• Quality Debt
• Defects
• Design Debt
• Software design resists change
• Testing Debt
• Manual testing
11 May 2009 © Rob Myers 2009 4
11 May 2009 © Rob Myers 2009 5
DILBERT: © Scott Adams/Dist. by United Feature Syndicate, Inc.
Why Wait?
11 May 2009 © Rob Myers 2009 6
Code TestMonths
Code TestWeeks
Code TestDays
Code Test
Run Test
Instead…
11 May 2009 © Rob Myers 2009 7
CodeTestWrite Test Run Test Run Test Run Test
Manual Testing
Writing Test Cases
Record
&
Playback
TDD
Combine Advantages
11 May 2009 © Rob Myers 2009 8
Manual
Later First
Automated
“Move Mount Fuji”
11 May 2009 9© Rob Myers 2009
Two Great Flavors
ATDD
• Written by testers, generally.
• For the team.
• Run frequently by testers and developers.
• New tests fail until the story is done. Old tests should all pass.
TDD
• Written by developers.
• For developers.
• Run frequently by testers and developers.
• All tests pass 100% during integration.
11 May 2009 © Rob Myers 2009 10
“Peanut Butter Cup” by Laura Godi
Benefits of ATDD
• Communication:
• Clarifies the meaning of the story.
• Story scope/boundaries: “Is it a defect?”
• Progress: “We’re done!”
• Concise detection of defects.
• Less time troubleshooting and re-testing.
• Relieves the Agilist’s Dilemma.
11 May 2009 © Rob Myers 2009 11
Team Activities by Iteration
11 May 2009 © Rob Myers 2009 12
Past Present Future
ATDD/TDD
Analysis/Estimation
Exploratory Testing
If Only…
• …We could automate the tests without making them brittle…
• …We could write tests without having to see the working product…
• …We could do so without having to write in a programming language…
11 May 2009 13© Rob Myers 2009
Inconceivable!
“You keep using that word. I do not think it
means what you think it means…”
-- Inigo Montoya
The Princess Bride (1987)
11 May 2009 14© Rob Myers 2009
Fit:
Framework for Integrated Tests
Invented by Ward Cunningham
11 May 2009 © Rob Myers 2009 15
• Slow.• Difficult to set up.• Numerous failures from one defect.• Failures do not identify the defect.
Interface-Centric Tools
11 May 2009 © Rob Myers 2009 16
Business-Value-Centric Tool
11 May 2009 © Rob Myers 2009 17
Fit is Unique
• We can test any “layer,” thus expanding our creative reach.
• We usually avoid the UI layer.
• Fit is simple. It does very little, and you can do so much with it.
11 May 2009 © Rob Myers 2009 18
The First Martian Hotel
11 May 2009 © Rob Myers 2009 19
The “Column” Fixture
• For functions and calculations.
• Can be used to “Arrange” the test.
11 May 2009 © Rob Myers 2009 20
Fit Output
11 May 2009 © Rob Myers 2009 21_s
What Fit does to Your Tests
11 May 2009 © Rob Myers 2009 22
Your Fit test (HTML, Excel) Fit report (HTML)
Fit
Fit Fixture
(Test Code)
System Under Test
The “Row” Fixture
• For asserting state. (No input.)
• For lists.
• Shows missing rows as failures.
• Shows extra rows as failures.
• Declarative.
11 May 2009 © Rob Myers 2009 23
The “Action” Fixture
• For a sequence of events.
• For simple scripting.
• Imperative.
11 May 2009 © Rob Myers 2009 24
Error Conditions
11 May 2009 © Rob Myers 2009 25
Tests Can Have Many Tables
11 May 2009 © Rob Myers 2009 26
A Team Using Fit for ATDD
1. Testers and other teammates enter tests into Fit pages.
2. Developers develop Fit fixtures for the tables.
3. The team collaborates on the shape of the tables and Fit fixtures until they express what needs to be accomplished.
4. They run the tests and watch them fail cleanly (RED, not YELLOW).
5. Developers develop until tests pass.
11 May 2009 © Rob Myers 2009 27
Fit Tools, and More Fit Tools
http://fit.c2.com/wiki.cgi?FitTools
11 May 2009 © Rob Myers 2009 28
Sample Test Organization:
Regression Focus
All Tests
Really Slow Tests
Acceptance Tests
Prior Iterations
Iteration 1 Iteration 2
Current Iteration
11 May 2009 © Rob Myers 2009 29
setup.html Files
All Tests
Really Slow Tests
Acceptance Tests
Prior Iterations
Iteration 1 Iteration 2
Current Iteration
11 May 2009 © Rob Myers 2009 30
setup.htm
setup.htm
setup.htm
aTest.htm
FITLIBRARY
And Other Improvements
11 May 2009 © Rob Myers 2009 31
The “Set Up” Fixture
11 May 2009 © Rob Myers 2009 32
The “Do” Fixture
11 May 2009 © Rob Myers 2009 33
Same Test, Additional Activities
11 May 2009 © Rob Myers 2009 34
Improving Upon RowFixture
• ArrayFixture: For ordered lists (without having to create a position() column).
• SubsetFixture: Ignores surplus items.
11 May 2009 © Rob Myers 2009 35
Does TDD Work?
11 May 2009 © Rob Myers 2009 36
“The results of the case studies indicatethat the pre-release defect density of thefour products decreased between 40% and90% relative to similar projects that didnot use the TDD practice. Subjectively, theteams experienced a 15–35% increase ininitial development time after adoptingTDD.”
http://research.microsoft.com/en-us/projects/esm/nagappan_tdd.pdf, Nagappan et al, © Springer Science + Business Media, LLC 2008
11 May 2009 © Rob Myers 2009 37
BIBLIOGRAPHY
11 May 2009 © Rob Myers 2009 38
Fit for Developing Software
– Rick Mugridge
Prentice Hall
July 9, 2005
ISBN-10:
0321269349
ISBN-13:
978-0321269348
11 May 2009 © Rob Myers 2009 39
The Art of Agile Development
– James Shore & Shane Warden
O'Reilly Media, Inc.
October 26, 2007
ISBN-10:
0596527675
ISBN-13:
978-0596527679
11 May 2009 © Rob Myers 2009 40