agile testing: solving the agilist\'s dilemma

Agile Testing:

Solving the Agilist’s

Dilemma

Star East

6 May 2009

11 May 2009 1© Rob Myers 2009

Rob Myers

• Rob.Myers@agileInstitute.com

• Teaching Agile courses since 2002

• XP/Agile coach since 1999

• Using TDD to build software since 1998

• Professional software development since 1986

• Blog: PowersOfTwo.agileInstitute.com

11 May 2009 2© Rob Myers 2009

The Agilist’s Dilemma

11 May 2009 © Rob Myers 2009 3

Debt

• Quality Debt

• Defects

• Design Debt

• Software design resists change

• Testing Debt

• Manual testing

11 May 2009 © Rob Myers 2009 4

11 May 2009 © Rob Myers 2009 5

DILBERT: © Scott Adams/Dist. by United Feature Syndicate, Inc.

Why Wait?

11 May 2009 © Rob Myers 2009 6

Code TestMonths

Code TestWeeks

Code TestDays

Code Test

Run Test

Instead…

11 May 2009 © Rob Myers 2009 7

CodeTestWrite Test Run Test Run Test Run Test

Manual Testing

Writing Test Cases

Record

&

Playback

TDD

Combine Advantages

11 May 2009 © Rob Myers 2009 8

Manual

Later First

Automated

“Move Mount Fuji”

11 May 2009 9© Rob Myers 2009

Two Great Flavors

ATDD

• Written by testers, generally.

• For the team.

• Run frequently by testers and developers.

• New tests fail until the story is done. Old tests should all pass.

TDD

• Written by developers.

• For developers.

• Run frequently by testers and developers.

• All tests pass 100% during integration.

11 May 2009 © Rob Myers 2009 10

“Peanut Butter Cup” by Laura Godi

Benefits of ATDD

• Communication:

• Clarifies the meaning of the story.

• Story scope/boundaries: “Is it a defect?”

• Progress: “We’re done!”

• Concise detection of defects.

• Less time troubleshooting and re-testing.

• Relieves the Agilist’s Dilemma.

11 May 2009 © Rob Myers 2009 11

Team Activities by Iteration

11 May 2009 © Rob Myers 2009 12

Past Present Future

ATDD/TDD

Analysis/Estimation

Exploratory Testing

If Only…

• …We could automate the tests without making them brittle…

• …We could write tests without having to see the working product…

• …We could do so without having to write in a programming language…

11 May 2009 13© Rob Myers 2009

Inconceivable!

“You keep using that word. I do not think it

means what you think it means…”

-- Inigo Montoya

The Princess Bride (1987)

11 May 2009 14© Rob Myers 2009

Fit:

Framework for Integrated Tests

Invented by Ward Cunningham

11 May 2009 © Rob Myers 2009 15

• Slow.• Difficult to set up.• Numerous failures from one defect.• Failures do not identify the defect.

Interface-Centric Tools

11 May 2009 © Rob Myers 2009 16

Business-Value-Centric Tool

11 May 2009 © Rob Myers 2009 17

Fit is Unique

• We can test any “layer,” thus expanding our creative reach.

• We usually avoid the UI layer.

• Fit is simple. It does very little, and you can do so much with it.

11 May 2009 © Rob Myers 2009 18

The First Martian Hotel

11 May 2009 © Rob Myers 2009 19

The “Column” Fixture

• For functions and calculations.

• Can be used to “Arrange” the test.

11 May 2009 © Rob Myers 2009 20

Fit Output

11 May 2009 © Rob Myers 2009 21_s

What Fit does to Your Tests

11 May 2009 © Rob Myers 2009 22

Your Fit test (HTML, Excel) Fit report (HTML)

Fit

Fit Fixture

(Test Code)

System Under Test

The “Row” Fixture

• For asserting state. (No input.)

• For lists.

• Shows missing rows as failures.

• Shows extra rows as failures.

• Declarative.

11 May 2009 © Rob Myers 2009 23

The “Action” Fixture

• For a sequence of events.

• For simple scripting.

• Imperative.

11 May 2009 © Rob Myers 2009 24

Error Conditions

11 May 2009 © Rob Myers 2009 25

Tests Can Have Many Tables

11 May 2009 © Rob Myers 2009 26

A Team Using Fit for ATDD

1. Testers and other teammates enter tests into Fit pages.

2. Developers develop Fit fixtures for the tables.

3. The team collaborates on the shape of the tables and Fit fixtures until they express what needs to be accomplished.

4. They run the tests and watch them fail cleanly (RED, not YELLOW).

5. Developers develop until tests pass.

11 May 2009 © Rob Myers 2009 27

Fit Tools, and More Fit Tools

http://fit.c2.com/wiki.cgi?FitTools

11 May 2009 © Rob Myers 2009 28

Sample Test Organization:

Regression Focus

All Tests

Really Slow Tests

Acceptance Tests

Prior Iterations

Iteration 1 Iteration 2

Current Iteration

11 May 2009 © Rob Myers 2009 29

setup.html Files

All Tests

Really Slow Tests

Acceptance Tests

Prior Iterations

Iteration 1 Iteration 2

Current Iteration

11 May 2009 © Rob Myers 2009 30

setup.htm

setup.htm

setup.htm

aTest.htm

FITLIBRARY

And Other Improvements

11 May 2009 © Rob Myers 2009 31

The “Set Up” Fixture

11 May 2009 © Rob Myers 2009 32

The “Do” Fixture

11 May 2009 © Rob Myers 2009 33

Same Test, Additional Activities

11 May 2009 © Rob Myers 2009 34

Improving Upon RowFixture

• ArrayFixture: For ordered lists (without having to create a position() column).

• SubsetFixture: Ignores surplus items.

11 May 2009 © Rob Myers 2009 35

Does TDD Work?

11 May 2009 © Rob Myers 2009 36

“The results of the case studies indicatethat the pre-release defect density of thefour products decreased between 40% and90% relative to similar projects that didnot use the TDD practice. Subjectively, theteams experienced a 15–35% increase ininitial development time after adoptingTDD.”

http://research.microsoft.com/en-us/projects/esm/nagappan_tdd.pdf, Nagappan et al, © Springer Science + Business Media, LLC 2008

11 May 2009 © Rob Myers 2009 37

BIBLIOGRAPHY

11 May 2009 © Rob Myers 2009 38

Fit for Developing Software

– Rick Mugridge

Prentice Hall

July 9, 2005

ISBN-10:

0321269349

ISBN-13:

978-0321269348

11 May 2009 © Rob Myers 2009 39

The Art of Agile Development

– James Shore & Shane Warden

O'Reilly Media, Inc.

October 26, 2007

ISBN-10:

0596527675

ISBN-13:

978-0596527679

11 May 2009 © Rob Myers 2009 40