Agile Testing, Risk, Uncertainty, and Why It All Works (the updated for AGILEEE, almost all new edition) Elisabeth Hendrickson

Quality Tree Software, Inc. www.qualitytree.com

esh@qualitytree.com

Last updated September 22, 2011 Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

What Does Agile Really Mean?

Agile software teams…

…Deliver value in the form of releasable software at frequent regular intervals (at least monthly)…

…At a sustainable pace…

…While adapting to the changing needs of the business.

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Source: Wikimedia Commons. Courtesy of Ray Kurzweil and Kurzweil Technologies, Inc.

Increasing Complexity

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Speculation Buildup Sp

ecul

ation

Analyze Design Implement Stabilize Big Bang Release

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Speculation, Lack of Alignment, and Risk

Intentions

Implementation

Actual Need

?

? ?

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Agile eliminates speculation buildup… Sp

ecul

ation

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

…only if we eliminate the speculation

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

“frAgile”!Sp

ecul

ation

Iterations…

Stabilize

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Every. Single. Time.

Empirical Evidence

speculation!

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

The organizations best positioned to succeed

are the ones that

LEARN

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Build

Measure Learn

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

How do we check alignment?

Intentions

Implementation

Actual Need

Get out of the building!

Steve Blank, author of Four Steps to the Epiphany

?

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

How do we check alignment?

Begin with the End in Mind. Capture examples with expectations. Automate

checking of expectations. Explore to discover risks.

Intentions

Implementation

Actual Need

?

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

The Letter and Spirit of a Story

Explicit Expectations

Implicit Expectations

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Check that all previously met expectations are still met even as the solution grows in

capability and complexity.

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

How do we check alignment?

Intentions

Implementation

Actual Need

Demo. Rehearse delivery. Deliver frequently. Explore with the

customer or proxy.

?

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Reducing Feedback Latency

Latency

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

What Does Agile Really Mean?

Agile software teams…

…Deliver value in the form of releasable software at frequent regular intervals (at least monthly)…

…At a sustainable pace…

…While adapting to the changing needs of the business.

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Intentions

Implementation

Actual Need

✔

✔

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

(OK, now let’s talk about “testing.”)

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

any activity that yields

about the extent to which our

, our , and the

are .

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Checking Alignment

Intentions

Implementation

Actual Need

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Every. Single. Time.

speculation!

Empirical Evidence

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

The Whole Team Succeeds or Fails TOGETHER

“Hey! There’s a hole in your side

of the boat.”

There is no “Us” and “Them.”

There is only Us.

As Alistair Cockburn says:

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Nine Key Technical Testing-Related Agile Practices

ATDD TDD Exploratory Testing

Collective Ownership

Automated Unit Tests

Automated System Tests

Continuous Integration

Rehearse Delivery

Automated Deploy

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Initiative

Test-First Mindset: Begin with the End in Mind

Code

Strategy How will I know my efforts have the result I intended?

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Acceptance-Test Driven Development (ATDD)

User Story Workshop Implementation

Agile-Friendly Test Automation Framework

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Continuous Integration (CI)

CI tools do automated builds, execute tests, and report the results

Developers practicing CI merge their changes locally & execute tests before checking in

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Version ALL Technical Assets with the Code

Unit Tests

Production Code

Source Control Repository

(Plus All other

technical assets)

Test “Fixture”

Code

Acceptance Tests

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Increase Visibility

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Automated Deploy

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Fail Early, Fail Fast

Failing early & fast gives us time to fix the problems.

Local Unit Tests Secs- Mins

Mins CI Build

Mins- Hrs

Automated Regression

Hrs- Days

Exploration

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Red Build Means Stop the Line

If a previously passing expectation fails, there’s a bug. Bugs slow everything down. To keep

sustainable pace, fix bugs fast.

We can just throw that bug on the pile

with the others.

Yuck.

But that will increase technical

debt & slow velocity.

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Exploratory Testing

Simultaneously…

…learning about the software

…designing tests

…executing tests

using feedback from the last

test to inform the next

(See Jon and James Bach’s work on Session-Based ET)

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Done Includes Tested

Implemented

+ Checked

+ Explored

== DONE

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

Every. Single. Time.

speculation!

Empirical Evidence

Copyr

ight

© 2

01

1 E

lisa

bet

h H

endri

ckso

n,

Qual

ity

Tre

e So

ftw

are,

Inc.

LEARN

TEST

Intentions

Implementation

Actual Need