exploratory testing in practice
DESCRIPTION
What is this thing called "Exploratory Testing"? How do you do it? Why is it important?TRANSCRIPT
Exploratory Testing
in Practice
Elisabeth Hendrickson Blog: testobsessed.com Twitter: @testobsessed
Slides Last updated June 22, 2012
Two Sides of Testing
Tested =
+ Checked Explored
That’s Nice…
…But we Believe in our
comprehensive, repeatable regression
tests.
Incidental Correctness
Given a series of 1’s and 0’s,
Compute the corresponding
decimal number
x
6
5
7
0
011
101
111
000
3
5
7
0 Result Input Expected
Missed Requirements
Hey, Where’s the motor for this
remote control car?
Motor?!? You never
said anything about a motor!
Real World Users Happen
You assume they’re
reasonable people
who will do
reasonable things.
They aren’t and
they won’t.
So What Is Exploratory Testing?
a systematic approach for
discovering risks
using rigorous
analysis techniques
coupled with
testing heuristics.
OK…But What Do You Actually Do?
Simultaneously…
…learning about the Target
…While designing tests
…And executing Them Immediately
using feedback from the last Little Experiment to inform the next
(Term coined by Cem Kaner. Definition refined by James Bach. Others have contributed to ET including
James Lyndsay, Jonathan Kohl, Michael Bolton.)
Experiment
Next, let’s try interrupting the
transaction here.
So it handled all the variations
on disconnected networks we
could think of…
Discover Things You Can Vary
11
Use State Models
Update System
Check Network
Idle
Time to connect
System Updated
No Network
Get Updates
Network Avail
No Updates
Content Downloaded
Use Data Models
Customer
Orders
Vendor
Products
Has 0 or More
Has 0 or More
Has 1 Or More
Use Deployment Diagrams
Local Server Farm
Agent AWS
Use Heuristics
Counts: 0, 1, Many
See the Test Heuristics Cheatsheet for more http://bit.ly/testcheatsheet
Position: Beginning, Middle, End
Selection: Some, None, All
Violate Domain-Specific Rules:
Age -1 IP address 999.999.999.999
Goldilocks: too big, too small,
just right
Interrupt States
Delete Dependent Data
In Flight Feedback at #AU2H
Jon Bach
(Exploring)
Matt Barcomb
(Coding)
Result: Feedback on Process AND Product
I’ll add things like that to the
acceptance criteria.
We need to think about that kind of
thing in future designs too.
We’ve automated the regression
test when we fixed the bug.
Pitfall: Going Off in the Weeds
it doesn’t work on 15 year old laptops running Windows 3.1!
Uh…
Why Would We
Care?
Focus with Charters
Explore (Area/feature)
With (Resources,
conditions, Dependency,
etc.)
to discover (information)
Structure with Time Boxed Sessions
Session
Session
Session
Session
Session
Session
Explore login in combination with bookmarks & history to discover information about interactions.
Explore the site with various browsers and settings to discover risks related to unsupported configurations.
Explore error handling using data type attacks to find problems related to data validation.
Charters Sessions
Explore Early, Explore Often
Exploratory Testing on Agile Projects
Within an Iteration
Automated Unit Tests
Automated Acceptance
Tests
Exploratory Testing
Define “done”, represent executable
requirements
Drive design, represent executable
specifications
Discover risks EARLY