a taste of exploratory testing
TRANSCRIPT
copyright ©TestingTimes 2013
A TASTE OF EXPLORATORY TESTING
Anne-Marie Charrett@charrett
Skype: Charretts Blog: Mavericktester.com
This work is licensed under the Creative Commons Attribution-NoDerivs 3.0 Unported License. To view a copy of this license,
visit http://creativecommons.org/licenses/by-nd/3.0/.
copyright ©TestingTimes 2013
A LONG TIME AGO...
copyright ©TestingTimes 2013
CHALLENGES
• Did the program run?
• Where the punch cards correct? (Source Code)
• Was the data correct ?
• Was the compiler correct? (Program Code)
• Did the program solve the problem?
copyright ©TestingTimes 2013
Test Strategy Test Spec Test Execution
Analysis
Documents/Code
Design Testing
Activities
SCRIPTED TESTING
Test Report
Strategy
Modelling
Decision Making
ModellingConfiguration
OperationObservationEvaluation
Configuration
Reporting
OperationObservationEvaluation Communicationcopyright ©TestingTimes 2013
copyright ©TestingTimes 2013
LEGACY
Requires a tester to be able to:
Often has formalised methodology based on IEEE829
Design Test Scripts upfront against Requirements
Requires Experienced Test Analyst Vs Tester
Indicate Expected Result upfront
Complete when 100% Test Executed
copyright ©TestingTimes 2013
PROGRESS
copyright ©TestingTimes 2013
HOW WE DISCOVER
copyright ©TestingTimes 2013
“Exploratory testing is an approach to software testing that is concisely described as simultaneous learning, test design and
test execution.” Cem Kaner
“The plainest definition of exploratory testing is test design and test execution at the same time.” J. Bach
copyright ©TestingTimes 2013
Activities
EXPLORATORY TESTING
Modelling
Decision Making
Configuration
Operation
Observation
Evaluation
Learning
Testing
Design
Strategy
copyright ©TestingTimes 2013
TESTER CENTRIC
DesignTesting
Learning
AutodidacticSelf Awareness
AutonomyCourage
Mental Engagement
• Download iHandy Coin Flip Free from Appstore
• Download iHandy Coin Flip Free from GooglePlay
TEST COIN FLIP FREE
Any Questions?
copyright ©TestingTimes 2013
copyright ©TestingTimes 2013
DISCUSSION
BUGS • flipping phone goes to adds
• coin changes automatically
• coin flip has pre Vibrate on Android is not working
• Flick Upward works when disabled
• there is no instruction screen, instruction menu or help text which I would normally expect (comparable product)
• flip the coin 20 times in a row. 18 out of 20 resulted in heads. Doesn't feel like random distribution...
• Screen Size not scalable
• Rapid Tapping makes it crash (Android)
• Windows 8 touch screen problems.
• Doesn’t support Android 2.2 (Download page says it should)
copyright ©TestingTimes 2013
WHAT DID YOU DO?
• Played around
• Tried out different things
• Figured out how it worked
• Saw something unusual - looked more
• Read up the information (on Website)
• Compared your findings SP
ONTANEO
US TEST
ING
--- James Bach & Michael Bolton - Rapid Software Testing
copyright ©TestingTimes 2013
“If you are not using the learning garnered from test
design and test execution in a continuous and rapid loop to
optimize the quality of the work, you are not doing
exploratory testing”
--Michael Bolton
copyright ©TestingTimes 2013
DELIBERATIVE TESTING
• Scripting in your head
• Requires considered response to testing
• Requires awareness of your approach
• May require use of tools or setting up of test data
• May require extensive investigation
--- James Bach & Michael Bolton - Rapid Software Testing
copyright ©TestingTimes 2013
Heuristics
A heuristic is a fallible method of solving a problem or making a decision.
• Requires Skill
• Context Dependent
• Less Visible
HEURISTICS
The essence of heuristics: http://www.satisfice.com/blog/archives/462
copyright ©TestingTimes 2013
GROCERY SHOPPING
You are purchasing three items at the store, at these prices:
• $19.95
• $39.98
• $29.97
About how much money are you spending?
copyright ©TestingTimes 2013
HEURISTICS
The fastest way to solve this problem is to round off and approximate. The first item costs about $20, the second about $40, and the third about $30; therefore, you are spending about $90 on your shopping spree.
Rounding is often an excellent heuristic for arriving quickly at approximate answers to mathematical problems.
copyright ©TestingTimes 2013
MENTAL ENGAGEMENT
• Alternation
• Branching and BackTracking
• Embrace Distraction
• Jump in & Play
• Focus & Defocus --- James Bach & Michael Bolton - Rapid Software Testing
copyright ©TestingTimes 2013
EXPLORATORY TESTING HEURISTICS
• FEW HICCUPS (Ways to Recognise Bugs) - Michael Bolton
• SFDIPOT (Ways to Model and generate test ideas) - James Bach
• Heuristic Test Strategy Model - James Bach
• http://www.qualityperspectives.ca/resources_mnemonics.html
copyright ©TestingTimes 2013
MANY WAYS OF ET
• Many different ways to use exploratory testing
• Mix level of documentation
• Mix level of reporting
• Mix level of learning
• Mix level of test design
• Mix level of testability
copyright ©TestingTimes 2013
• elements of design, execution, interpretation, and learning are performed by the same person;
• the design, execution, interpretation, and learning happen together, rather than being separated in time;
• the tester is making her own choices about what to test, when to test it, and how to test it—the tester may use any automation or tools in support of her testing, or none at all, as she sees fit;
• everything that has been learned so far, including the result of the last test, informs the tester’s choices about the next test;
• the tester is focused on revealing new information, rather than confirming existing knowledge about the product;
• in general, the tester is varying aspects of her tests rather than repeating them, except where the repeating aspects of the test are intended to support the discovery of new information.
Michael Bolton - Evolving Understanding of Exploratory Testing
SUMMARY
copyright ©TestingTimes 2013
FURTHER READING
• Exploratory Skills and Dynamics
• http://www.developsense.com/resources.html
• Heuristic Test Strategy Model
• Test Heuristics Cheat Sheet Elisabeth Hendrickson's
• You Are Not Done Yet Michael Hunter's Coverage
copyright ©TestingTimes 2013
QUESTIONS?
