a taste of exploratory testing

A TASTE OF EXPLORATORY TESTING

Anne-Marie Charrett@charrett

Skype: Charretts Blog: Mavericktester.com

This work is licensed under the Creative Commons Attribution-NoDerivs 3.0 Unported License. To view a copy of this license,

visit http://creativecommons.org/licenses/by-nd/3.0/.

A LONG TIME AGO...

CHALLENGES

• Did the program run?

• Where the punch cards correct? (Source Code)

• Was the data correct ?

• Was the compiler correct? (Program Code)

• Did the program solve the problem?

Test Strategy Test Spec Test Execution

Analysis

Documents/Code

Design Testing

Activities

SCRIPTED TESTING

Test Report

Strategy

Modelling

Decision Making

ModellingConfiguration

OperationObservationEvaluation

Configuration

Reporting

LEGACY

Requires a tester to be able to:

Often has formalised methodology based on IEEE829

Design Test Scripts upfront against Requirements

Requires Experienced Test Analyst Vs Tester

Indicate Expected Result upfront

Complete when 100% Test Executed

PROGRESS

HOW WE DISCOVER

“Exploratory testing is an approach to software testing that is concisely described as simultaneous learning, test design and

test execution.” Cem Kaner

“The plainest definition of exploratory testing is test design and test execution at the same time.” J. Bach

Activities

EXPLORATORY TESTING

Modelling

Decision Making

Configuration

Operation

Observation

Evaluation

Learning

Testing

Design

Strategy

TESTER CENTRIC

DesignTesting

Learning

AutodidacticSelf Awareness

AutonomyCourage

Mental Engagement

• Download iHandy Coin Flip Free from Appstore

• Download iHandy Coin Flip Free from GooglePlay

TEST COIN FLIP FREE

Any Questions?

DISCUSSION

BUGS • flipping phone goes to adds

• coin changes automatically

• coin flip has pre Vibrate on Android is not working

• Flick Upward works when disabled

• there is no instruction screen, instruction menu or help text which I would normally expect (comparable product)

• flip the coin 20 times in a row. 18 out of 20 resulted in heads. Doesn't feel like random distribution...

• Screen Size not scalable

• Rapid Tapping makes it crash (Android)

• Windows 8 touch screen problems.

• Doesn’t support Android 2.2 (Download page says it should)

WHAT DID YOU DO?

• Played around

• Tried out different things

• Figured out how it worked

• Saw something unusual - looked more

• Read up the information (on Website)

• Compared your findings SP

ONTANEO

US TEST

--- James Bach & Michael Bolton - Rapid Software Testing

“If you are not using the learning garnered from test

design and test execution in a continuous and rapid loop to

optimize the quality of the work, you are not doing

exploratory testing”

--Michael Bolton

DELIBERATIVE TESTING

• Scripting in your head

• Requires considered response to testing

• Requires awareness of your approach

• May require use of tools or setting up of test data

• May require extensive investigation

--- James Bach & Michael Bolton - Rapid Software Testing

Heuristics

A heuristic is a fallible method of solving a problem or making a decision.

• Requires Skill

• Context Dependent

• Less Visible

HEURISTICS

The essence of heuristics: http://www.satisfice.com/blog/archives/462

GROCERY SHOPPING

You are purchasing three items at the store, at these prices:

• $19.95

• $39.98

• $29.97

About how much money are you spending?

HEURISTICS

The fastest way to solve this problem is to round off and approximate. The first item costs about $20, the second about $40, and the third about $30; therefore, you are spending about $90 on your shopping spree.

Rounding is often an excellent heuristic for arriving quickly at approximate answers to mathematical problems.

MENTAL ENGAGEMENT

• Alternation

• Branching and BackTracking

• Embrace Distraction

• Jump in & Play

• Focus & Defocus --- James Bach & Michael Bolton - Rapid Software Testing

EXPLORATORY TESTING HEURISTICS

• FEW HICCUPS (Ways to Recognise Bugs) - Michael Bolton

• SFDIPOT (Ways to Model and generate test ideas) - James Bach

• Heuristic Test Strategy Model - James Bach

• http://www.qualityperspectives.ca/resources_mnemonics.html

MANY WAYS OF ET

• Many different ways to use exploratory testing

• Mix level of documentation

• Mix level of reporting

• Mix level of learning

• Mix level of test design

• Mix level of testability

• elements of design, execution, interpretation, and learning are performed by the same person;

• the design, execution, interpretation, and learning happen together, rather than being separated in time;

• the tester is making her own choices about what to test, when to test it, and how to test it—the tester may use any automation or tools in support of her testing, or none at all, as she sees fit;

• everything that has been learned so far, including the result of the last test, informs the tester’s choices about the next test;

• the tester is focused on revealing new information, rather than confirming existing knowledge about the product;

• in general, the tester is varying aspects of her tests rather than repeating them, except where the repeating aspects of the test are intended to support the discovery of new information.

Michael Bolton - Evolving Understanding of Exploratory Testing

SUMMARY

a taste of exploratory testing

News & Politics

exploratory testing 101 - welcome - pqa testing · sbtm...

exploratory testing in sessions

exploratory testing workshop in risk-based agile testing

exploratory testing stew 2016

exploratory testing – black or white? -...

exploratory testing usability test types + environment ·...

a taste of exploratory testing

why do exploratory testing

introduction to exploratory testing

the awkward teenager of testing - exploratory testing -...

exploratory testing...

the nature of exploratory · pdf filethe nature of...

exploratory testing

exploratory software testing - bcs · freestyle exploratory...

exploratory mobile testing webinar_xbosoft_jean_annharrison

exploratory testing explained

bitter taste: taste testing (phenotype) and pcr (genotype)

testworksconf: experience exploratory testing

exploratory testing slides

myths about exploratory testing - agile testing days