-
Exploratory Tes.ng in the Enterprise
CQAA June 22, 2011
-
Partner, DeveloperTown www.DeveloperTown.com
Past President, Associa.on for SoFware Tes.ng www.Associa.onForSoFwareTes.ng.org
Ar.cles and Blogs www.MichaelDKelly.com
Author of the chapter on “Session Based Test Management” in upcoming book How to Reduce the Cost of So0ware Tes4ng
-
A STORY
-
• 3 testers instead of 15 • 66% (or $200,000) cheaper than budgeted • two-‐week shorter window for tes.ng • delivered on .me • no serious defects in produc.on
HOW?
-
“Simultaneous learning, test design, and test execu.on.”
-‐ James Bach
-
pure scripted freestyle exploratory
charters vague scripts
fragmentary test cases (scenarios) roles
-‐image from Jon Bach
-
the tester, the way they work,
and their ability to work in the future, is as important as
the product being tested
-
What makes them “exploratory?”
-
Self Management
-
Developing Ideas
-
Examining the Product
-
There’s always resistance
-
A desire for visible, track-‐able, progress
-
It’s difficult to quan.ta.vely measure coverage
-
There’s extraordinary pressure for “reuse” and “cost control”
-
Session Based Test Management
-
• charters
-
• charters • .me-‐boxed
-
• charters • .me-‐boxed • session notes
-
• charters • .me-‐boxed • session notes • debriefs
-
• charters • .me-‐boxed • session notes • debriefs • team priori.za.on
-
• charters • .me-‐boxed • session notes • debriefs • team priori.za.on • ad-‐hoc test documenta.on
-
• charters • .me-‐boxed • session notes • debriefs • team priori.za.on • ad-‐hoc test documenta.on • ad-‐hoc test automa.on
-
• charters • .me-‐boxed • session notes • debriefs • team priori.za.on • ad-‐hoc test documenta.on • ad-‐hoc test automa.on • dynamic metrics and repor.ng
-
Focus on dealing with uncertainty
-
START
-
Begin by implemen.ng a specific prac.ce
-
Add exploratory tes.ng days to the end of exis.ng test cycles
-
Carve out a small por.on of an exis.ng project and apply exploratory tes.ng to that por.on
-
Run a pilot project
-
Responding to concerns
-
the Michael Jordan effect
-
“We don’t do agile.”
-
“What about regression tes.ng?”
-
“We hand our test scripts off for test automa.on…”
-
“How do you provide traceability?”
-
“Your percent complete keeps changing!”
-
“Don’t we lose the ability to add people late in
the project if we get sideways?”
-
“Our testers aren’t subject mager experts.”
-
“We can’t afford all senior testers.”
-
“How do you make charters repeatable?”
-
“What about es.ma.on?”
-
Ques.ons?
-
Reading Materials
• Bach, James. "Exploratory Tes.ng Explained." First published as a chapter in The Test Prac..oner, 2002. Available online at: hgp://www.sa.sfice.com/ar.cles/et-‐ar.cle.pdf
• Bach, James and Jon Bach. "Exploratory Tes.ng Dynamics." Available online at: hgp://www.sa.sfice.com/ar.cles/et-‐dynamics.pdf
• Bach, Jon. "Session Based Test Management." First published in SoFware Tes.ng and Quality Engineering, November 2000. Available online at: hgp://www.sa.sfice.com/ar.cles/sbtm.pdf
• Bach, James. "A Low-‐Tech Tes.ng Dashboard." Presented at STAREast, 1999. Available online at: hgp://www.sa.sfice.com/presenta.ons/dashboard.pdf
• Bach, Jon. "How to Manage and Measure Exploratory Tes.ng." Quardev, 2006. Available online at: hgp://www.quardev.com/content/whitepapers/how_measure_exploratory_tes.ng.pdf
• Pegy, Kenn. "Reflec.ons on the Use of Session-‐Based Exploratory Tes.ng As the Primary Test Methodology for SoFware in an Agile Environment." Presented at the Indianapolis Workshops on So0ware Tes4ng, April 2005. Available online at: hgp://www.indianapolisworkshops.com/docs/Reflec.ons_on_the_use_of_Session-‐Based_Exploratory_Tes.ng_in_an_Agile_Environment.doc
• Pegy, Kenn. "Transi.oning from Standard V&V to Rapid Tes.ng Prac.ces in a Chao.c Project Environment." Presented at the Conference of the Associa4on for So0ware Tes4ng, 2007. Available online at: hgp://www.associa.onforsoFwaretes.ng.org/documents/Kenn_Pegy_Transi.oning_from_Standard_V&V_to_Rapid_Tes.ng_Prac.ces.pdf
• Wood, Bill and David James. "Applying Session-‐Based Tes.ng to Medical SoFware." Medical Device and Diagnos.c Industry Magazine, May 2003. Available online at: hgp://www.mddionline.com/ar.cle/applying-‐session-‐based-‐tes.ng-‐medical-‐soFware
• Kelly, Michael. “The benefits of exploratory tes.ng in agile environments.” SearchSoFwareQuality.com, 2009. Available online at: hgp://searchsoFwarequality.techtarget.com/.p/0,289483,sid92_gci1350741,00.html
• Kelly, Michael. “Using session-‐based test management for exploratory tes.ng.” SearchSoFwareQuality.com, 2009. Available online at: hgp://searchsoFwarequality.techtarget.com/.p/0,289483,sid92_gci1352925_mem1,00.html
• Kelly, Michael. “Using session-‐based test management for exploratory tes.ng coverage problems.” SearchSoFwareQuality.com, 2009. Available online at: hgp://searchsoFwarequality.techtarget.com/.p/0,289483,sid92_gci1355475_mem1,00.html
• Kelly, Michael. “Solving problems with session-‐based test management.” SearchSoFwareQuality.com, 2009. Available online at: hgp://searchsoFwarequality.techtarget.com/.p/0,289483,sid92_gci1355815_mem1,00.html