test estimation hacks: tips, tricks and tools webinar

#TESTESTIMATION HACKS

April 21 2016

Matt Heusser – Excelon Development Justin Rohrman- Excelon Development

Tips, Tricks and Tools

Join the conversation – use the hashtag #testestimation on Twitter

HOW TO WATCH THIS WEBINAR


• Audio for this webinar is delivered through your computer. Make sure your speakers are turned up or use a set of headphones.

• If your audio quality is choppy, it could be your internet connection.

• You can customize your webinar viewing experience by increasing, decreasing or minimizing the size of the widgets on your screen.

• If you have questions, enter them in the widget on the left.

Robust test management platform purpose-built to help agile teams centralize, organize and accelerate software testing

ABOUT QASYMPHONY

Agile Transformation: People, Process and Tools to Make Your Transformation SuccessfulWednesday, April 27th at 10am ET

Guest Speaker: Andrew Strickland• Andrew Stickland is Head of Client

Services for Clearvision. With over 25 years experience in IT, Andrew is a specialist in agile transformation and tool adoption. Working with the likes of HSBC, Thales and Proctor & Gamble, Andrew can draw on a wealth of experience as a senior Atlassian consultant delivering some of Clearvision's biggest projects..

UPCOMING WEBINAR


STAR EAST – MAY 4-5 IN ORLANDO

• Visit QASymphony booth #18

• Schedule a meeting with us at calendly.com/stareast

• Check out Mike Cooper’s presentation at 3pm on Thursday, May 5th.

• Get more information at stareast.techwell.com.


Matt Heusser, Principal Consultant, Excelon DevelopmentAs Managing Director of Excelon Development, Matt Heusser consults, trains, and contributes on software projects. Probably best known for his writing, Matt is the lead editor of "How to Reduce The Cost of Software Testing" (Taylor and Francis, 2011), editor for Stickyminds.com, and recipient of the 2015 Most Popular Online Contributor to Agile at the Agile Awards. A 2014 recipient of the Most Infuential Agile Test Professional Person Award (MAITPP) in Potsdam, Germany, Matt also served as the lead organizer of the Software Testing World Cup. A former member of the board of directors of the Association for Software Testing, Matt is also the creator of the Lean Software Testing family of methods.

OUR PRESENTERS

Justin Rorhman, Senior Consultant, Excelon DevelopmentJustin has been a professional software tester in various capacities since 2005. In his current role, Justin is a consulting software tester and writer working with Excelon Development. Outside of work, he is currently serving on the Association For Software Testing Board of Directors as President helping to facilitate and develop projects like BBST, WHOSE, and the annual conference CAST. Justin is deeply interested in software testing and delivery, and also in helping organizations fix problems in measurement and metrics programs.


• This webinar will be recorded and available on-demand tomorrow. You will get an email when it is available.

• Join the conversation on Twitter using the hashtag #testestimation

• Use the Q&A widgets to ask questions during the webinar.

• At the end of the webinar, you will be asked to take a short survey.

HOUSEKEEPING


THE ‘WHY’ OF ESTIMATES


OUR AGENDA


• Why Estimate

• Fundamental Issues in Test Estimation

• Estimating Models

• Decomposition, Comparison, Timeboxed, Prediction, Guru Method, Aggregate, Consensus

• Test Estimation Vs Test Negotiation

• Responding to Tough Questions

• Q&A

• Conclusions

Goal: Make your life betterby helping you answer “How Long Will it Take?”

In a way that will stick!

THE ‘WHY’ OF ESTIMATES


• Schedule Control• Cost Control

• Comparison Shopping• Governance

ISSUES: WHY IT’S HARD


• Most projects are linear

• Testing is not

• Time can be spent test, documenting, fixing, or waiting

• Beyond the power of test

• The quality of the software pre-testing

• The amount of time waiting for fixes

• The bug injection rate

• 100 changes ~40%:168 changes

• 100 changes ~20%: 129 changes

METHOD: FUNCTIONAL DECOMPOSITION


You will need a pen and paper for this exercise

METHOD: FUNCTIONAL DECOMPOSITION


Pros ConsPeople “get it” Inaccurate

Culturally Accepted Assumes 40hrs/week of productivity

Easy to sell Assumes linear work

Padding is de rigueur Kind of sleazy to sell

Life is too short to play games

METHOD: COMPARISON


“Breadbox analysis”

METHOD: COMPARISON


Pros Cons

Fast …if possible

Lack of std definitions

Fails with bad data

Hard to get good data

Scope tends to creep

METHOD: TIMEBOXED


“How long will it take you to test for a day”

METHOD: TIMEBOXED


Pros Cons

Easy! Unpredictable

Doesn’t always work

Hard to explain “why”

METHOD: PREDICTION


We don’t do 40 hours of on-project work in a week.

Let’s count the number weactually get done.

And plan on that.

METHOD: PREDICTION / PROJECTION


Pros Cons

Accurate …if people want to hear it

Defensible Requires some data

Empirical/data-backed Misses black swans

May hurt creativity in test

Inaccurate with pure averages

METHOD: GURU METHOD


Step 1: Pick a GuruStep 2: Ask them for a dateStep 3: Use that

METHOD: AGGREGATE & CONSENSUS


Gather estimates from peers

Average them

Or (better)

Explore differences

METHOD: AGGREGATE & CONSENSUS


Pros Cons

Requires time/effort Forced consensus effect

Wisdom of Crowds Outliers most interesting

Gets buy-in What theoretical basis?

Hard to sell Easy to do poorly

Habituated response effect

METHOD: ALL THE THINGS


Use several methodsAsk why are they different



Pros Cons

Accurate Very expensive

Mature Haystack effect

Responsible “Too much theory”

Educates “Shouldn’t you be working?”

All methods broken



Pros Cons

Accurate Very expensive

Mature Haystack effect

Responsible “Too much theory”

Educates “Shouldn’t you be working?”

All methods broken

Fix: Start by gathering data, then build your own defensible models of project performance. Learn to educate without preaching. Give and take.

SIDEBAR: EFFORT VS TIME


It’s a mind problem

… and the fix is data

TEST ESTIMATION: OR TEST NEGOTIATION


Matts first day on his first job

Alternatives to “yes” and “no”

COMMUNICATIONS: PROBABILITY DISTRIBUTIONS


50% confidence – “Most Likely”90% confidence interval – “Protected”

RESPONDING TO TOUGH QUESTIONS


• Provide Data• Make it transparent• Make it a conversation• Provide options:

• Cut from the top• Let the bottom go• More/Less time

Q&A


Questions?

Thank you
