cognitive biases in testing - srikanth krishnan - eurostar 2012

Srikanth Krishnan, Oracle

Cognitive Biases in Testing

www.eurostarconferences.com

@esconfs#esconfs

Cognitive Biases in Testing

Srikanth S KrishnanSr. Director QAOracle India

Guess the percentage of African Nations in the

UN.

What is a Cognitive Bias ?

It is the human tendency to make systematic errors in certain circumstances based on cognitive factors rather than evidence.

Heuristics

Statistical mis-estimates

Memory errors

Social Constructs

Tester’s Delusion Spots

Test Planning

Test Design

Test Advocacy

Anchoring

Guess the percentage of African Nations in the

UN.

Anchoring

Will you be happier if you get a 100% hike?

Bidding after stating the last 2 digits of social security number

Why are black pearls more expensive?

occurs when people consider a particular value for an unknown quantity before estimating that quantity

Effect on Estimation

New Feature vs. regression testing of existing features

Effect on Negotiation

Relativity

Would you mind walking 5 min to save Rs 100 on a Rs

500 pen?

Would you mind walking 5 min to save Rs 100 on a Rs 7000 suite?

Decoy Effect or Asymmetric dominance effect

Option A Option B

Features 400 300

Cost 3800 2000

Option A

Option B

Option C

Features

400 300 350

Cost 3800 2000 4500

People not only compare things, but also compare things that are easily comparable.

The Price of Ownership

Endowment Effect

How much would you pay for a Coffee Mug ?

Applies to ideas as well as objects.

People not only compare things, but also compare things that are easily comparable.

The more effort you put into something the more ownership you feel. IKEA Effect

Ex. Manager from Finance

Test repository clean up

Which one will you get rid of?

(assume same coverage)

Suite1 Created with 100 man days effort

Suite2 Created with 30 man days effort

Effect of Expectation

Confirmation bias: Self-Fulfilling Prophesy

How do you get more women in the orchestra?

Positive vs. negative Test cases.

Seeing patterns where none exist.

exist.

exist.

20

12

4

31

Influence of great UI.

Negativity Bias

More attention is given to negative than positive experiences

The lingering effect of a nasty bug.

Sunk Costs

Concorde Fallacy

“I have spent so much time doing the setups.”

Concorde Fallacy

Why test a project that has been scrapped?

Cost of Zero

Amazon’s free shipping

AOL’s unlimited plan

Concorde Fallacy

No Cost to QA

Paradox of Choice

Concorde Fallacy

More is not always better

Burn your bridges

Concorde Fallacy

Why don’t we keep this server just in case?

Procrastination

Concorde Fallacy

Choose your own deadline

Concorde Fallacy

The hockey-stick effort curve

Concorde Fallacy

Milestones and distributed deadlines

Resurrection to Rationality

overcoming

channeling

References and Acknowledgements

Predictably Irrational – Dan ArielyNudge – Thaler & SunsteinBlink – Malcolm GladwellSway – Brafman (Ori & Rom)Wikipedia

Concorde Fallacy

Thank You