T17 Personal Excellence 5/5/16 13:30
(Almost) Everything I Know about Testing I Learned Playing Poker
Presented by:
Matthew Eakin
Centric Consulting
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073 888-‐-‐-‐268-‐-‐-‐8770 ·∙·∙ 904-‐-‐-‐278-‐-‐-‐0524 -‐ [email protected] -‐ http://www.stareast.techwell.com/
Matthew Eakin Centric Consulting With more than twenty years of technical, leadership, and planning experience, Matthew Eakin is the national automation architect (Ruby/Cucumber) with the software quality assurance and testing practice of Centric Consulting. His experience in all aspects of the SDLC, combined with development skills, has helped Matthew to be an effective agile testing practitioner and coach. He speaks extensively on Gherkin Scripting and is in high demand as a behavior-‐driven development and acceptance test-‐driven development coach. Matthew now leads his workshops—Agile Testing, Gherkin Scripting, and A Manual Testers Guide to the Ruby/Cucumber Framework—nationally and teaches customized variations of the programs at client sites.
NOTICE: Proprietary and Confidential
This material is proprietary to Centric Consulting, LLC. It contains trade secrets and information which is solely the property of Centric Consulting, LLC. This
material is solely for the Client’s internal use. This material shall not be used, reproduced, copied, disclosed, transmitted, in whole or in part, without the express
consent of Centric Consulting, LLC.
© 2016 Centric Consulting, LLC. All rights reserved
(Almost) Everything I know
about testing I learned playing
poker
Matthew Eakin, National Automation Architect [email protected] @MatthewEakin MatthewEakin
At Centric, we deliver unmatched client experiences that center on
collaboration. Working together, we implement lasting solutions that work.
Solutions that matter.
Texas Hold ‘Em & Software Testing
Some testing concepts we are going to see in action: 1.Knowledge of the domain
2.Risk Analysis
3.Planning vs. Writing a Test Plan
4.Adjusting your efforts as you go
5.Knowledge gathering
6.Estimation
7.Resource allocation
What we are going to do…
At least 2 hands of Texas Hold ‘Em
Hand 1: “Agile”
•Constant evaluation of risk
•Constant knowledge gathering
•Constant planning and adjusting
Hand 2: “Waterfall”
•Risk evaluation conducted up-front (even though our knowledge is limited)
•Knowledge is gathered up-front (even though our knowledge is limited)
•Planning conducted up-front (even though our knowledge is limited)
•Adjusting is discouraged. You MUST follow the Plan!!
Hand 3 and beyond: You choose the approach…
Rules of Texas Hold ‘Em (in a nutshell)
•All players are dealt 2 cards face down. You can look at them, but don’t
show anyone.
•A round of betting will follow
•5 additional cards will be dealt into the “Community” face up. •3 cards will be dealt first followed by a round of betting
•1 card will be dealt followed by a round of betting
•1 card will be dealt followed by a final round of betting
•Make the best 5 card hand
•Hand order: •Royal flush – 10-A all same suit
•4 of a kind – 4 cards match, i.e. 9-9-9-9-3
•Full House – 3 of one card, 2 of another, i.e. K-K-K-5-5
•Flush – all cards same suit, i.e. 2-3-6-8-Q all clubs
•Straight – all cards in a row, i.e. 4-5-6-7-8
•3 of a kind – 3 cards match, i.e. K-K-K-5-7, 2-2-2-7-9
•2 pair – i.e. K-K & 6-6 in your hand
•Pair – 2 cards match, i.e. K-K, 6-6
•High card in hand
Relationship to testing
What is your testing teams knowledge of each
domain?
•Knowledge of Testing
•Knowledge/experience with the process (agile or waterfall)
•Technical Knowledge
•Knowledge of the Application
The Deal
•Everyone is dealt 2 cards
•At this point you know very little
about the deck of cards or what
cards other players could potentially
have.
9
Assessing Risk
Risk Assessment: Key concept to both Poker &
Software Testing
•In Poker: assess your probability of winning the hand with your cards
•Probability of your cards
•Probability of opponents cards (what you think they have)
•Probability with chip stacks
•Conduct risk analysis on every card drop
•Conduct risk analysis on every bet
•Conduct risk analysis on every player action
Relation to testing
Testing is Risk Minimization
•In Testing: use risk analysis to determine what to build/test first
•Conduct risk analysis on product backlog
•Conduct risk analysis on sprint backlog
•Conduct risk analysis on every User Story
•Conduct daily risk analysis on work in progress
Betting
•As each player bets you begin to
know more about:
• Their skill level
• Their aversion to risk
• What they have in their hand
•Hint: players won’t bet unless they
think they can win the hand, right?
WRONG!!! • Beware the bluff
12
• Everyone bets
Relation to Testing
13
Resource Allocation
• As a project progresses you learn more about:
• the risks associated with these changes of the application,
• your testing team
• the entire project team
• You might want to allocate more/less resources to one area of the app
• You might want to focus more on automation
Bluffing
• With every code drop you learn more about the code base/application
• Is the project really 80% done?
• Are the developers bluffing?
• Does this change the risk? Change resource allocation?
The Flop
•The flop •3 cards are dealt face-up. All players can use these cards to create the best 5
card hand possible.
•You now know a little more about the deck of cards.
•You conduct another risk assessment of your now 5 cards and determine if
you should bet or fold.
14
Relation to Testing
15
Planning
“Planning is everything. Plans are nothing.”
–Field Marshal Helmuth Graf von Moltke
• Plan for (almost) every possible outcome. When it happens you are
ready for it
• Do you have the tools/equipment/skills to handle each outcome?
• What will you do when only 50% of your scripts pass? What if 100% on
first pass?
• What if development delivers code late? What if scope changes? What if
your test environment isn’t available on day 1 of testing?
• Many “risks” are identified in a test plan. But what are you going to do if
they actually happen?
• Can you plan out the rest of the hand?
The Flop - betting
16
• Everyone bets
– As each player bets continue to conduct risk assessments on them. Does their betting indicate they have a good hand? A bad hand?
The Turn & the Turn Bet
•The Turn •1 card is dealt face-up (4 total in the
community cards).
17
• Everyone bets
– Can you figure out what hand everyone has yet?
– Can you predict your final hand?
Relation to Testing
18
Predicting – a hypothesis – what do you think will happen?
• As a project progresses can you predict how/when it will end?
• Can you predict your final hand?
• Can you predict opponents final hand?
• Tools to help you predict
• Burn-up/Burn-down charts
• Run Reports
• Defect Reports
• Very important for testers to be able to predict the future
• Discuss bad news early
• Critical team discussions
The River & the Final Bet
•The river •The final card is dealt face-up into the
community cards (there are now 5).
19
• Final bets
– Your last chance to get everyone's $$$
The Show
•Before we show our hands… •Can you guess each players hand?
•Who do you think won the hand?
•The show •Everyone shows their cards.
•Best hand takes the pot.
20
What we learned…
•Understand your teams knowledge of the domain
•Constant risk analysis
•Adjust resources as needed
•Planning is critical
•Can you predict the outcome of a project accurately?
21
The 2nd Hand – the “Waterfall Hand”
•We are going to play this hand as if we were testing in a
waterfall project.
•All planning will be done up-front rather than as we go.
•All risk analysis will be conducted up-front rather than as we
go.
•Resources will be allocated up-front rather than as we go.
•Your knowledge of the requirement is assumed to be 100%.
23
The Test Plan
•Each player is given a Test Plan which they must follow
EXACTLY for the entire round.
•Plan will include: •What every other players cards will be
•Cards in the Board will be revealed.
•Did you win?
24
Sample Test Plan
Deal:
Player 1 will be dealt Q, 10
Player 2 will be dealt 9, 4
Player 3 will be dealt 3, 9
Player 4 will be dealt 6, 7
Player 5 will be dealt 8, 10
Player 6 (you) will be dealt A, K
Flop: Cards will be 5, 8, 9
Turn: Card will be Q
River: Card will be J
You will end up with a Straight A, K, Q, J, 10
You will lose to Player 4 who has a Straight Flush 5 , 6, 7, 8, 9
Relation to Testing
26
Estimating in Waterfall
• Every Test Manager has had to provide a resource estimate for a project
they know almost nothing about.
• Have your estimates every been accurate? +_ 50%?
• Estimates are very difficult when there is so much unknown
• Based on past experiences you can start getting closer on you
estimates
• Tools can help
The Deal
•Everyone is dealt 2 cards •Players must stick to their Test Plan.
28
• Are there any problems with the Test Plan?
– Too bad, you must stick to the Test Plan!!!
• Did anything happen that was not covered in the Test Plan?
– How did you react?
The Flop
•3 cards are dealt to the Board •Players must stick to their Test Plan.
29
• Are there any problems with the Test Plan?
– Too bad, you must stick to the Test Plan!!!
Change Requests
30
A change request has been introduced. Since the code delivered was not as expected, numerous questions were asked of the business/BA. Some requirements have been changed and new ones have been introduced. Result:
• Each player can now change their Test Plan
– Conduct a new Risk Assessment on each Player.
– Evaluate your own stack and create a new betting plan for the Turn and River.
The Turn & the Turn Bet
•The Turn •1 card is dealt face-up (4 total in the
community cards).
31
• Everyone bets
– Can you figure out what hand everyone has yet?
– Can you predict your final hand?
The River & the Final Bet
•The river •The final card is dealt face-up into the
community cards (there are now 5).
32
• Final bets
– Your last chance to get everyone's $$$
The Show
•Before we show our hands… •Can you guess each players hand?
•Who do you think won the hand?
•The show •Everyone shows their cards.
•Best hand takes the pot.
33
What we learned…
•Writing out a Test Plan based on critical assumptions we have
very little information about is dangerous.
•A Test Plan does not fit every situation.
•Following a Test Plan, even though it doesn’t match reality, will
result in disaster for the tester.
34
NOTICE: Proprietary and Confidential
This material is proprietary to Centric Consulting, LLC. It contains trade secrets and information which is solely the property of Centric Consulting, LLC. This
material is solely for the Client’s internal use. This material shall not be used, reproduced, copied, disclosed, transmitted, in whole or in part, without the express
consent of Centric Consulting, LLC.
© 2016 Centric Consulting, LLC. All rights reserved
Thank You
Matthew Eakin, National Automation Architect [email protected] @MatthewEakin MatthewEakin
Test Plan Player 1
Deal:
Player 1 will be dealt K, 10
Player 2 will be dealt 9, 4
Player 3 will be dealt 3, 9
Player 4 will be dealt 6, 7
Player 5 will be dealt 8, 10
Player 6 will be dealt A, K
Flop: Cards will be 5, 8, 9
Turn: Card will be Q
River: Card will be J
You will end up with a Straight K, Q, J, 10, 9
You will lose to Player 4 who has a Straight Flush 5, 6, 7, 8, 9
Test Plan Player 2
Deal:
Player 1 will be dealt Q, 10
Player 2 will be dealt 6, 7
Player 3 will be dealt 3, 9
Player 4 will be dealt 9, 4
Player 5 will be dealt 8, 10
Player 6 will be dealt A, K
Flop: Cards will be 5, 8, 9
Turn: Card will be Q
River: Card will be J
You will end up with a Straight Flush 5 , 6, 7, 8, 9
You WIN!!!!!
Test Plan Player 3
Deal:
Player 1 will be dealt Q, 10
Player 2 will be dealt 9, 4
Player 3 will be dealt 3, 9
Player 4 will be dealt 6, 7
Player 5 will be dealt 8, 10
Player 6 will be dealt A, K
Flop: Cards will be 5, 8, 9
Turn: Card will be Q
River: Card will be J
You will end up with a Pair 9, 9, Q, J, 8
You will lose to Player 4 who has a Straight Flush 5 , 6, 7, 8, 9
Test Plan Player 4
Deal:
Player 1 will be dealt Q, 10
Player 2 will be dealt 9, 4
Player 3 will be dealt 3, 9
Player 4 will be dealt 6, 7
Player 5 will be dealt 8, 10
Player 6 will be dealt A, K
Flop: Cards will be 5, 8, 9
Turn: Card will be Q
River: Card will be J
You will end up with a Straight Flush 5 , 6, 7, 8, 9
You WIN!!!!!
Test Plan Player 5
Deal:
Player 1 will be dealt Q, 10
Player 2 will be dealt 9, 4
Player 3 will be dealt 8, 10
Player 4 will be dealt 6, 7
Player 5 will be dealt 3, 9
Player 6 will be dealt A, K
Flop: Cards will be 5, 8, 9
Turn: Card will be Q
River: Card will be J
You will end up with a Pair 9, 9, Q, J, 8
You will lose to Player 4 who has a Straight Flush 5 , 6, 7, 8, 9
Test Plan Player 6
Deal:
Player 1 will be dealt Q, 10
Player 2 will be dealt 9, 4
Player 3 will be dealt 3, 9
Player 4 will be dealt A, K
Player 5 will be dealt 8, 10
Player 6 will be dealt 6, 7
Flop: Cards will be 5, 8, 9
Turn: Card will be Q
River: Card will be J
You will end up with a Straight Flush 5 , 6, 7, 8, 9
You WIN!!!!!