Download - Classic Mistakes in Software Testing
![Page 1: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/1.jpg)
Classic Testing Mistakes:Revisited
Matthew [email protected]
Presented at the Better Software ConferenceSan Francisco, CA - Sept. 21st, 2005
Contributing peer reviewers:
James Bach
Paul Carvalho
Michael Kelly
Harry Robinson
![Page 2: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/2.jpg)
Organization
• Classic Mistakes: A different approach
• The mistakes enumerated– Test Management Mistakes– Test Automation Mistakes– Development Mistakes– Test Strategy Mistakes
• Root Causes
• What to do tomorrow
![Page 3: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/3.jpg)
On the shoulders of pioneers
![Page 4: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/4.jpg)
![Page 5: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/5.jpg)
![Page 6: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/6.jpg)
Classic Mistake #1:De-humanize the test process
Test Management Mistakes
AKA Management by Spreadsheet,
Management by Email,
Management by MS Project …
![Page 7: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/7.jpg)
Classic Mistake #2: Testers Responsible for Quality
Quality Assurance
“It’s strange that QA let that bug slip through”
Test Management Mistakes
![Page 8: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/8.jpg)
Classic Mistake #3:IV&V Determines ship date
Do they really?
Test Management Mistakes
![Page 9: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/9.jpg)
Classic Mistake #4:Task-based status reporting
• Examples:– Testing is “on schedule”– Testing “should be done by Tuesday”
• Consequences– Loss of credibility– Bad information for decision makers
Test Management Mistakes
![Page 10: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/10.jpg)
Classic Mistake #5:Evaluating testers by bugs found
… and developers by number of bugs injected
• Consequences:– Friction– Focus on easy-to-find yet trivial bugs
(usability)– Information hiding
Test Management Mistakes
![Page 11: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/11.jpg)
Classic Mistakes #6Inappropriate Models for Test Improvement
•NO
![Page 12: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/12.jpg)
Classic Mistake #7:Lack of test training for developers
• Testing is a skill.
• It won’t appear like magic.
Development Testing Mistakes
![Page 13: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/13.jpg)
Classic Mistake #8:Separate devs and testers
• To create friction, emphasize division
• Anything that increases the length of the feedback loop is bad.
• To improve get rid of waste and tighten the feedback loop.
Development Testing Mistakes
![Page 14: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/14.jpg)
Mistake #9:When late, add Test Automation
• Someone has to learn the tool
• Someone has to record the scripts
Test Automation Mistakes
![Page 15: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/15.jpg)
Mistake #10:Mine Field Test Automation
Test Automation Mistakes
![Page 16: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/16.jpg)
Mistake #11:Hiring for test tool skills
• Technology skills can be taught
• Talent can’t
• The “Hit the Ground Running” Argument
Test Automation Mistakes
![Page 17: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/17.jpg)
Classic Mistake #12:Insufficient diversity in test strategy
• Examples:– Only requirements based testing– Only coverage testing
• Consequence:– Missing entire classifications of defects
Test Strategy Mistakes
![Page 18: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/18.jpg)
Classic Mistake #13:Over-reliance on scripted testing
All the testing we did, meticulously pulling down every menu and seeing if it worked right, didn't uncover the showstoppers that made it impossible to do what the product was intended to allow. Trying to use the product, as a customer would, found these showstoppers in a minute.
- Joel Spolsky, JoelOnSoftware.com
Test Strategy Mistakes
![Page 19: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/19.jpg)
Mistake #14:Untrained exploratory Testing
Test Strategy Mistakes
• “Just think creatively”
• “Try to break it”
• Exploratory Testing is a discipline
![Page 20: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/20.jpg)
Classic Mistake #15: Test ‘Engineers’ and ‘Executors’
Test Strategy Mistakes
![Page 21: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/21.jpg)
Classic Mistake #16:Vacuous Documentation
• Examples:– The issue resolution document– Physical signoff/check marks– Elaborate test case templates
• Consequence:– Time spent documenting is time not spent
testing
Test Strategy Mistakes
![Page 22: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/22.jpg)
Mistake #17:Trying to fix things beyond your reach
The Meta-Mistake
![Page 23: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/23.jpg)
Don’t treat the symptoms
Find & Fix the root cause!
![Page 24: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/24.jpg)
Root Cause #1Lack of Systems Thinking in Testing
• The law of unintended consequences
Root Causes
![Page 25: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/25.jpg)
Root Cause #2:Translation Problems
Example:
- “You need to completely test this module”
Root Causes
![Page 26: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/26.jpg)
Root Cause #3:Process Myopia
• Example:– The [in]famous Issue Resolution Document– “We don’t do things that way here”– Elevating process over skills
• Solutions: – The ear of the king / History Lessons– If your boss doesn’t care – ignore it
Root Causes
![Page 27: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/27.jpg)
Root Cause #4:Technology Myopia
• Example– “Use XML on the next project”– “I just bought 5 copies of WinRunner …”
• Solution: – If you’re technical, they need you to do it– If you’re a manager, focus on business impact
and risk
Root Causes
![Page 28: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/28.jpg)
Root Cause #5:Fred Taylor in the organization
• Examples:– Factory Mentality– High Specialization– Mixing of skill sets is verboten
• Solutions:– Peopleware, or anything by Weinberg– First Break all the rules – Jim Collins– Lead. Insulate your team.
Root Causes
![Page 29: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/29.jpg)
A ‘new’ methodology
• Root Cause Analysis• Pareto Analysis• Drive out waste/tighten the feedback loop
• Then worry about better practices
(Image from Rapid Development, (c) 1996 by Steve McConnell. Used with permission from the author)
What to do tomorrow
![Page 30: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/30.jpg)
Why
• New practices take permission
• It is hard (but possible) to get more done by adding work
• It is easy to get more done by subtracting work
• So start by looking to remove worst practices
![Page 31: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/31.jpg)
What to do tomorrow
• Discuss
• Q&A
![Page 32: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/32.jpg)
Where to go for more• “Classic Mistakes in Software Testing”, Brian
Marick, STQE, 1997
• Rapid Development, Steve McConnell
• An Introduction to general systems thinking, Gerold Weinberg, 1975
• Lessons Learned in Software Testing, Kaner, Bach, Pettichord
![Page 33: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/33.jpg)
Bonus Section
![Page 34: Classic Mistakes in Software Testing](https://reader038.vdocument.in/reader038/viewer/2022102706/557e7928d8b42a7b5c8b55b8/html5/thumbnails/34.jpg)
Root Cause #6:Pressure for short-term results
• Example:– “Ship to make 4th quarter numbers”
• Putting off problems instead of addressing them
• Solution:– Save your team– Professionalism means something– The Quake Example
Root Causes