Download - Better Testing
![Page 1: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/1.jpg)
better testingbetter testing
© 2006 Reginald Braithwaitewww.braithwaite-lee.com
© 2006 Reginald Braithwaitewww.braithwaite-lee.com
![Page 2: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/2.jpg)
who are we?who are we?
We ship software.
We have operational authority over the process, the “how.” And we are held accountable for the consequences of our choices.
We ship software.
We have operational authority over the process, the “how.” And we are held accountable for the consequences of our choices.
![Page 3: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/3.jpg)
matthew 8:9matthew 8:9
“For I also am a man under authority, having under myself soldiers: and I say to this one, Go, and he goeth; and to another, Come, and he cometh; and to my servant, Do this, and he doeth it.”
“For I also am a man under authority, having under myself soldiers: and I say to this one, Go, and he goeth; and to another, Come, and he cometh; and to my servant, Do this, and he doeth it.”
![Page 4: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/4.jpg)
what we won’t saywhat we won’t say
We aren’t going to explain why we should test. We aren’t going to talk about how to become a QA manager. We’re looking for the extra edge.
This is about how to improve the testing we already do and already trust.
We aren’t going to explain why we should test. We aren’t going to talk about how to become a QA manager. We’re looking for the extra edge.
This is about how to improve the testing we already do and already trust.
![Page 5: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/5.jpg)
psyche: hardly any metrics
psyche: hardly any metrics
In the South Seas there is a cargo cult of people…
Richard Feynman
In the South Seas there is a cargo cult of people…
Richard Feynman
![Page 6: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/6.jpg)
“The function of QA is to know (and articulate) the quality of the product at all times in the development cycle.”
Jim McCarthy
“The function of QA is to know (and articulate) the quality of the product at all times in the development cycle.”
Jim McCarthy
![Page 7: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/7.jpg)
“A management report is a report that is used to make a management decision. All other reports are window dressing.”
Peter Holden
“A management report is a report that is used to make a management decision. All other reports are window dressing.”
Peter Holden
![Page 8: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/8.jpg)
holden defines ‘better testing’
holden defines ‘better testing’
Any test that provides information used to make a management decision is better than a test that only provides window dressing.
Any test that provides information used to make a management decision is better than a test that only provides window dressing.
![Page 9: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/9.jpg)
the management decisionthe management decision
![Page 10: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/10.jpg)
ship, continue, or quit?ship, continue, or quit?
This is the fundamental management decision in software development. Even if you feel you have no choice about quitting, you must be able to articulate whether you can ship.
This is the fundamental management decision in software development. Even if you feel you have no choice about quitting, you must be able to articulate whether you can ship.
![Page 11: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/11.jpg)
We don’t go on because it’s ready;we go on because it’s 11:30.”
Lorne Michaels
We don’t go on because it’s ready;we go on because it’s 11:30.”
Lorne Michaels
“
![Page 12: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/12.jpg)
testing readinesstesting readiness
How many bugs?
How much functionality?
How many bugs?
How much functionality?
![Page 13: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/13.jpg)
the end?the end?
You have a plan. Execute on it.
If you don’t ship, you must not be trying hard enough. Right?
You have a plan. Execute on it.
If you don’t ship, you must not be trying hard enough. Right?
![Page 14: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/14.jpg)
the planthe plan
Measure whether we can ship at regular intervals. When the answer is ‘ship,’ ship.
What’s the problem?
Measure whether we can ship at regular intervals. When the answer is ‘ship,’ ship.
What’s the problem?
![Page 15: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/15.jpg)
ass, you, meass, you, me
The plan assumes that the software becomes ‘more ready’ over time.
Must defects fall over time?
Must functionality increase towards readiness over time?
The plan assumes that the software becomes ‘more ready’ over time.
Must defects fall over time?
Must functionality increase towards readiness over time?
![Page 16: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/16.jpg)
sotto vocesotto voce
Some teams do not report how much the software does. They report the discrepancy between what the software does and what they want it to do.
If your team works this way, your management is more complex than teams that report what the software does. But this doesn’t affect the choice of tests and test processes.
Some teams do not report how much the software does. They report the discrepancy between what the software does and what they want it to do.
If your team works this way, your management is more complex than teams that report what the software does. But this doesn’t affect the choice of tests and test processes.
![Page 17: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/17.jpg)
are we converging?are we converging?
Are defects falling over time? Are we confident they will continuing to fall? Do we know which activities present the greatest risk of introducing defects?
Is functionality increasing? Are we climbing ladders or slipping on snakes?
Are defects falling over time? Are we confident they will continuing to fall? Do we know which activities present the greatest risk of introducing defects?
Is functionality increasing? Are we climbing ladders or slipping on snakes?
![Page 18: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/18.jpg)
ass, you, me tooass, you, me too
The plan assumes that we can tell how many defects are present with an acceptable certainty.
The plan assumes that we can tell how much functionality is present with acceptable certainty.
The plan assumes that we can tell how many defects are present with an acceptable certainty.
The plan assumes that we can tell how much functionality is present with acceptable certainty.
![Page 19: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/19.jpg)
add uncertainty to readiness
add uncertainty to readiness
How many bugs with what level of certainty?
How much functionality with what level of certainty?
How many bugs with what level of certainty?
How much functionality with what level of certainty?
![Page 20: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/20.jpg)
testing, revisitedtesting, revisited
Measure how many defects and how much functionality we have.
Articulate convergence towards readiness and uncertainty about readiness.
Measure how many defects and how much functionality we have.
Articulate convergence towards readiness and uncertainty about readiness.
![Page 21: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/21.jpg)
a better plana better plan
Measure whether we can ship at regular intervals. Ship if we’re ready with acceptable certainty.
Change the production process if we aren’t converging. Change the testing process if we aren’t certain about our state.
Measure whether we can ship at regular intervals. Ship if we’re ready with acceptable certainty.
Change the production process if we aren’t converging. Change the testing process if we aren’t certain about our state.
![Page 22: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/22.jpg)
testing readinesstesting readiness
Everyone’s an expert on whether the software is ready.
My sole tip today is to get everyone on the same page.
Nothing smarts more than shipping the wrong application.
Everyone’s an expert on whether the software is ready.
My sole tip today is to get everyone on the same page.
Nothing smarts more than shipping the wrong application.
![Page 23: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/23.jpg)
getting on the same pagegetting on the same page
Shared and/or executable requirements. See FITnesse.
Screen shots are good. Movies are better. The key is to have development and QA testing the same thing.
That means that developers need to be able to reproduce every bug. If they can’t, investigate: there’s a process or environment problem.
Shared and/or executable requirements. See FITnesse.
Screen shots are good. Movies are better. The key is to have development and QA testing the same thing.
That means that developers need to be able to reproduce every bug. If they can’t, investigate: there’s a process or environment problem.
![Page 24: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/24.jpg)
testing convergencetesting convergence
You must test consistently over time.
You must test against the source repository with precision. This allows development to manage their risks.
You must test consistently over time.
You must test against the source repository with precision. This allows development to manage their risks.
![Page 25: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/25.jpg)
snakes or ladders?snakes or ladders?
You must regression test: you must know whether you are on a snake or a ladder.
Report regressions/persists/come-backs separately from new bugs. Use this information to manage production activities.
You must regression test: you must know whether you are on a snake or a ladder.
Report regressions/persists/come-backs separately from new bugs. Use this information to manage production activities.
![Page 26: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/26.jpg)
managing uncertaintymanaging uncertainty
We manage uncertainty around readiness by choosing how much to test, what to test, and when to test it.
And we build uncertainty into our management report.
We manage uncertainty around readiness by choosing how much to test, what to test, and when to test it.
And we build uncertainty into our management report.
![Page 27: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/27.jpg)
what to test?what to test?
Start with the acceptance tests for promised functionality.
Add regression tests for every bug as you go along.
It’s really that simple.
Start with the acceptance tests for promised functionality.
Add regression tests for every bug as you go along.
It’s really that simple.
![Page 28: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/28.jpg)
sumsum
Tie every test to a management decision. If it isn’t tied to a management decision, either start managing or drop the test.
Tie every management decision to tests. If it isn’t ties to tests, either stop deciding or start testing.
Tie every test to a management decision. If it isn’t tied to a management decision, either start managing or drop the test.
Tie every management decision to tests. If it isn’t ties to tests, either stop deciding or start testing.
![Page 29: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/29.jpg)
overtimeovertime
![Page 30: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/30.jpg)
what tests support these decisions?
what tests support these decisions?
Adding a function to the software without slipping the date?
Trying to work harder/smarter so we have higher quality?
“Easter? No way. We need this March 15th or you’ll be writing your résumé!”
Adding a function to the software without slipping the date?
Trying to work harder/smarter so we have higher quality?
“Easter? No way. We need this March 15th or you’ll be writing your résumé!”
![Page 31: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/31.jpg)
what decisions do we make on these tests?what decisions do we make on these tests?
Time between bug injection and detection?
Unit test code coverage and acceptance test code coverage?
Average number of come-backs per bug? Ratio of come-backs to bugs?
Average bug age?
Time between bug injection and detection?
Unit test code coverage and acceptance test code coverage?
Average number of come-backs per bug? Ratio of come-backs to bugs?
Average bug age?
![Page 32: Better Testing](https://reader035.vdocument.in/reader035/viewer/2022062216/558bcfa0d8b42aab0b8b47ac/html5/thumbnails/32.jpg)
thanks!thanks!