testing techniques and strategies
TRANSCRIPT
![Page 1: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/1.jpg)
1
Lecture 14: Testing Techniques and Strategies
SACHIN
![Page 2: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/2.jpg)
Today’s TopicsChapters 17 & 18 in SEPA 5/eTesting Principles & TestabilityTest CharacteristicsBlack-Box vs. White-Box TestingFlow Graphs & Basis Path TestingTesting & Integration Strategies
2SACHIN
![Page 3: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/3.jpg)
Software TestingOpportunities for human error
Specifications, design, codingCommunication
“Testing is the ultimate review”Can take 30-40% of total effortFor critical apps, can be 3 to 5 times all
other efforts combined!
3SACHIN
![Page 4: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/4.jpg)
Testing ObjectivesExecute a program with the intent of
finding errorsGood tests have a high probability of
discovering errorsSuccessful tests uncover errors‘No errors found’: not a good test!Verifying functionality is a secondary goal
4SACHIN
![Page 5: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/5.jpg)
Testing PrinciplesTests traceable to requirementsTests planned before testing Pareto principle: majority of errors traced
to minority of componentsComponent testing first, then
integrated testingExhaustive testing is not possibleIndependent tests: more effective
5SACHIN
![Page 6: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/6.jpg)
Software Testability
OperabilityObservabilityControllabilityDecomposability
6
Characteristics that lead totestable software:
• Simplicity• Stability• Understandability
SACHIN
![Page 7: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/7.jpg)
Operability
System has few bugsNo bugs block execution of testsProduct evolves in functional stages
7
The better it works, the more efficientlyit can be tested
SACHIN
![Page 8: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/8.jpg)
Observability
Distinct output for each inputStates & variables may be queriedPast states are loggedFactors affecting output are visibleIncorrect output easily identifiedInternal errors reportedSource code accessible
8
What you see is what you test
SACHIN
![Page 9: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/9.jpg)
Controllability
All possible outputs can be generated by some input
All code executable by some inputStates, variables directly controlledInput/output consistent, structuredTests are specified, automated, and
reproduced
9
The better we can control the software,the more the testing can be automated
SACHIN
![Page 10: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/10.jpg)
Decomposability
Independent modulesModules can be tested separately
10
By controlling the scope of testing, wecan more quickly isolate problems andperform smarter retesting
SACHIN
![Page 11: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/11.jpg)
Simplicity
Minimum feature setMinimal architectureCode simplicity
11
The less there is to test, the morequickly we can test it
SACHIN
![Page 12: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/12.jpg)
Stability
Changes made to system:are infrequentare controlleddon’t invalidate existing tests
Software recovers from failure
12
The fewer the changes, the fewer thedisruptions to testing
SACHIN
![Page 13: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/13.jpg)
Understandability
Design is well-understoodDependencies are well understoodDesign changes are communicatedDocumentation is:
accessiblewell-organizedspecific, detailed and accurate
13
The fewer the changes, the fewer thedisruptions to testing
SACHIN
![Page 14: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/14.jpg)
Test CharacteristicsGood test has a high probability of finding
an errorGood test is not redundantA good test should be “best of breed”A good test is neither too simple nor too
complex
14SACHIN
![Page 15: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/15.jpg)
Test Case Design‘Black Box’ Testing
Consider only inputs and outputs‘White Box’ or ‘Glass Box’ Testing
Also consider internal logic paths, program states, intermediate data structures, etc.
15SACHIN
![Page 16: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/16.jpg)
White-Box TestingGuarantee that all independent paths have
been testedExercise all conditions for ‘true’ and ‘false’Execute all loops for boundary conditionsExercise internal data structures
16SACHIN
![Page 17: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/17.jpg)
Why White-Box Testing?More errors in ‘special case’ code which is
infrequently executedControl flow can’t be predicted accurately
in black-box testingTypo errors can happen anywhere!
17SACHIN
![Page 18: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/18.jpg)
Basis Path TestingWhite-box method [McCabe ‘76]Analyze procedural designDefine basis set of execution pathsTest cases for basis set execute every
program statement at least once
18SACHIN
![Page 19: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/19.jpg)
Basis Path Testing [2]
19
Flow Graph: Representation of Structured Programming Constructs
[From SEPA 5/e]
SACHIN
![Page 20: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/20.jpg)
20
Cyclomatic Complexity
V(G)=E-N+2 = 4
Independent Paths1: 1,112: 1,2,3,4,5,10,1,113: 1,2,3,6,8,9,10,1,114: 1,2,3,6,7,9,10,1,11
V(G): upper bound on number of teststo ensure all code has been executed
[From SEPA 5/e]
SACHIN
![Page 21: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/21.jpg)
Black Box TestingFocus on functional requirementsIncorrect / missing functionsInterface errorsErrors in external data accessPerformance errorsInitialization and termination errors
21SACHIN
![Page 22: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/22.jpg)
Black Box Testing [2]How is functional validity tested?What classes of input will make good test
cases?Is the system sensitive to certain inputs?How are data boundaries isolated?
22SACHIN
![Page 23: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/23.jpg)
Black Box Testing [3]What data rates and volume can the system
tolerate?What effect will specific combinations of data
have on system operation?
23SACHIN
![Page 24: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/24.jpg)
Comparison TestingCompare software versions“Regression testing”: finding the outputs
that changed Improvements vs. degradationsNet effect depends on frequency and
impact of degradationsWhen error rate is low, a large corpus can
be used
24SACHIN
![Page 25: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/25.jpg)
Generic Testing StrategiesTesting starts at module level and moves
“outward”Different testing techniques used at
different timesTesting by developer(s) and independent
testersTesting and debugging are separate
activities
25SACHIN
![Page 26: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/26.jpg)
Verification and ValidationVerification
“Are we building the product right?”Validation
“Are we building the right product?”Achieved by life-cycle SQA activities,
assessed by testing“You can’t create quality by testing”
26SACHIN
![Page 27: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/27.jpg)
27
Organization of Testing
[From SEPA 5/e]
SACHIN
![Page 28: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/28.jpg)
28
Logarithmic Poissonexecution-time model
With sufficient fit,model predicts testingtime required to reachacceptable failure rate
[From SEPA 5/e]
SACHIN
![Page 29: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/29.jpg)
29
[From SEPA 5/e]
SACHIN
![Page 30: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/30.jpg)
30
PRO: Higher-level (logic) modules tested earlyCON: Lower-level (reusable) modules tested late
[From SEPA 5/e]
SACHIN
![Page 31: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/31.jpg)
31
PRO: Lower-level (reusable) modules tested earlyCON: Higher-level (logic) modules tested late
[From SEPA 5/e]
SACHIN
![Page 32: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/32.jpg)
Hybrid ApproachesSandwich Integration: combination of top-
down and bottom-upCritical Modules
address several requirementshigh level of controlcomplex or error pronedefinite performance requirements
Test Critical Modules ASAP!
32SACHIN
![Page 33: Testing Techniques and Strategies](https://reader033.vdocument.in/reader033/viewer/2022061115/5462d59bb1af9f9d3d8b479f/html5/thumbnails/33.jpg)
33SACHIN