![Page 1: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/1.jpg)
Testing Fundamentals
Need for, Black box, White box, Minimal output
![Page 2: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/2.jpg)
Need for Testing
n Software systems are inherently complexn Large systems 1 to 3 errors per 100 lines of code (LOC)
Test–2
![Page 3: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/3.jpg)
Need for Testing
n Software systems are inherently complexn Large systems 1 to 3 errors per 100 lines of code (LOC)
n Extensive verification and validiation is required to build quality software
Test–3
![Page 4: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/4.jpg)
Need for Testing
n Software systems are inherently complexn Large systems 1 to 3 errors per 100 lines of code (LOC)
n Extensive verification and validiation is required to build quality softwaren Verification
n Does the software meet its specifications
Test–4
![Page 5: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/5.jpg)
Need for Testing
n Software systems are inherently complexn Large systems 1 to 3 errors per 100 lines of code (LOC)
n Extensive verification and validiation is required to build quality softwaren Verification
n Does the software meet its specificationsn Validation
n Does the software meet its requirements
Test–5
![Page 6: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/6.jpg)
Need for Testing
n Software systems are inherently complexn Large systems 1 to 3 errors per 100 lines of code (LOC)
n Extensive verification and validiation is required to build quality softwaren Verification
n Does the software meet its specificationsn Validation
n Does the software meet its requirements
n Testing is used to determine whether there are faults in a software system
Test–6
![Page 7: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/7.jpg)
Need for Testing
n The process of testing is to find the minimum number of test cases that can produce the maximum number of failures to achieve a desired level of confidence
Test–7
![Page 8: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/8.jpg)
Need for Testing
n The process of testing is to find the minimum number of test cases that can produce the maximum number of failures to achieve a desired level of confidence
n Cost of validation should not be under estimated
Test–8
![Page 9: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/9.jpg)
Need for Testing
n The process of testing is to find the minimum number of test cases that can produce the maximum number of failures to achieve a desired level of confidence
n Cost of validation should not be under estimated
n The cost of testing is extremely highn Anything that we can do to reduce it is worthwhile
Test–9
![Page 10: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/10.jpg)
Testing Not Enough
n Exhaustive testing is not usually possible
Test–10
![Page 11: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/11.jpg)
Testing Not Enough
n Exhaustive testing is not usually possibleTesting can determine the presence of faults, never their absence
Dikstra
Test–11
![Page 12: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/12.jpg)
Testing Not Enough
n Exhaustive testing is not usually possibleTesting can determine the presence of faults, never their absence
Dikstra
n Test to give us a high level of confidence
Test–12
![Page 13: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/13.jpg)
Test Strategy
n Identify test criterian What are the goals for comparing the system against its
specificationn Reliability, completeness, robustness
Test–13
![Page 14: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/14.jpg)
Test Strategy
n Identify test criterian What are the goals for comparing the system against its specification
n Reliability, completeness, robustness
n Identify target components for testingn In an OO system the classes and class hierarchies
Test–14
![Page 15: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/15.jpg)
Test Strategy
n Identify test criterian What are the goals for comparing the system against its specification
n Reliability, completeness, robustness
n Identify target components for testingn In an OO system the classes and class hierarchies
n Generate test casesn Produce test cases that can identify faults in an
implementation
Test–15
![Page 16: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/16.jpg)
Test Strategy
n Identify test criterian What are the goals for comparing the system against its specification
n Reliability, completeness, robustness
n Identify target components for testingn In an OO system the classes and class hierarchies
n Generate test casesn Produce test cases that can identify faults in an implementation
n Execute test cases against target components
Test–16
![Page 17: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/17.jpg)
Test Strategy
n Identify test criterian What are the goals for comparing the system against its specification
n Reliability, completeness, robustness
n Identify target components for testingn In an OO system the classes and class hierarchies
n Generate test casesn Produce test cases that can identify faults in an implementation
n Execute test cases againsr target components
n Evaluationn If expected outputs are not produced, a bug report is issued
Test–17
![Page 18: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/18.jpg)
Test Plans
n Specify a set of test input
Test–18
![Page 19: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/19.jpg)
Test Plans
n Specify a set of test input
n For each input give the expected output
Test–19
![Page 20: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/20.jpg)
Test Plans
n Specify a set of test input
n For each input give the expected output
n Run the program and document the actual output
Test–20
![Page 21: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/21.jpg)
Test Plans
n Specify a set of test input
n For each input give the expected output
n Run the program and document the actual output
n Example – square root programn Input: 4.0n Expected Output: 2.0n Actual Output: 1.99999 looks all right
n Input: -4n Expected Output: Invalid inputn Actual Output: 1.99999 Uh oh! problem here
Test–21
![Page 22: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/22.jpg)
Black Box Testing – Data Coverage
n Testing based on input and output alonen Do not consider underlying implementation
Test–22
![Page 23: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/23.jpg)
Black Box Testing – Data Coverage
n Testing based on input and output alonen Do not consider underlying implementation
n Test cases are generated from the specificationn Pre and post conditions
Test–23
![Page 24: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/24.jpg)
Black Box Testing – Data Coverage
n Testing based on input and output alonen Do not consider underlying implementation
n Test cases are generated from the specificationn Pre and post conditions
n Specific kinds of black box testingn Random testing
n Generate random inputsn Easy to generate casesn good at detecting failuesn Must be able to easily generate expected output
Test–24
![Page 25: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/25.jpg)
Black Box Testing – Data Coverage
n Testing based on input and output alonen Do not consider underlying implementation
n Test cases are generated from the specificationn Pre and post conditions
n Specific kinds of black box testingn Random testing
n Generate random inputsn Easy to generate casesn good at detecting failuesn Must be able to easily generate expected output
n Partition testingn See next slides
Test–25
![Page 26: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/26.jpg)
Partition Testing
n Cannot try all possible inputs
Test–26
![Page 27: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/27.jpg)
Partition Testing
n Cannot try all possible inputsn Partition input into equivalence classes
n Every value in a class should behave similarly
Test–27
![Page 28: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/28.jpg)
Partition Testing
n Cannot try all possible inputsn Partition input into equivalence classes
n Every value in a class should behave similarly
n Test casesn just before boundaryn just after a boundaryn on a boundaryn one from the middle of an equivalence class
Test–28
![Page 29: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/29.jpg)
Partition Testing
n Cannot try all possible inputsn Partition input into equivalence classes
n Every value in a class should behave similarly
n Test casesn just before boundaryn just after a boundaryn on a boundaryn one from the middle of an equivalence class
n Loopsn Zero times through the bodyn Once through the bodyn Many times through the body
Test–29
![Page 30: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/30.jpg)
Partition Testing examples
n Example 1 – Absolute value – 2 equivalence classes
n Values below zero and values above zero
n 5 test cases: large negative, -1, 0, +1, large positive
Test–30
![Page 31: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/31.jpg)
Partition Testing examples
n Example 2 – Tax rates – 3 equivalence classesn 0 .. $29,000 at 17%
n 29,001 .. 35,000 at 26%
n 35,001 ... at 29%
n 13 Test cases 0 1 15,000 28,999 29,000 29,001 29,002 30,000 34,999 35,000 35,001 35,002 50,000
Test–31
![Page 32: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/32.jpg)
Partition Testing examples
n Example 3 – Insert into a sorted list -- max 20 elements
Test–32
![Page 33: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/33.jpg)
Partition Testing examples
n Example 3 – Insert into a sorted list -- max 20 elements
n About 25 test cases
Test–33
![Page 34: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/34.jpg)
Partition Testing examples
n Example 3 – Insert into a sorted list -- max 20 elements
n About 25 test casesn Boundary conditions on the list
n empty boundary – length 0, 1, 2n full boundary – length 19, 20 , 21n middle of range – length 10
Test–34
![Page 35: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/35.jpg)
Partition Testing examples
n Example 3 – Insert into a sorted list -- max 20 elements
n About 25 test casesn Boundary conditions on the list
n empty boundary – length 0, 1, 2n full boundary – length 19, 20 , 21n middle of range – length 10
n Boundary conditions on insertingn just before & after first list elementn just before & after last list elementn into the middle of the list
Test–35
![Page 36: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/36.jpg)
Partition Testing examples
n Example 3 – Insert into a sorted list -- max 20 elements
n About 25 test casesn Boundary conditions on the list
n empty boundary – length 0, 1, 2n full boundary – length 19, 20 , 21n middle of range – length 10
n Boundary conditions on insertingn just before & after first list elementn just before & after last list elementn into the middle of the list
n Suppose an error occurs when adding to the upper end of a full listn Devise additional test cases to test hypothesis as to why
the error occurs
Test–36
![Page 37: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/37.jpg)
White Box Testing
n Use internal properties of the system as test case generation criteria
Test–37
![Page 38: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/38.jpg)
White Box Testing
n Use internal properties of the system as test case generation criteria
n Generate cases based onn Statement blocksn Paths
Test–38
![Page 39: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/39.jpg)
White Box Testing
n Use internal properties of the system as test case generation criteria
n Generate cases based onn Statement blocksn Paths
n Identify unintentionaln Infinite loopsn Illegal pathsn Unreachable program text (dead code)
Test–39
![Page 40: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/40.jpg)
White Box Testing
n Use internal properties of the system as test case generation criteria
n Generate cases based onn Statement blocksn Paths
n Identify unintentionaln Infinite loopsn Illegal pathsn Unreachable program text (dead code)
n Need test cases for exceptions and interrupts
Test–40
![Page 41: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/41.jpg)
Statement Coverage
n Make sure every statement in the program is executed at least once
n Examplen if a < b then c = a+b ; d = a*b /* 1 */
else c = a*b ; d = a+b /* 2 */if c < d then x = a+c ; y = b+d /* 3 */ else x = a*c ; y = b*d /* 4 */
n Statement coverage – can do with 2 testsn Execute 1 & 3 with a < b & a+b < a*b
n a = 2 ; b = 5n Execute 2 & 4 with a >= b & a*b >= a+b
n a = 5 ; b = 2
1
3 4
2
T
T
F
F
Test–41
![Page 42: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/42.jpg)
Statement Coverage
n Loops – only 1 test requiredn Execute body at least oncen If there are choices in the body, need to execute body with
enough tests to execute the statements in the body, analogous to example 1.
Body
T
F
Test–42
![Page 43: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/43.jpg)
Top Down Testing
n Test upper levels of program first
Test–43
![Page 44: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/44.jpg)
Top Down Testing
n Test upper levels of program first
n Use stubs for lower levels
Test–44
![Page 45: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/45.jpg)
Top Down Testing
n Test upper levels of program first
n Use stubs for lower levelsn Stubs are dummy procedures that have "empty"
implementations – do nothing
Test–45
![Page 46: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/46.jpg)
Top Down Testing
n Test upper levels of program first
n Use stubs for lower levelsn Stubs are dummy procedures that have "empty"
implementations – do nothingn For functions return a constant value
Test–46
![Page 47: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/47.jpg)
Top Down Testing
n Test upper levels of program first
n Use stubs for lower levelsn Stubs are dummy procedures that have "empty"
implementations – do nothingn For functions return a constant value
n Test calling sequences for procedures and simple cases for upper levels.
Test–47
![Page 48: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/48.jpg)
Bottom Up Testing
n Use test drivers
Test–48
![Page 49: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/49.jpg)
Bottom Up Testing
n Use test driversn Have complete implementation of subprograms
Test–49
![Page 50: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/50.jpg)
Bottom Up Testing
n Use test driversn Have complete implementation of subprogramsn Create a special main program to call the subprograms with
a sequence of test cases
Test–50
![Page 51: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/51.jpg)
Bottom Up Testing
n Use test driversn Have complete implementation of subprogramsn Create a special main program to call the subprograms with
a sequence of test casesn Can be interactive, semi-interactive, or non-interactive
Test–51
![Page 52: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/52.jpg)
Bottom Up Testing
n Use test driversn Have complete implementation of subprogramsn Create a special main program to call the subprograms with
a sequence of test casesn Can be interactive, semi-interactive, or non-interactive
n See minimal output test program slides
Test–52
![Page 53: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/53.jpg)
Mixed Top & Bottom Testing
n Use scaffolding
Test–53
![Page 54: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/54.jpg)
Mixed Top & Bottom Testing
n Use scaffoldingn Have some stubs
Test–54
![Page 55: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/55.jpg)
Mixed Top & Bottom Testing
n Use scaffoldingn Have some stubsn Have some special test drivers
Test–55
![Page 56: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/56.jpg)
Mixed Top & Bottom Testing
n Use scaffoldingn Have some stubsn Have some special test driversn Like the scaffolding around a building while it is being built
or repaired
Test–56
![Page 57: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/57.jpg)
Mixed Top & Bottom Testing
n Use scaffoldingn Have some stubsn Have some special test driversn Like the scaffolding around a building while it is being built
or repairedn Not a part of the final product but necessary to complete
the task
Test–57
![Page 58: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/58.jpg)
Regressive Testing
n Rerun all previous tests whenever a change is made
Test–58
![Page 59: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/59.jpg)
Regressive Testing
n Rerun all previous tests whenever a change is maden Changes can impact previously working programs
Test–59
![Page 60: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/60.jpg)
Regressive Testing
n Rerun all previous tests whenever a change is maden Changes can impact previously working programsn So retest everything
Test–60
![Page 61: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/61.jpg)
Regressive Testing
n Rerun all previous tests whenever a change is maden Changes can impact previously working programsn So retest everythingn Must also test the changes
Test–61
![Page 62: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/62.jpg)
Minimal Output Testing
n Reading and comparing expected with actual output is tedious and error prone
Test–62
![Page 63: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/63.jpg)
Minimal Output Testing
n Reading and comparing expected with actual output is tedious and error prone
n Task should be automated – especially for regressive testing
Test–63
![Page 64: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/64.jpg)
Minimal Output Testing
n Reading and comparing expected with actual output is tedious and error prone
n Task should be automated – especially for regressive testing
n Build the expected output into the test driver and compare with the actual output
Test–64
![Page 65: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/65.jpg)
Minimal Output Testing
n Reading and comparing expected with actual output is tedious and error prone
n Task should be automated – especially for regressive testing
n Build the expected output into the test driver and compare with the actual output
n Report only if expected ≠ actualn Output states which test failed, the expected and the actual
outputs
Test–65
![Page 66: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/66.jpg)
Minimal Output Testing
n Reading and comparing expected with actual output is tedious and error prone
n Task should be automated – especially for regressive testing
n Build the expected output into the test driver and compare with the actual output
n Report only if expected ≠ actualn Output states which test failed, the expected and the actual
outputs
n Successful test runs only output the messagen Tests passed
Test–66
![Page 67: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/67.jpg)
Minimal Output Testing – 2
n Example checking a stack implementation Stack list = new Stack(); verifyEmpty("1", list);
list.add("a"); verifyL1("2", list, "[ a ]");list.remove(); verifyEmpty("3", list);list.add("a");list.add("b"); verifyL2("4", list, "[ b , a ]");
if (list.contains("c")) println("5 shouldn't contain c");if (!list.contains("a")) println("6 should contain a");if (!list.contains("b")) println("7 should contain b");
list.add("c"); verifyL3("8", list, "[ c , b , a ]");list.remove(); verifyL2("9", list, "[ b , a ]");list.remove(); verifyL1("10", list, "[ a ]");list.remove(); verifyEmpty("11", list);
n Verify routines do the appropriate test and output messages
Test–67
![Page 68: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/68.jpg)
Assertions
n Running programs with a life jacket
Test–68
![Page 69: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/69.jpg)
Assertions
n Running programs with a life jacketn Assertions are tested whenever the program is executed,
unlike testing which is done off-line
Test–69
![Page 70: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/70.jpg)
Assertions
n Running programs with a life jacketn Assertions are tested whenever the program is executed, unlike
testing which is done off-line
n Assertions can be put into programs using if...then statementsn Some languages such as Eiffel have them built inn C has the assert macro
Test–70
![Page 71: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/71.jpg)
Assertions
n Running programs with a life jacketn Assertions are tested whenever the program is executed, unlike
testing which is done off-line
n Assertions can be put into programs using if...then statementsn Some languages such as Eiffel have them built inn C has the assert macro
n The condition compares expected with actual values and executes an exception if the assertion fails
Test–71
![Page 72: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/72.jpg)
Inspections & Walkthroughs
n Manual or computer aided comparisons of software development products
Test–72
![Page 73: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/73.jpg)
Inspections & Walkthroughs
n Manual or computer aided comparisons of software development productsn specifications, program text, analysis documents
Test–73
![Page 74: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/74.jpg)
Inspections & Walkthroughs
n Manual or computer aided comparisons of software development productsn specifications, program text, analysis documents
n Documents are paraphrased by the authors
Test–74
![Page 75: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/75.jpg)
Inspections & Walkthroughs
n Manual or computer aided comparisons of software development productsn specifications, program text, analysis documents
n Documents are paraphrased by the authors
n Walkthroughs are done by going through the execution paths of a program, comparing its outputs with those of the paraphrased documents
Test–75
![Page 76: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/76.jpg)
Inspections & Walkthroughs
n Manual or computer aided comparisons of software development productsn specifications, program text, analysis documents
n Documents are paraphrased by the authors
n Walkthroughs are done by going through the execution paths of a program, comparing its outputs with those of the paraphrased documents
n Walkthrough teamn Usually 4-6 people
Test–76
![Page 77: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/77.jpg)
Inspections & Walkthroughs
n Manual or computer aided comparisons of software development productsn specifications, program text, analysis documents
n Documents are paraphrased by the authors
n Walkthroughs are done by going through the execution paths of a program, comparing its outputs with those of the paraphrased documents
n Walkthrough teamn Usually 4-6 peoplen Elicit questions, facilitate discussion
Test–77
![Page 78: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/78.jpg)
Inspections & Walkthroughs
n Manual or computer aided comparisons of software development productsn specifications, program text, analysis documents
n Documents are paraphrased by the authors
n Walkthroughs are done by going through the execution paths of a program, comparing its outputs with those of the paraphrased documents
n Walkthrough teamn Usually 4-6 peoplen Elicit questions, facilitate discussionn Interactive process
Test–78
![Page 79: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/79.jpg)
Inspections & Walkthroughs
n Manual or computer aided comparisons of software development productsn specifications, program text, analysis documents
n Documents are paraphrased by the authors
n Walkthroughs are done by going through the execution paths of a program, comparing its outputs with those of the paraphrased documents
n Walkthrough teamn Usually 4-6 peoplen Elicit questions, facilitate discussionn Interactive processn Not done to evaluate people
Test–79
![Page 80: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/80.jpg)
Inspections & Walkthroughs
n Manual or computer aided comparisons of software development productsn specifications, program text, analysis documents
n Documents are paraphrased by the authors
n Walkthroughs are done by going through the execution paths of a program, comparing its outputs with those of the paraphrased documents
n Walkthrough teamn Usually 4-6 peoplen Elicit questions, facilitate discussionn Interactive processn Not done to evaluate people
Test–80
![Page 81: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/81.jpg)
Inspections & Walkthroughs
n Psychological side effects
n If a walkthrough is going to be performed, developers frequently write easy to read program text
n This clarity can help make future maintenance easier
Test–81
![Page 82: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/82.jpg)
OO Testing
n Using OO technology can have effects on three kinds of testingn Intra featuren Inter feature testingn Testing class hierarchies
Test–82
![Page 83: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/83.jpg)
OO Testing
n Using OO technology can have effects on three kinds of testingn Intra featuren Inter feature testingn Testing class hierarchies
n Intra feature testingn Features can be tested much as procedures & functions are
tested in imperative languages
Test–83
![Page 84: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/84.jpg)
OO Testing
n Using OO technology can have effects on three kinds of testingn Intra featuren Inter feature testingn Testing class hierarchies
n Intra feature testingn Features can be tested much as procedures & functions are tested
in imperative languages
n Inter feature testingn Test an entire class against an abstract data type
(specification)n Some inter-feature testing methods specify correct
sequences of feature calls, and use contracts to derive test cases
Test–84
![Page 85: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/85.jpg)
Testing Class Hierarchies
n May have re-test inherited features
Test–85
![Page 86: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/86.jpg)
Testing Class Hierarchies
n May have re-test inherited features
n For testing a hierarchy it is usually best to test from the top downn Start with base classesn Test each feature in isolationn Then test feature interactions
Test–86
![Page 87: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/87.jpg)
Testing Class Hierarchies
n May have re-test inherited features
n For testing a hierarchy it is usually best to test from the top downn Start with base classesn Test each feature in isolationn Then test feature interactions
n Build test historiesn Associate test cases with featuresn History can be inherited with the class, allowing for reuse of
test cases
Test–87
![Page 88: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/88.jpg)
Testing Classes
n Consider the class as the basic unit
Test–88
![Page 89: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/89.jpg)
Testing Classes
n Consider the class as the basic unit
n Like unit (module) testingn Usually tested in isolationn Surround class with stubs and drivers
Test–89
![Page 90: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/90.jpg)
Testing Classes
n Consider the class as the basic unit
n Like unit (module) testingn Usually tested in isolationn Surround class with stubs and drivers
n Exercise each feature in turnn Create test cases for each feature
Test–90
![Page 91: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/91.jpg)
Testing Classes
n Consider the class as the basic unit
n Like unit (module) testingn Usually tested in isolationn Surround class with stubs and drivers
n Exercise each feature in turnn Create test cases for each feature
n Often a test driver is createdn Simple menu that can be used to exercise each feature with
inputs from a test file
Test–91
![Page 92: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/92.jpg)
Testing Classes
n Consider the class as the basic unit
n Like unit (module) testingn Usually tested in isolationn Surround class with stubs and drivers
n Exercise each feature in turnn Create test cases for each feature
n Often a test driver is createdn Simple menu that can be used to exercise each feature with
inputs from a test file
n Need to recompile when switching drivers
Test–92
![Page 93: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/93.jpg)
Integration Testing
n Testing with all the components assembled
Test–93
![Page 94: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/94.jpg)
Integration Testing
n Testing with all the components assembled
n Focuses on testing the interfaces between components
Test–94
![Page 95: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/95.jpg)
Integration Testing
n Testing with all the components assembled
n Focuses on testing the interfaces between components
n Usually want to do this in a piece by piece fashion
Test–95
![Page 96: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/96.jpg)
Integration Testing
n Testing with all the components assembled
n Focuses on testing the interfaces between components
n Usually want to do this in a piece by piece fashionn Avoid a big bang test
Test–96
![Page 97: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/97.jpg)
Integration Testing
n Testing with all the components assembled
n Focuses on testing the interfaces between components
n Usually want to do this in a piece by piece fashionn Avoid a big bang testn Incremental testing and incremental integration is
preferable
Test–97
![Page 98: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/98.jpg)
Integration Testing
n Testing with all the components assembled
n Focuses on testing the interfaces between components
n Usually want to do this in a piece by piece fashionn Avoid a big bang testn Incremental testing and incremental integration is
preferablen Integrate after unit testing a component
Test–98
![Page 99: Testing Fundamentals - York University · Testing Fundamentals Need for, Black box, White box, Minimal output . Need for Testing n Software systems are inherently complex n Large](https://reader030.vdocument.in/reader030/viewer/2022040805/5e426454211e8b76af615427/html5/thumbnails/99.jpg)
Integration Testing
n Testing with all the components assembled
n Focuses on testing the interfaces between components
n Usually want to do this in a piece by piece fashionn Avoid a big bang testn Incremental testing and incremental integration is preferable
n Integrate after unit testing a component
n Bottom up and top down approaches may be used
Test–99