software testing
DESCRIPTION
TRANSCRIPT
![Page 1: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/1.jpg)
Software Testing Strategies
based onChapter 13
1
![Page 2: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/2.jpg)
Software Testing
2
Testing is the process of exercising a program with the Testing is the process of exercising a program with the specific intent of finding errors prior to delivery specific intent of finding errors prior to delivery to the end user.to the end user.
errorserrors
requirements conformancerequirements conformance
performanceperformance
an indicationan indicationof qualityof quality
![Page 3: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/3.jpg)
Who Tests the Software?
3
developerdeveloper independent testerindependent tester
Understands the system Understands the system
but, will test "gently"but, will test "gently"
and, is driven by "and, is driven by "deliverydelivery""
Must learn about the system,Must learn about the system,but, will attempt to but, will attempt to breakbreak it it
and, is driven by and, is driven by qualityquality
![Page 4: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/4.jpg)
Software TestingGoalsTypes of testsLevels of testsTest measuresTest plan
![Page 5: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/5.jpg)
GoalsVerification: Have we built the software
right?
Validation: Have we built the right software?
Bug-free, meets specs
Meets customers’ needs
Are Verification and Validation different or variation of the same ideaMy argument: “meeting specs” should equal “meeting customer needs”... not generally true (my kitchen, satellite systems)
![Page 6: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/6.jpg)
Software TestingGoalsTypes of testsLevels of testsTest measuresTest plan
most of this discussion focuses on verification(more specifically bug testing)
![Page 7: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/7.jpg)
Types of testsBlack boxWhite box
![Page 8: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/8.jpg)
Black box tests
input output
interface
1. Does it perform the specified functions?2.Does it handle obvious errors in input?3.Ariane5 – lousy error handling4.Classic ints vs floats, yards vs meters Black box should catch these if there is adequate “test coverage”
![Page 9: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/9.jpg)
Example: ordered list of ints
class OrdInts
create
getFirst
getNext
insert
delete
L=create()L.insert(5)L.insert(-1)L.insert(-1)p=L.getFirst()print (p)L.delete(p)p=L.getFirst()print(p)p=L.getNext(p)print(p)p=L.getNext(p)
-1-15error
![Page 10: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/10.jpg)
Black box testsAdvantage: black box tester≠developer is
unbiased by implementation details. e.g., Use Case testing, just work through all the Use Cases
Disadvantage: black box tester is uninformed about implementation detailsunnecessary tests – test same thing in different
wayinsufficient tests – can miss the extremes,
especially if actual use follows a different pattern
![Page 11: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/11.jpg)
black box tests
Input Code
choose good distribution of input – hope good distribution of code tested
x
x
x x
![Page 12: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/12.jpg)
unnecessary tests
Input Code
large range of input may exercise a small part of codee.g., operator test of satellite control stations, run through eachinput and output light/key option. Testing same functions, whereas no one had a test for my map function.
![Page 13: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/13.jpg)
insufficient tests
Input Code
a small range of input may exercise a large range of codebut can you ‘know’ this without knowing the code? Did we miss the 20%
![Page 14: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/14.jpg)
sufficient tests
Input Code
complexcode
a small range of input may exercise a small but important/error-prone region of code
![Page 15: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/15.jpg)
White box tests Based on code
test 1
test 2
![Page 16: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/16.jpg)
Example: ordered list of intsclass ordInts {
public: …
private: int vals[1000];
int maxElements=1000;…
}
![Page 17: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/17.jpg)
Example: ordered list of ints
bool testMax(){ L=create(); num=maxElements; for (int i=0; i<=num; i++)
print iL.insert(i)
print maxElements;}
![Page 18: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/18.jpg)
White box testsAdvantage:
design tests to achieve good code coverage and avoid duplication
can stress complicated, error-prone codecan stress boundary values (fault injection)
Disadvantage:tester=developer may have biasif code changes, tests may have to be
redesigned (is this bad?)
![Page 19: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/19.jpg)
Example: ordered list of ints
class OrdInts
create
getFirst
getNext
insert
delete
L=create()L.insert(1)L.insert(2)L.insert(3)
L.insert(1001)p=L.getFirst()print(p)p=L.getNext(p)print p
123
…
……
![Page 20: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/20.jpg)
Types of testsBlack box: test based on interface, through
interfaceWhite box: test based on code, through code
Testing strategy can/should include all approaches!
My experience: Black Box = non developer, outside testing idiots in your case who?White Box = developer, part of development process in your case who?Gray Box = hated, non developer within developer organization in your case who?
![Page 21: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/21.jpg)
Software Testing
21
White-Box TestingWhite-Box Testing
... our goal is to ensure that ... our goal is to ensure that allall statements and conditionsstatements and conditions have have
been executed at least been executed at least onceonce ... ...
Black-BoxBlack-Box Testing Testing
requirementsrequirements
eventseventsinputinput
outputoutput
![Page 22: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/22.jpg)
Basis Path Testing
22
First, we compute the cyclomatic complexity:
number of simple decisions + 1
or
number of enclosed areas + 1
In this case, V(G) = 4
White-Box TestingWhite-Box Testing
![Page 23: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/23.jpg)
Basis Path Testing
23
Next, we derive the Next, we derive the independent paths:independent paths:
Since V(G) = 4,Since V(G) = 4,there are four pathsthere are four paths
Path 3: 1,2,3,6,7,8Path 3: 1,2,3,6,7,8Path 2: 1,2,3,5,7,8Path 2: 1,2,3,5,7,8Path 1: 1,2,4,7,8Path 1: 1,2,4,7,8
Path 4: 1,2,4,7,2,4,...7,8Path 4: 1,2,4,7,2,4,...7,8
Finally, we derive testFinally, we derive testcases to exercise these cases to exercise these paths.paths.
11
22
3344
55 66
77
88
White-Box TestingWhite-Box Testing
A number of industry studies have indicated A number of industry studies have indicated that the higher V(G), the higher the probability that the higher V(G), the higher the probability or errors.or errors.
![Page 24: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/24.jpg)
Loop Testing
24
Nested Nested LoopsLoops
ConcatenatedConcatenated Loops Loops Unstructured Unstructured
LoopsLoops
Simple Simple looploop
White-Box TestingWhite-Box Testing
Why is loop testing important?
![Page 25: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/25.jpg)
Equivalence Partitioning & Boundary Value Analysis
25
If x = 5 then …
What would be the equivalence classes?
If x > -5 and x < 5 then …
White-Box TestingWhite-Box TestingBlack-Box Black-Box
TestingTesting
![Page 26: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/26.jpg)
Comparison Testing
26
Used only in situations in which the reliability of software is absolutely critical (e.g., human-rated systems)Separate software engineering teams develop
independent versions of an application using the same specification
Each version can be tested with the same test data to ensure that all provide identical output
Then all versions are executed in parallel with real-time comparison of results to ensure consistency
Black-BoxBlack-Box Testing Testing
![Page 27: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/27.jpg)
Levels of tests
UnitIntegrationSystemSystem integration
white
black
![Page 28: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/28.jpg)
Testing measures (white box)Code coverage – individual modulesPath coverage – sequence diagramsCode coverage based on complexity – test
of the risks, tricky part of code (e.g., Unix “you are not expected to understand this” code)
![Page 29: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/29.jpg)
Levels of Testing
29
Unit testing Integration testing Validation testing
Focus is on software requirements System testing
Focus is on system integration Alpha/Beta testing
Focus is on customer usage Recovery testing
forces the software to fail in a variety of ways and verifies that recovery is properly performed Security testing
verifies that protection mechanisms built into a system will, in fact, protect it from improper penetration
Stress testing executes a system in a manner that demands resources in abnormal quantity, frequency, or volume
Performance Testing test the run-time performance of software within the context of an integrated system
![Page 30: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/30.jpg)
Unit TestingTests the smallest individually executable code
units.Usually done by programmers. Test cases might be selected based on code, specification, intuition,
etc.
Tools:Test driver/harnessCode coverage analyzerAutomatic test case generator
![Page 31: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/31.jpg)
Integration TestingTests interactions between two or more units or components. Usually done by programmers. Emphasizes interfaces.
Issues: In what order are units combined?How do you assure the compatibility and
correctness of externally-supplied components?
![Page 32: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/32.jpg)
Integration TestingHow are units integrated? What are the
implications of this order?
Top-down => need stubs; top-level tested repeatedly.
Bottom-up => need drivers; bottom-levels tested repeatedly.
Critical units first => stubs & drivers needed; critical units tested repeatedly.
![Page 33: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/33.jpg)
Integration TestingPotential Problems:Inadequate unit testing.Inadequate planning & organization for
integration testing.Inadequate documentation and testing of
externally-supplied components.
![Page 34: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/34.jpg)
Stages of Testing
Testing in the LargeSystem TestingEnd-to-End Testing Operations Readiness TestingBeta Testing Load TestingStress TestingPerformance TestingReliability Testing Regression Testing
![Page 35: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/35.jpg)
System Testing
Test the functionality of the entire system.
Usually done by professional testers.
![Page 36: Software testing](https://reader036.vdocument.in/reader036/viewer/2022081412/54558d34af79593f4a8b89e3/html5/thumbnails/36.jpg)
Realities of System Testing
Not all problems will be found no matter how thorough or systematic the testing.
Testing resources (staff, time, tools, labs) are limited.
Specifications are frequently unclear/ambiguous and changing (and not necessarily complete and up-to-date).
Systems are almost always too large to permit test cases to be selected based on code characteristics.