software test coverage testtcov 1 test coverage measures code coverage measurement levels of code...
TRANSCRIPT
Software Test Coverage
TEST TCOV
1 Test Coverage Measures
Code Coverage Measurement
Levels of Code Coverage
Method Coverage
Method, Statement, Branch & Path Coverage
White Box Test Coverage
Data Coverage Measurement
Levels of Data Coverage
Class Coverage
Interface Coverage
State Coverage
Data Table Coverage
2
3
6
7
5
4
8
9
10
13 Data Coverage Report
Functional Coverage Measurement
Levels of Functional Coverage
Black Box Test Coverage
Functional Test Coverage
Predicted Error Coverage
Inheriting Test Coverage
Non-Transitivity of Test Coverage
CRUD Object Access Coverage
CRUD Object Access Coverage Matrix
Document Coverage
Differences in Test Coverage
14
15
18
19
17
16
20
21
22
11 23
12 24
Test Coverage Measures
TEST TCOV-1
Test Coverage
FunctionalCoverage
CodeCoverage
DataCoverage
Object Coverage
Attribute Coverage
Value Coverage
State Coverage
Function Coverage
Function Outcome Coverage
Function Chain Coverage
Function State Coverage
Method Coverage
Statement Coverage
Branch Coverage
Path Coverage
Code Coverage Measurement
TEST TCOV-2
MethodCoverage
Path
Coverage
Coverage
Coverage
Branch
Statement Test Coverage = 100%could mean:
Every Method was invoked
Every Statement was executed
Every Branch was traversed
Every Control Path was traversed
Levels of Code Coverage
TEST TCOV-3
Method coverage
Interface coverage
Statement coverage
Branch coverage
Path coverage
1
2
3
4
5
Increasing Reliability
Increasing Costs
A_Retcode A_Method (Param_A1, Param_A2, Param_A3){
Invoke Method
B_Retcode B_Method (Param_B1, Param_B2, Param_B3){ if (Param_B1 > Attr_B9) return 1;
}Statement Coverage
C_Retcode C_Method (Param_C1, Param_C2, Param_C3){ if (Param_C1 > Attr_C9) return 1 else continue; return 0;}
Branch Coverage
1 Test
1 Test
2 Tests
Method Coverage
TEST TCOV-4
F1
B1
B2
B3
A4
E2
E1
A2
A3
Method Coverage:(1 test case)
Statement Coverage:(2 test cases)
Branch Coverage:(3 test cases)
Path Coverage:(5 test cases)
F1, B1, E1 (Method Coverage)
F1, B1, E1 (Statement Coverage)F1, B1, B2, A2, B3, A3, A4, E2
F1, B1, E1 (Branch Coverage)F1, B1, B2, A2, B3, A3, A4, E2F1, B1, B2, B3, A4, E2
F1, B1, E1 (Path Coverage)F1, B1, B2, A2, B3, A3, A4, E2F1, B1, B2, A2, B3, A4, E2F1, B1, B2, B3, A3, A4, E2F1, B1, B2, B3, A4, E3
Test Cases
Method, Statement, Branch and Path Coverage
TEST TCOV-5
C1 = All Methods are executed at least once.(Method Coverage)
C2 = All Statements of all Methods are executed at least once. (Statement Coverage)
C3 = All Branches of all Methods are executed at least once.(Branch Coverage)
D1 = All Attributes of an Object are altered. (Attribute Coverage)
D2 = All Attributes of an Object take on all specified values.(Value Coverage)
D3 = All Combinations of all values occur.(State Coverage)
S1 = All Messages are sent at least once.(Interface Coverage)
S2 = All Parameters of all Messages are altered. (Parameter Coverage)
S3 = All Combinations of all Parameters occur.(Interface State Coverage)
White-Box Coverage Measures (Class Test)
TEST TCOV-6
Data Coverage Measurement
TEST TCOV-7
ObjectCoverage
State
Coverage
Coverage
Coverage
Value
Attribute Test Coverage = 100%could mean:
Every Object was instantiated
Every Attribute was altered
Every relevant value was set
Every possible state was generated.
Levels of Data Coverage
TEST TCOV-8
1
2
3
4
5
Increasing Reliability
Increasing Costs
Window/File/DB Coverage
Object Coverage
Attribute Coverage
Value Coverage
State Coverage
Object StatesMethods
AccountNumberAccountOwner
AccountBalanceAccountStatus
CreditLimit
Open_Account
Deposit_Money
Withdraw_Money
Transfer_Money
Close_Account
Account non existant
AccountBalance = 0
AccountBalance = plus
AccountBalance = minus,but < CreditLimit
AccountBalance = minusand > CreditLimit
Account closed
Attributes
How many test cases are required to test all Methods with all Object states?
Class Coverage
TEST TCOV-9
Parameter Coverage
= the Interface is generated one time with a repesentative value for each individual Parameter
Value Coverage
= the Interface is generated as many times as there are representative values for each Parameter.
State Coverage
= the Interface is generated as many times as there are combinations of representative values for all Parameters.
Interface Coverage
TEST TCOV-10
All Paths
All Paths
All Branches
All Branches
AllState
Transitions
AllState
Transitions
AllStates
AllStates
AllEvents
AllEvents
Subsumption Hierarchy
State Coverage
TEST TCOV-11
1 OBJECT-A
2 ATTR-A1 'A' 'B' 'C'2 ATTR-A2 100 200 300 4002 ATTR-A3 'XYZ' 'ABC' 'EFG'2 ATTR-A4 -1 0 +1
1 PANEL-B
2 FIELD-B1 50 100 1502 FIELD-B2 100 -50 02 FIELD-B3 'XY' 'AB' 'EF'2 FIELD-B4 'B' 'A' 'C'
Value Coverage:
Every relevant Value of every Attribute & Field should be set at least one time.(you need as many Objects as the longest array of values)
Data Table Coverage
TEST TCOV-12
Data Coverage Report
TEST TCOV-13
| DATA COVERAGE REPORT | | MODUL NAME: SVVL104 DATUM: 98.05.17 |+--------------------------------------------------------------------------------------------+| DATA NAME |RECORD NAME |BASE NAME|PREDIC.|INPUT|OUTPUT|NOT USED|+--------------------------------------------------------------------------------------------+| LISTNUMMER |FILLER |SVVL104 | | * | * | * || K-ENDE-DER-LISTE |ALLGEMEINE-VIS-KONSTANTEN|SVVL104 | | * | | * || CURRENT-DATE-MONAT |ALLGEMEINE-VIS-VARIABLEN |SVVL104 | | | * | || CURRENT-DATE-TAG |ALLGEMEINE-VIS-VARIABLEN |SVVL104 | | | * | || CURRENT-DATE-JAHR |ALLGEMEINE-VIS-VARIABLEN |SVVL104 | | | * | || SH-ENDE-DER-LISTE |ALLGEMEINE-VIS-SCHALTER |SVVL104 | * | | * | || SUMME-AWGR-VERK-MG |LISTSTEUERDATEN |SVVL104 | | | * | || FIRMA |ABSATZWEGGRUPPENSTATISTIK|SVVL104 | | * | * | || TEXT-ENDE-DER-AUSWERTUNG |ABSATZWEGGRUPPENSTATISTIK|SVVL104 | | | * | || FIRMA-GRUPPE |LISTPARAMETER |SVVL104 | | * | * | || LISTENNUMMER |LISTPARAMETER |SVVL104 | * | * | * | || FIRMA-GRUPPE |LISTPARAMETER |SVVL104 | | | * | || LISTENNUMMER |LISTPARAMETER |SVVL104 | | | * | |+--------------------------------------------------------------------------------------------+| || TOTAL NUMBER OF (NOT AUTOMATIC) DATA ITEMS REFERENCED = 13 || NUMBER OF (NOT AUTOMATIC) DATA ITMES USED = 11 || DATA COVERAGE RATION = 85% |
S O F T E S T
Functional Coverage Measurement
TEST TCOV-14
FunctionCoverage
Function State
Coverage
Coverage
Coverage
Function Chain
Function Outcome
% of Application Functions
Levels of Functional Coverage
TEST TCOV-15
Every application use case is executed at least once.
Every application function is executed at least once.
Every application function outcome is produced at least once.
Every application function sequence is traversed at least once.
Every possible combination of application functions is tested.
1
2
3
4
5
Increasing CostsIncreasing Reliability
F1 = All specified Functions are tested at least once.
F2 = All specified Functions are tested with all valid Arguments at leastone time.
F3 = All specified Functions are tested with all valid Arguments and all expected Results at least once.
Results (n) = Function (Arguments (m))
Result (i) = Function (Argument (j))
i = 1 to n j = 1 to m
Black-Box Test Coverage (Component Test)
TEST TCOV-16
Functional Test Coverage
TEST TCOV-17.1
TRAN:In|OutART_NR:999Amount:999Use Case Coverage = > 1 Test Case
TRANART_NRAmount
One update transaction
Function Coverage = > 2 Test Cases
> one Increase> one Decrease
TRANSART-NRMenge
Function Outcome Coverage
= > 6 Test Cases
> one Increase without Storno> one Increase with Storno of an existing order, generated earlier> one Increase with an error message> one Decrease without an order> one Decrease which generates an order> one Decrease with an error message
TEST TCOV-17.2
TRANSART_NRAmount
Function State Coverage = > 10 Test Cases
> an Increase with invalid Art_Nr.> an increase with invalid amount> an increase to a non existing article> an increase with no storno> an increase with storno of existing order> a decrease with invalid Art_Nr.
> a decrease with invalid amount> a decrease to a non existing article> a decrease without an order> a decrease which generates an order> an invalid transaction
Predicted Error Coverage
TEST TCOV-18
100
200
300
400
500
1 2 3 4 5 6
M o n t h s
N u
m b
e r
o
f
E r
r o
r s
Specification Errors
Design Errors
CodingErrors
UnitTest
Integration Test
System Test
Base Class
Derived Class
Is tested first with all possible variations
Inherits test coverage ofthe base Class.
Statement,Branch,Method & State Coverage
Inheriting Test Coverage
TEST TCOV-19
BaseClass
SubClassX
BaseClass
SubClassY
BaseClass
SubClassZ
Environment X Environment Y Environment Z
The Base Class must be retested in every new target Environment.
According to Perry/Kaiser Study of 1990
Non Transitivity of Test Coverage
TEST TCOV-20
For each Object Type Create = 4 Test Cases e.g. Account Retrieve
UpdateDelete
For each Object State Create = 16 Test Cases e.g. plus Account Retrieve minus Account Update overdrawn Account Delete locked Account
Testing the Creation, Retrieval, Update and Deletion of persistent Objects(relative to one, several or all Object Types and all Object States)
CRUD Object Access Coverage
TEST TCOV-21
Use Cases
A_UseCase
B_UseCase
C_UseCase
D_UseCase
E_UseCase
N_UseCase
C R U D C R U D C R U D
Test Object A_Object B_Object C_Object
According to R. Binder, 1999
CRUD Object Access Coverage Matrix
TEST TCOV-22
Class test Component test System test
Requirements Documentation
User Documentation
Use case Documentation
Application Object Model
System Object Model
Object Interaction Model
Interface Specification
Class Specification
Method Specification
X
X
X X
X X
X
X
X X
X
X
Software Document Software Testing Level
Document Coverage
TEST TCOV-23
Unit = Procedural ModuleUnit-Testing = Module TestWhite-Box Test is in the foregroundData are shared (global)Functional DecompositionFunctional CouplingData and Call CouplingSub Program calls are stubbedEntity/Relationship-DiagramsData Flow DiagramsStructure DiagramsReuse is secondaryDevelopment is TOP-DOWNWaterfall Project Model
Unit = Classe or ObjektUnit-Testing = Class TestGrey-Box Test is in the foregroundData are encapsulatedClass Inheritance and AssociationObject CouplingInheritance and Message CouplingForeign Methods are invokedObject Sequence DiagramsObject State Transition Diagrams Object Collaboration DiagramsReuse is primaryDevelopment is BOTTOM-UPIterative Project Model
Procedural Software Object-orientierted Software
Differences between procedural and object-oriented Software
Differences in Test Coverage
TEST TCOV-24