CoWTeSt: A Cost

Weighted Test Strategy

Test Planning StrategyObjectives:Objectives:

Decide which and how many test cases Decide which and how many test cases should be executed.should be executed.

Give a practical help to managers to Give a practical help to managers to support test planning and evaluate the support test planning and evaluate the impact of testing the phase on the cost of impact of testing the phase on the cost of the final product.the final product.

Features (Exclusively) UML based: It uses the Use Cases (Exclusively) UML based: It uses the Use Cases

and Interaction diagrams developed during and Interaction diagrams developed during analysis and design phasesanalysis and design phases

Cowtest strategyCowtest strategy: : development of a systematic development of a systematic method for UML design-based integration test of method for UML design-based integration test of complex system.complex system.

Based exclusively on the use of UML diagrams: Based exclusively on the use of UML diagrams:

• immediately usable from industries already using UML immediately usable from industries already using UML

(no additional modeling or design costs); (no additional modeling or design costs); • analysis can be started soon, and done contemporarily analysis can be started soon, and done contemporarily

with project development; with project development;

DiDifferent strategies for test cases fferent strategies for test cases sselectionelection

Presentation Scheme

Test caseTest case

Test case

Test case

Test caseTest case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Obj 1 Obj 2 Obj 3

Method1()

Method3()

Method2()

UUII

TT

CoWTeStCoWTeSt

CoWTeSt: A Cost Weighted Test Strategy

Weighted Tree Derivation:Weighted Tree Derivation:1.1. Starting from the main UCD the UCs and Starting from the main UCD the UCs and

SDs are organized in a sort of hierarchical SDs are organized in a sort of hierarchical treetree

2.2. Deduce the Critical Profile: annotate Deduce the Critical Profile: annotate every arch with a value representing every arch with a value representing ““the the importanceimportance”” of the associate node (be i of the associate node (be itt a a UC or a SD scenarioUC or a SD scenario))

3.3. Test Case Derivation: applTest Case Derivation: apply y UIT for test UIT for test derivationderivation

SiteKeeper_SD

CallManagement_SD

ConnDisc_SD...

ConnDisc_SD...

NAM NRM

DiscofConn Conn

SK

REM C

CM

TcSOcS

S

OrigCall_SD...

OrigCa...

Connect_SD

Setup_SD

0.050.05 0.6

0.3

0.1 0.3 0.30.1

0.2

0.4 0.40.2 0.45

0.450.1

SiteKeeper_SD

CallManagement_SD

ConnDisc_SD...

ConnDisc_SD...

NAM NRM

DiscofConn Conn

SK

REM C

CM

TcSOcS

S

OrigCall_SD...

OrigCa...

Connect_SD

Setup_SD

0.050.05 0.6

0.3

0.1 0.3 0.30.1

0.2

0.4 0.40.2 0.45

0.450.1

Integration StageThe The first integration stagefirst integration stage is represented by the main is represented by the main

UC and the SDs (if any), which are children of this UC and the SDs (if any), which are children of this node (hence they are at level 2 of the tree).node (hence they are at level 2 of the tree).

TheThe i-th integration stagei-th integration stage is represented by the UCs is represented by the UCs positioned in at positioned in at i-thi-th level of the tree and every SDs, level of the tree and every SDs, children of these nodes, situated at children of these nodes, situated at i+1i+1-th level.-th level.

NOTE: In the NOTE: In the i-thi-th integration stage the SDs at level integration stage the SDs at level i+1 i+1 are are consideredconsidered,, because they represent the interaction between because they represent the interaction between the different components that realize the functionalities the different components that realize the functionalities described in the UCs at described in the UCs at i-th i-th level of the tree.level of the tree.

SiteKeeper_SD

CallManagement_SD

ConnDisc_SD...

ConnDisc_SD...

NAM NRM

DiscofConn Conn

SK

REM C

CM

TcSOcS

S

OrigCall_SD...

OrigCa...

Connect_SD

Setup_SD

0.050.05 0.6

0.3

0.1 0.3 0.30.1

0.2

0.4 0.40.2 0.45

0.450.1

3rd Integration Stage

Test Case SelectionFor every node, the product of all the nodes weights For every node, the product of all the nodes weights

on the complete path from the root to this node on the complete path from the root to this node represents its represents its final weight..

Int-Stage Leaves names Critical profile 2nd Stage/NTest 3rd Stage/NTest 4th stage/NTest

1st Stage SK 1            SK_SD 0.3 0.3 150 0.3 150 0.3 150SK_NDchild 0.7 - - - - - -

2nd Stage NAM 0.05 0.05 25 0.05 25 0.05 25NRM 0.05 0.05 25 0.05 25 0.05 25CM 0.6 - - - - - -CM_SD 0.2 0.12 60 0.12 60 0.12 60CM_NDchild 0.8 48 240 - - - -

3rd Stage EM 0.1     0.06 30 0.06 30S 0.3     - - - -S_SD 0.2     0.036 18 0.036 18S_NDchild 0.8     0.144 72 - -C 0.3     - - - -C_SD 0.10     0.018 9 0.018 9C_NDchild 0.90     0.162 81 - -R 0.1     0.06 30 0.06 30

4th Stage OcS 0.4         0.072 36TcS 0.4         0.072 36DiscofConn 0.45         0.081 41Conn 0.45         0.081 41

Test Case Selection

Leaves names Critical profile 2nd Stage/NTest 3rd Stage/NTestSK 1        SK_SD 0.3 0.3 150 0.3 150SK_NDchild 0.7 - - - -NAM 0.05 0.05 25 0.05 25NRM 0.05 0.05 25 0.05 25CM 0.6 - - - -CM_SD 0.2 0.12 60 0.12 60CM_NDchild 0.8 48 240 - -EM 0.1     0.06 30S 0.3     - -S_SD 0.2     0.036 18S_NDchild 0.8     0.144 72C 0.3     - -C_SD 0.10     0.018 9C_NDchild 0.90     0.162 81R 0.1     0.06 30OcS 0.4        TcS 0.4        

SiteKeeper_SD

CallManagement_SD

ConnDisc_SD...

ConnDisc_SD...

NAM NRM

DiscofConn Conn

SK

REM C

CM

TcSOcS

S

OrigCall_SD...

OrigCa...

Connect_SD

Setup_SD

0.050.05 0.6

0.3

0.1 0.3 0.30.1

0.2

0.4 0.40.2 0.45

0.450.1

80% Functional Coverage

Leaves names 4th Stage weights

70%coveragenwf /MinNtest

80%coverage/nwf /MinNtest

90%coverage/nwf /MinNtest

100%coverage/nwf /MinNtest

SK_SD 0.3 0.413 5 0.354 5 0.137 6 0.3 17

CM_SD 0.12 0.165 2 0.141 2 0.126 3 0.12 7

DiscofConn 0.081 0.111 2 0.95 2 0.085 2 0.081 5

Conn 0.081 0.111 2 0.095 2 0.085 2 0.081 5

OcS 0.072 0.1 1 0.085 2 0.076 2 0.072 4

TcS 0.072 0.1 1 0.085 2 0.076 2 0.072 4

EM 0.06     0.071 1 0.063 2 0.06 4

R 0.06     0.071 1 0.063 2 0.06 4

NAM 0.05         0.052 1 0.05 3

NRM 0.05         0.052 1 0.05 3

S_SD 0.036             0.036 2

C_SD 0.018             0.018 1

Total   72.3%

13 84.6%

17 94.6%

23 100% 59

Selection of leaves for different coverage levels

Leaves names 4th Stage weights

70%coveragenwf /MinNtest

80%coverage/nwf /MinNtest

SK_SD 0.3 0.413 5 0.354 5

CM_SD 0.12 0.165 2 0.141 2

DiscofConn 0.081 0.111 2 0.95 2

Conn 0.081 0.111 2 0.095 2

OcS 0.072 0.1 1 0.085 2

TcS 0.072 0.1 1 0.085 2

EM 0.06     0.071 1

R 0.06     0.071 1

NAM 0.05        

NRM 0.05        

S_SD 0.036        

C_SD 0.018        

Total   72.3% 13 84.6% 17

Selection of leaves for different coverage levels

Presentation Scheme

Test caseTest case

Test case

Test case

Test caseTest case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Obj 1 Obj 2 Obj 3

Method1()

Method3()

Method2()

UUII

TT

CoWTeStCoWTeSt

A method to systematicallyA method to systematically derive derive Integration Test suites from UML diagrams:Integration Test suites from UML diagrams:• Use case diagrams• Sequence (Interaction) diagrams• Class diagrams

Incremental test strategy (over the Use Case Incremental test strategy (over the Use Case organizational structure)organizational structure)

Seven steps involvedSeven steps involved

Use Interaction Test methodology

Step 1: UML design analysis and search of relevant Use Cases

•Use Case diagram of uci.argo.KERNEL

Step 2/1: Analysis of Sequence and Class diagrams involved in the selected Use Case

•Sequence diagram of uci.argo.Kernel.USERMODEL

Step 2/2: Class diagram analysis

Class diagram of uci.argo.Kernel.USERMODEL

Designer

Decision Model

_decisions

DecisionModel()GetDecisions()

GoalModel

_goals

GoalModel()GetGoals()

Decision

Decision(name:String;priority:int)GetName()GetPriority()

Goal

Goal(name:String;priority:int)GetName()GetPriority()

1:1..* uses1:1..* uses

*

contains

*

contains

Step 3 Test Units definition

Test UnitsTest Units:: those system those system units separately units separately testable for exercising a possible use of testable for exercising a possible use of system:system:

Test UnitsTest Units: : DecisionModel, Decision, GoalModel, DecisionModel, Decision, GoalModel,

Goal.Goal.

Step 4: Research of Settings and Interactions Categories- - SettingsCategoriesSettingsCategories:: parameters and inputs relevant parameters and inputs relevant in the Test Unit.in the Test Unit.

- - InteractionsCategoriesInteractionsCategories:: objects interactions, objects interactions, exchanged messagesexchanged messages..

Categories:Categories:

DecisionModelDecisionModelSettings: Settings: __decisionsdecisionsInteractions:Interactions: DecisionModel() DecisionModel() getDecisions()getDecisions()DecisionDecisionInteractions: Interactions:

Decision(name:String,priority:int)Decision(name:String,priority:int)GetName()GetName()GetPriority()GetPriority() ........................

Step 5: Test Specification construction

For each identified category, we consider all For each identified category, we consider all its possible values and constraints.its possible values and constraints.

Test Specification:DecisionModel:

Settings: Interactions: _decisions getDecisions()

Naming Opening a new fileStorage Opening a saved fileModularity After a modification and before saving

Inheritance After a modification and after saving

Stereotypes DecisionModel()Relationship.. Class constructor

Step 6: Search of Message Sequences and Test Case definition Message Sequences: set of Messages (in temporal

order) , involved in a SD, used by objects to define and elaborate specific functionalities.

Step 6: Search of Message Sequences and Test Case definition Test Case: is constructed, from a Test

Specification, for each possible combination of choices of every category involved in a Message Sequence

Test Case getDecisions()getName()/getPriority()

Opening a saved file_decisions

NamingNote: Such a Test Case must be repeated for all possible values of _decisions

Step 7: Definition of UseCase Test Suite and Incremental Construction of TestFrame

UseCase Test Suite Design Critics

Use Case Test Suite Kernel

UseCase Test Suite UserModel

UseCase Test Suite ToDoList

Test Frame uci.argo.Kernel

Cow_Suite: Cow pluS UIT Environment

Tool for automatingTool for automatingTest cases derivation, from UITTest cases derivation, from UITTest management strategy, from Test management strategy, from

COWTeSTCOWTeST Uses UML diagrams already developed for Uses UML diagrams already developed for

the analysis and design phase the analysis and design phase no no additional effort or information.additional effort or information.

Rational Rose environment Rational Rose environment

Cow_Suite Architecture Uses REI, Uses REI, Rational Rose Extensibily Interface.Rational Rose Extensibily Interface. Composed by six different components:Composed by six different components:

1.1. MDL AnalyserMDL Analyser2.2. Test Tree GeneratorTest Tree Generator3.3. Weights ManagerWeights Manager4.4. Test Case GeneratorTest Case Generator5.5. Test Case BuilderTest Case Builder6.6. Test Case CheckerTest Case Checker

Graphics interface: organized in three Graphics interface: organized in three tabs:tabs: CowTreeCowTree UIT StructureUIT Structure Test SpecificationTest Specification

1.MDL Analyser Analysis of the .mdl file of RoseAnalysis of the .mdl file of Rose Search the information necessary to apply Search the information necessary to apply

test selection strategy:test selection strategy: Actors and Use Cases;Actors and Use Cases; Sequence diagrams and their messages;Sequence diagrams and their messages; Classes with attributes and methods;Classes with attributes and methods; Activity diagramsActivity diagrams

The information is passed as the input to the The information is passed as the input to the next component.next component.

2.Test Tree Generator All the UCDs and SDs from .mdl file are All the UCDs and SDs from .mdl file are

organized in a organized in a hierarchical treehierarchical treeusing the explicit diagrams links or using the UC using the explicit diagrams links or using the UC associations and classes relationsassociations and classes relations

To To each node each node we associate:we associate: Level identification number;Level identification number; Default weight Default weight

such that the sum of this weight plus the such that the sum of this weight plus the weightsweights

associated to all its brothers is equal to 1. associated to all its brothers is equal to 1.

3.Weights Manager Interacts with the user for:Interacts with the user for:

a)a) Assigning the real weight to each node:Assigning the real weight to each node:– Selecting directly a nodeSelecting directly a node– Selecting a level numberSelecting a level number Check Check that the sum of the user assigned weights of a that the sum of the user assigned weights of a

level nodes is equal to 1.level nodes is equal to 1.

b)b) Choosing the criterion for test case selection:Choosing the criterion for test case selection:– Fixing the maximum number of test cases to be Fixing the maximum number of test cases to be

executed executed – Fixing the percentage of functionalities to be covered.Fixing the percentage of functionalities to be covered.

4.Test Case Generator queries the user for the deepest integration level queries the user for the deepest integration level

he/she is interested in and calculate the he/she is interested in and calculate the final final weightweight of every leaf. of every leaf.

implements the implements the UITUIT method for test generation method for test generation associates to each SD its test cases organized in a associates to each SD its test cases organized in a

hierarchicalhierarchical manner. manner. calculates and visualizes the calculates and visualizes the number of test cases number of test cases

for each SDfor each SD depending on the chosen selection depending on the chosen selection criterion.criterion.

associates to each SD the associates to each SD the framesframes of the test cases of the test cases that should be instantiated. that should be instantiated.

Test Case FrameTEST CASE 8.2TEST CASE 8.2Description:Description:PreCondition:PreCondition:

Test Case 7Test Case 7 Flow of event:Flow of event:getEnterpriseForSourceAddress(caller)getEnterpriseForSourceAddress(caller)

[else][else]routingResult =doLRQ(caller, callee, Enterprise, routingResult =doLRQ(caller, callee, Enterprise, callcase,AccessAgent, BGAResult)callcase,AccessAgent, BGAResult)

CategoriesCategories::SettingsCategories:SettingsCategories:PEnterprise =PEnterprise =Callee =Callee =BGAResult ……BGAResult ……

InteractionsCategories:InteractionsCategories:getEnterpriseForSourceAddress =getEnterpriseForSourceAddress =doLRQ =doLRQ =PostCondition:PostCondition:Comment:Comment:

5.Test Case Builder Interaction with the user for test case Interaction with the user for test case

implementation:implementation:For the necessary parameters;For the necessary parameters;Adding and removing categories;Adding and removing categories;Changing operations value or even Changing operations value or even

test cases structure.test cases structure. The changes involving the UML design are The changes involving the UML design are

finally saved in a new file.finally saved in a new file.

6. Test Case Checker The tool maintains information about the test The tool maintains information about the test

cases generation. cases generation. This module will realize the This module will realize the comparison of comparison of

different versionsdifferent versions of the same .mdl file. of the same .mdl file.

The discovered differences, like, for example, the The discovered differences, like, for example, the existence of new test cases or changes in those existence of new test cases or changes in those

already already generated, are saved into a separate file.generated, are saved into a separate file.

The evaluation of the cost and impact required for The evaluation of the cost and impact required for the updates to the test plan with respect to the the updates to the test plan with respect to the “official version” is derived analyzing this file. “official version” is derived analyzing this file.

References:

F. Basanieri, A. Bertolino, “A Practical Approach F. Basanieri, A. Bertolino, “A Practical Approach to UML-based Derivation of Integration Tests”, to UML-based Derivation of Integration Tests”, QWE2000, Bruxelles, 20-24 November, 3TQWE2000, Bruxelles, 20-24 November, 3T..Basanieri, F., Bertolino, A., Marchetti, E., Basanieri, F., Bertolino, A., Marchetti, E., “C“CoWTeSt: A Cost Weighed Test StrategyoWTeSt: A Cost Weighed Test Strategy”, ”, accepted for: accepted for: ESCOM-SCOPE 2001, London, ESCOM-SCOPE 2001, London, EnglandEngland,, 2-4 April 2001.2-4 April 2001. Basanieri, F., Bertolino, A., Marchetti, E.Basanieri, F., Bertolino, A., Marchetti, E.,, RiboliniRibolini,, A., “An Automated Test Strategy Based A., “An Automated Test Strategy Based on UML Diagrams” submitted toon UML Diagrams” submitted to:: ICSE 2001 ICSE 2001 WAPATV WAPATV woworkshop.rkshop.