software engineering unit 6 ch14

3/18/20093/18/2009 S.Sreenivasa RaoS.Sreenivasa Rao 1

Chapter 14Chapter 14Software Testing TechniquesSoftware Testing Techniques

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Testability:Testability: how easily a software can be how easily a software can be testedtested

•• OperabilityOperability—— the better it works, the more efficiently it can be tested. the better it works, the more efficiently it can be tested. -- It It operates cleanlyoperates cleanly

•• ObservabilityObservability——What you see is what you test. The results of each test What you see is what you test. The results of each test case are readily observed (clear and visible variables and statecase are readily observed (clear and visible variables and states, incorrect s, incorrect output, output, ……))

•• ControllabilityControllability——the degree to which testing can be automated and the degree to which testing can be automated and optimizedoptimized

•• DecomposabilityDecomposability——testing can be targeted to independent modulestesting can be targeted to independent modules•• SimplicitySimplicity——reduce complex architecture and logic to simplify testsreduce complex architecture and logic to simplify tests•• StabilityStability——few changes are requested during testingfew changes are requested during testing•• UnderstandabilityUnderstandability——of the designof the design

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


What is a What is a ““GoodGood”” Test?Test?

•• A good test has a high probability of finding an A good test has a high probability of finding an errorerror

•• A good test is not redundant.A good test is not redundant.•• A good test should be A good test should be ““best of breedbest of breed”” (best (best

between those with similar intent) between those with similar intent) •• A good test should be neither too simple nor A good test should be neither too simple nor

too complextoo complex

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Test Case DesignTest Case Design"Bugs lurk in corners "Bugs lurk in corners and congregate at and congregate at boundaries ..."boundaries ..."

Boris Boris BeizerBeizer

OBJECTIVEOBJECTIVE

CRITERIACRITERIA

CONSTRAINTCONSTRAINT

to uncover errorsto uncover errors

in a complete mannerin a complete manner

with a minimum of effort and timewith a minimum of effort and time

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Exhaustive TestingExhaustive Testing

loop < 20 Xloop < 20 X

There are 10 possible paths! If we execute oneThere are 10 possible paths! If we execute onetest per millisecond, it would take 3,170 years totest per millisecond, it would take 3,170 years totest this program!!test this program!!

1414

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Selective TestingSelective Testing

loop < 20 Xloop < 20 X

Selected pathSelected path

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Software TestingSoftware Testing

Methods

Strategies

white-boxmethods

black-boxmethods

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


WhiteWhite--Box TestingBox Testing

... our goal is to ensure that all ... our goal is to ensure that all statements and conditions have statements and conditions have been executed at least once ...been executed at least once ...

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Why Cover?Why Cover?logic errors and incorrect assumptions logic errors and incorrect assumptions are inversely proportional to a path's are inversely proportional to a path's execution probabilityexecution probability

we often we often believe believe that a path is not that a path is not likely to be executed; in fact, reality is likely to be executed; in fact, reality is often counter intuitiveoften counter intuitive

typographical errors are random; it's typographical errors are random; it's likely that untested paths will contain likely that untested paths will contain some some

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net

3/18/20093/18/2009 S.Sreenivasa RaoS.Sreenivasa Rao

Basic Path TestingBasic Path Testing

•• Define a basic set of execution pathsDefine a basic set of execution paths•• Guarantee that every statement is Guarantee that every statement is

executed at least onceexecuted at least once•• Using a logical complexity measure of a Using a logical complexity measure of a

procedural design and use it as a guide for procedural design and use it as a guide for defining a basic set of execution path defining a basic set of execution path

10

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Flow Graph NotationFlow Graph Notation

•• A representation for control flow of A representation for control flow of programprogram

•• Each structured construct Each structured construct --> a flow graph > a flow graph symbolsymbol

11

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Independent program pathIndependent program path

•• Any path through the program that introduces at Any path through the program that introduces at least one new set of processing statements or a least one new set of processing statements or a new condition.new condition.

•• An independent path must move along at least An independent path must move along at least one edge that has not been traversed before the one edge that has not been traversed before the path is defined.path is defined.

•• The basic set is not unique.The basic set is not unique.•• How do we know how many paths to look for?How do we know how many paths to look for?

–– CyclomaticCyclomatic complexitycomplexity12

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Basis Path Testing: Basis Path Testing: CyclomaticCyclomaticComplexityComplexityFirst, we compute the cyclomaticcomplexity:

number of simple decisions + 1 (simple decisions: predicate node)

or

number of enclosed areas + 1or

E-N+2 (E number of edges, N number of nodes)

In this case, V(G) = 4

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


CyclomaticCyclomatic ComplexityComplexityA number of industry studies have indicated A number of industry studies have indicated that the higher V(G), the higher the probability that the higher V(G), the higher the probability or errors.or errors.

V(G)V(G)

modulesmodules

modules in this range are modules in this range are more error pronemore error prone

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Basis Path TestingBasis Path TestingNext, we derive the Next, we derive the independent paths:independent paths:

Since V(G) = 4,Since V(G) = 4,there are four pathsthere are four paths

Path 1: 1,2,3,6,7,8Path 1: 1,2,3,6,7,8Path 2: 1,2,3,5,7,8Path 2: 1,2,3,5,7,8Path 3: 1,2,4,7,8Path 3: 1,2,4,7,8Path 4: 1,2,4,7,2,4,...7,8Path 4: 1,2,4,7,2,4,...7,8

Finally, we derive testFinally, we derive testcases to exercise these cases to exercise these paths.paths.

11

22

3344

55 66

77

88

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Basis Path Testing NotesBasis Path Testing Notes

you don't need a flow chart, you don't need a flow chart, but the picture will help when but the picture will help when you trace program pathsyou trace program paths

count each simple logical test, count each simple logical test, compound tests count as 2 or compound tests count as 2 or moremore

basis path testing should be basis path testing should be applied to critical modulesapplied to critical modules

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Deriving test casesDeriving test cases

•• Using the design or code, draw a flow Using the design or code, draw a flow diagramdiagram

•• Determine the Determine the cyclomaticcyclomatic complexity of complexity of the flow graphthe flow graph

•• Determine a basis set of independent Determine a basis set of independent pathspaths

•• Prepare test cases to force the execution Prepare test cases to force the execution of each pathof each path 17

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Graph MatricesGraph Matrices•• A tabular representation of a flow graphA tabular representation of a flow graph•• A graph matrix is a square matrix whose size A graph matrix is a square matrix whose size

(i.e., number of rows and columns) is equal to (i.e., number of rows and columns) is equal to the number of nodes on a flow graphthe number of nodes on a flow graph–– Each row and column corresponds to an identified Each row and column corresponds to an identified

node, and node, and –– matrix entries correspond to connections (an edge) matrix entries correspond to connections (an edge)

between nodes. between nodes. •• By adding a By adding a link weightlink weight to each matrix entry, to each matrix entry,

the graph matrix can become a powerful tool the graph matrix can become a powerful tool for evaluating program control structure during for evaluating program control structure during testingtesting

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


•• Link weights can beLink weights can be–– The probability of execution of that edgeThe probability of execution of that edge–– The processing time for execution that edgeThe processing time for execution that edge–– The memory requiredThe memory required–– The resource requiredThe resource required

•• Help to design test casesHelp to design test cases19

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Other Control Structure TestingOther Control Structure Testing

•• Condition testingCondition testing —— a test case design method a test case design method that exercises the logical conditions contained that exercises the logical conditions contained in a program modulein a program module

•• Data flow testingData flow testing —— selects test paths of a selects test paths of a program according to the locations of program according to the locations of definitions (D) and uses (U) of variables in the definitions (D) and uses (U) of variables in the program (every DU chain be covered at least program (every DU chain be covered at least once)once)

•• Loop testing Loop testing –– focus on validity of loop focus on validity of loop constructsconstructs

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Loop TestingLoop Testingfour different classes of loopfour different classes of loop

Nested Nested LoopsLoops

ConcatenatedConcatenatedLoops Loops Unstructured Unstructured

LoopsLoops

Simple Simple looploop

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Loop Testing: Simple Loop Testing: Simple LoopsLoopsMinimum conditionsMinimum conditions——Simple LoopsSimple Loops

1. skip the loop entirely1. skip the loop entirely2. only one pass through the loop2. only one pass through the loop3. two passes through the loop3. two passes through the loop4. m passes through the loop m < n4. m passes through the loop m < n5. (n5. (n--1), n, and (n+1) passes through 1), n, and (n+1) passes through the loopthe loop

where n is the maximum number where n is the maximum number of allowable passesof allowable passes

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Loop Testing: Nested LoopsLoop Testing: Nested Loops

Start at the innermost loop. Set all outer loops to their Start at the innermost loop. Set all outer loops to their minimum iteration parameter values.minimum iteration parameter values.

Test the min+1, typical, maxTest the min+1, typical, max--1 and max for the 1 and max for the innermost loop, while holding the outer loops at their innermost loop, while holding the outer loops at their minimum values.minimum values.

Move out one loop and set it up as in step 2, holding all Move out one loop and set it up as in step 2, holding all other loops at typical values. Continue this step until other loops at typical values. Continue this step until the outermost loop has been tested.the outermost loop has been tested.

NestedNested LoopsLoops

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Loop Testing: Concatenated Loop Testing: Concatenated LoopsLoops

If the loops are independent of one another If the loops are independent of one another then treat each as a simple loopthen treat each as a simple loopelse* treat as nested loopselse* treat as nested loops

endifendif* *

for example, the final loop counter value of loop 1 is for example, the final loop counter value of loop 1 is used to initialize loop 2.used to initialize loop 2.

Concatenated LoopsConcatenated Loops

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


BlackBlack--Box TestingBox Testing

requirementsrequirements

eventseventsinputinput

outputoutput

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


•• Incorrect or missing functionsIncorrect or missing functions•• Interface errorsInterface errors•• Errors in Data structures or external data Errors in Data structures or external data

base accessbase access•• Behavior or performance errorsBehavior or performance errors•• initialization and termination errorsinitialization and termination errors

26

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


BlackBlack--Box TestingBox Testing

•• How is functional validity tested?How is functional validity tested?•• How is system behavior and performance tested?How is system behavior and performance tested?•• What classes of input will make good test cases?What classes of input will make good test cases?•• Is the system particularly sensitive to certain input Is the system particularly sensitive to certain input

values?values?•• How are the boundaries of a data class isolated?How are the boundaries of a data class isolated?•• What data rates and data volume can the system What data rates and data volume can the system

tolerate?tolerate?•• What effect will specific combinations of data have on What effect will specific combinations of data have on

system operation?system operation?

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


GraphGraph--Based MethodsBased Methods

newfile

menu select generates

(generation time < 1.0 sec)document

window

documenttex

t

is represented as

contains

Attributes:

background color: whitetext color: default color

or preferences

(b)

object#1

Directed link

(link weight)object

#2

object#3

Undirected link

Parallel links

Node weight(value

)

(a)

allows editingof

To understand the To understand the objects that are objects that are modeled in modeled in software and the software and the relationships that relationships that connect these connect these objectsobjects

In this context, we In this context, we consider the term consider the term ““objectsobjects”” in the broadest in the broadest possible context. It possible context. It encompasses data encompasses data objects, traditional objects, traditional components (modules), components (modules), and objectand object--oriented oriented elements of computer elements of computer software.software.

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Graph based testingGraph based testing

•• Transaction flow modelingTransaction flow modeling•• Finite state modelingFinite state modeling•• Data flow modelingData flow modeling•• Timing modeling (sequence of execution Timing modeling (sequence of execution

and its time)and its time)

29

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Equivalence PartitioningEquivalence Partitioning

•• Divides the input domain of a program Divides the input domain of a program into classes of data from which test cases into classes of data from which test cases can be derived.can be derived.

•• Define test cases that uncovers classes of Define test cases that uncovers classes of errorserrors

30

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


•• If an Input condition isIf an Input condition is–– A rangeA range

•• One valid and two invalid equivalence classesOne valid and two invalid equivalence classes

–– A specific valueA specific value•• One valid and two invalid equivalence classesOne valid and two invalid equivalence classes

–– A member of a setA member of a set•• One valid and one invalid equivalence classesOne valid and one invalid equivalence classes

–– BooleanBoolean•• One valid and one invalid equivalence classesOne valid and one invalid equivalence classes 31

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Boundary value checkingBoundary value checking

•• A greater number of errors occurs at the A greater number of errors occurs at the boundaries of the input domainboundaries of the input domain

32

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Comparison Testing (NComparison Testing (N--versioning)versioning)

•• Used only in situations in which the reliability of Used only in situations in which the reliability of software is absolutely critical (e.g., humansoftware is absolutely critical (e.g., human--rated systems)rated systems)–– Separate software engineering teams develop Separate software engineering teams develop

independent versions of an application using the independent versions of an application using the same specificationsame specification

–– Each version can be tested with the same test data Each version can be tested with the same test data to ensure that all provide identical output to ensure that all provide identical output

–– Then all versions are executed in parallel with realThen all versions are executed in parallel with real--time comparison of results to ensure consistencytime comparison of results to ensure consistency

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Orthogonal Array TestingOrthogonal Array Testing•• Used when the number of input parameters is Used when the number of input parameters is

small and the values that each of the small and the values that each of the parameters may take are clearly boundedparameters may take are clearly bounded

•• Dispersed uniformly throughout the test domain Dispersed uniformly throughout the test domain (balancing property)(balancing property)

One input item at a time L9 orthogonal array

XY

Z

XY

Z

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


OOTOOT——Test Case DesignTest Case DesignBerardBerard [BER93] proposes the following approach:[BER93] proposes the following approach:

11.. Each test case should be uniquely identified and should be expliEach test case should be uniquely identified and should be explicitly citly associated with the class to be tested,associated with the class to be tested,

2.2. The purpose of the test should be stated,The purpose of the test should be stated,

3.3. A list of testing steps should be developed for each test and shA list of testing steps should be developed for each test and should ould contain [BER94]:contain [BER94]:

a.a. a list of specified states for the object that is to be testeda list of specified states for the object that is to be tested

b.b. a list of messages and operations that will be exercised as a list of messages and operations that will be exercised as a consequence of the testa consequence of the test

c.c. a list of exceptions that may occur as the object is testeda list of exceptions that may occur as the object is tested

d.d. a list of external conditions (i.e., changes in the environment a list of external conditions (i.e., changes in the environment external to the software that must exist in order to properly coexternal to the software that must exist in order to properly conduct nduct the test)the test)

e.e. supplementary information that will aid in understanding or supplementary information that will aid in understanding or implementing the test.implementing the test.

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


Testing MethodsTesting Methods•• FaultFault--based testing (same for conventional)based testing (same for conventional)

–– The tester looks for plausible faults The tester looks for plausible faults •• aspects of the implementation of the system that may result aspects of the implementation of the system that may result

in defects. in defects. –– To determine whether these faults exist, test cases are designedTo determine whether these faults exist, test cases are designed

to exercise the design or code. to exercise the design or code.

•• Class Testing and the Class HierarchyClass Testing and the Class Hierarchy–– Inheritance does not obviate the need for thorough testing of alInheritance does not obviate the need for thorough testing of all l

derived classes. derived classes. –– In fact, it can actually complicate the testing process.In fact, it can actually complicate the testing process.

•• ScenarioScenario--Based Test DesignBased Test Design–– ScenarioScenario--based testing concentrates on what the user does, not based testing concentrates on what the user does, not

what the product does. what the product does. –– This means capturing the tasks (via useThis means capturing the tasks (via use--cases) that the user has cases) that the user has

to perform, then applying them and their variants as tests.to perform, then applying them and their variants as tests.

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


OOT Methods: Random Testing class basedOOT Methods: Random Testing class based

•• Random testing: Random testing: check class instance life historiescheck class instance life histories

–– identify operations applicable to a classidentify operations applicable to a class–– define constraints on their usedefine constraints on their use–– identify a minimum test sequenceidentify a minimum test sequence

•• an operation sequence that defines the minimum life history of tan operation sequence that defines the minimum life history of the class (object)he class (object)

–– generate a variety of random (but valid) test generate a variety of random (but valid) test sequencessequences

•• exercise other (more complex) class instance life historiesexercise other (more complex) class instance life histories

Like for an instance of Account class: open, setup, depositLike for an instance of Account class: open, setup, deposit, , withdraw, close And open, setup, deposit, [deposit, withdraw, withdraw, close And open, setup, deposit, [deposit, withdraw, balance, balance, creditlimitcreditlimit], withdraw, close], withdraw, close

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


OOT Methods: Partition OOT Methods: Partition TestingclassTestingclass basedbased

•• Partition TestingPartition Testing–– reduces the number of test cases required to test a class in reduces the number of test cases required to test a class in

much the same way as equivalence partitioning for much the same way as equivalence partitioning for conventional softwareconventional software

–– statestate--based partitioningbased partitioning•• categorize and test operations based on their ability to change categorize and test operations based on their ability to change the state of a classthe state of a class

–– attributeattribute--based partitioningbased partitioning•• categorize and test operations based on the attributes that theycategorize and test operations based on the attributes that they useuse•• For example: partition the operations according to their use/modFor example: partition the operations according to their use/modify/donify/don’’t use or t use or

modify an attributemodify an attribute

–– categorycategory--based partitioningbased partitioning•• categorize and test operations based on the generic function eaccategorize and test operations based on the generic function each performsh performs•• For example: operations can be categorized as For example: operations can be categorized as

initialization/computation/queries/terminationinitialization/computation/queries/termination

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


OOT Methods: InterOOT Methods: Inter--Class TestingClass Testing(During integration)(During integration)

•• RandomRandom–– Like in class testingLike in class testing

•• PartitioningPartitioning–– In (intraIn (intra--)class testing: state)class testing: state--based, attributebased, attribute--based, based,

categorycategory--basedbasedHere:Here:–– Partition according to messages which serve different Partition according to messages which serve different

classesclasses–– StateState--basedbased

•• BehavioralBehavioral–– StateState--basedbased

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


OOT Methods: InterOOT Methods: Inter--Class Testing Class Testing (During integration)(During integration)

•• RandomRandom–– For each client class, use the list of class operators to For each client class, use the list of class operators to

generate a series of random test sequences. The operators generate a series of random test sequences. The operators will send messages to other server classes.will send messages to other server classes.

–– For each message that is generated, determine the For each message that is generated, determine the collaborator class and the corresponding operator in the collaborator class and the corresponding operator in the server object.server object.

–– For each operator in the server object (that has been For each operator in the server object (that has been invoked by messages sent from the client object), determine invoked by messages sent from the client object), determine the messages that it transmits.the messages that it transmits.

–– For each of the messages, determine the next level of For each of the messages, determine the next level of operators that are invoked and incorporate these into the operators that are invoked and incorporate these into the test sequencetest sequence

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net


OOT Methods: Behavior TestingOOT Methods: Behavior Testingemptyacctopen setup Accnt

set upacct

deposit(initial)

workingacct

withdrawal(final)

deadacct close

nonworkingacct

deposit

withdrawbalance

creditaccntInfo

Figure 14.3 St at e diagram f or Account class (adapt ed f rom [ KIR94] )

The tests to be The tests to be designed should designed should achieve all state achieve all state coverage [KIR94]. coverage [KIR94]. That is, the That is, the operation operation sequences should sequences should cause the cause the Account class to Account class to make transition make transition through all through all allowable statesallowable states

ww

w.jntuw

orld.com

ww

w.jntuw

orld.com

ww

w.jw

jobs.net

software engineering unit 6 ch14

Education