A Comparison of Software Testing Using the A Comparison of Software Testing Using the Object-Oriented Paradigm and Traditional TestingObject-Oriented Paradigm and Traditional Testing

by by

Jamie GordonJamie GordonBob RoggioBob Roggio

University of North FloridaUniversity of North Florida

Jacksonville, FloridaJacksonville, Florida

In summary, the literature for hybrid methodologies and more specifically for using RUP and SCRUM lacks of empirical evidence to define their nature or elements and how both processes should be carried out.

1

Introductory Introductory CommentsComments

•Software testing: Critical part of any software development.Software testing: Critical part of any software development.

•OOT gives rise to critical complex testing issues arise.OOT gives rise to critical complex testing issues arise.

•Current researchCurrent researchPartial comparisons between OOT and traditionalPartial comparisons between OOT and traditional

Within OOT, some address inheritance and polymorphismWithin OOT, some address inheritance and polymorphism

•So much moreSo much more

2

Objectives of TestingObjectives of Testing

Test to find faults: minimum of cost and effortTest to find faults: minimum of cost and effort

Effective test case planning Effective test case planning developing test developing test cases leading to discovery errors.cases leading to discovery errors.

This paper presents This paper presents

Complexity issues in OO-paradigm and Complexity issues in OO-paradigm and

Issues that may impact test case planning.Issues that may impact test case planning.

3

Test Cases for Traditional TestingTest Cases for Traditional Testing

Traditional Testing: usually based on Input Process Traditional Testing: usually based on Input Process Output Output

Works well for the Von-Neuman model of processing: Works well for the Von-Neuman model of processing: inputs determine outputs . inputs determine outputs .

Can view testing as an 'ordered pair' <input, expected Can view testing as an 'ordered pair' <input, expected output> output> No concerns are expressed for No concerns are expressed for ''programprogram statestate.'.'

Expected outputs expressed in terms of values Expected outputs expressed in terms of values produced by inputs. produced by inputs.

Importantly, this implies test cases can be determined Importantly, this implies test cases can be determined staticallystatically for dynamic testing. for dynamic testing.

4

Test Cases for Object-Oriented Test Cases for Object-Oriented TestingTesting

Very different in OO testing. Very different in OO testing.

Objects in program have ‘Objects in program have ‘statestate’ impacted by input ’ impacted by input parameters and computations parameters and computations

‘‘State ChangesState Changes’ may not at all be evident by the outputs of ’ may not at all be evident by the outputs of the program. the program.

OO provides functionality not provided in traditional: such as OO provides functionality not provided in traditional: such as encapsulation, reusability, and more.encapsulation, reusability, and more.

5

Testing LevelsTesting Levels

•Ease of writing OO programs does not translate into testing! Ease of writing OO programs does not translate into testing!

•Researchers claim testing in OO Researchers claim testing in OO increases effortincreases effort for for adequate testingadequate testing

•Traditional testingTraditional testing: Essentially unit and system testing. : Essentially unit and system testing.

•OOTOOT adds class and cluster testing. adds class and cluster testing.

6

Unit TestingUnit Testing

• OOT & Traditional Testing: methods/routines tested OOT & Traditional Testing: methods/routines tested independentlyindependently

• IIndependent unitsndependent units Don't call other units/methods that use global data Don't call other units/methods that use global data

Major IssuesMajor Issues in testing methods in OOT: in testing methods in OOT: Methods Methods cannot interact with other classes, cannot interact with other classes, Method Method attributes may be attributes may be changedchanged by methods, by methods, Method may be Method may be dependentdependent on its other class's on its other class's methods. methods. Other methods called by this method Other methods called by this method in the class in the class must be deemed correct.must be deemed correct. T Thus hus testing individual methods is much more difficult.testing individual methods is much more difficult.

(Testing (Testing cannot occur in a linear order cannot occur in a linear order and must be determined by a different method and must be determined by a different method such as flow graphs or graph based techniques.)such as flow graphs or graph based techniques.)

7

Unit TestingUnit TestingA case can be made for usingA case can be made for using drivers and stubs drivers and stubs and thus simulating dependent and thus simulating dependent classes. classes.

A driver is written when a A driver is written when a method is dependent upon another for data method is dependent upon another for data

A stub is a method that is A stub is a method that is handed data to process handed data to process when the module that when the module that processes the data has not yet been written or when the module has processes the data has not yet been written or when the module has

not not been adequately tested. been adequately tested.

Drivers and Stubs commonly found in testing classes in a Drivers and Stubs commonly found in testing classes in a hierarchical designhierarchical design. .

In OO testing drivers and stubsIn OO testing drivers and stubs must be extended to entire class; complicates must be extended to entire class; complicates testing.testing.

Unit testing in OO – not an easy comparison; not terribly useful either!Unit testing in OO – not an easy comparison; not terribly useful either!

8

Class TestingClass Testing

OO’s version of unit testing.OO’s version of unit testing.

Class testingClass testing: tests methods as they : tests methods as they interactinteract with other methods. with other methods.

• For any non-trivial task, For any non-trivial task, testing methods in isolationtesting methods in isolation not terribly not terribly useful,useful, Methods are meant to interact. Methods are meant to interact.

But But other called methodsother called methods in other classes must need be in other classes must need be

thoroughly thoroughly tested tested beforehand beforehand oror The The dependent dependent classes need to be simulated classes need to be simulated

9

Cluster Testing Cluster Testing Cluster Testing: No clear correspondence to traditional testingCluster Testing: No clear correspondence to traditional testing

Cluster Testing: an Cluster Testing: an extensionextension of class testing of class testing Purpose: ensure group/cluster of Purpose: ensure group/cluster of classes classes interactinteract correctly. correctly.

"Cluster of classes" is a group of classes "Cluster of classes" is a group of classes Are Are dependentdependent and and cooperatecooperate with each with each other directly. other directly.

Cooperating classes Cooperating classes must have been must have been thoroughly tested thoroughly tested ahead ahead preferably through class and unit testing. (more preferably through class and unit testing. (more ahead) ahead)

10

Integration and System TestingIntegration and System Testing

Traditional testing involves Traditional testing involves testing units together, as a testing units together, as a wholewhole Includes dependent Includes dependent methods and/or dependency on global data. methods and/or dependency on global data.

OO testing: OO testing: extensionextension of cluster testing. of cluster testing.

Clusters are combined ==> the whole system. Clusters are combined ==> the whole system. Tested with all the dependencies intact. Tested with all the dependencies intact.

11

Object Oriented Features that Affect Object Oriented Features that Affect TestingTesting

This paper will address seven major OO features that significantly This paper will address seven major OO features that significantly complicate effective OO testing:complicate effective OO testing:

1. Encapsulation1. Encapsulation

2. Inheritance2. Inheritance

3. Polymorphism3. Polymorphism

4. Cohesion (assists in effective OO Testing)4. Cohesion (assists in effective OO Testing)

5. Coupling5. Coupling

6. Dynamic Binding6. Dynamic Binding

7. Abstraction7. Abstraction 12

EncapsulationEncapsulation

EncapsulationEncapsulation: : Used to restrict access to some of an Used to restrict access to some of an object's object's attributesattributes and its and its methodsmethods. .

Traditional programming Traditional programming Is NotIs Not an issue; an issue; Programs are typically Programs are typically full units full units whose private/protected whose private/protected methods methods are not modified are not modified by outside by outside programs.programs.

OO programmingOO programming: : ISIS an Issue: an Issue: DifficultDifficult to observe object interactions with variables and to observe object interactions with variables and methods methods not visiblenot visible outside the class outside the class

13

EncapsulationEncapsulation• Visibility Issues and State Issues for EncapsulationVisibility Issues and State Issues for Encapsulation: : More difficult to be More difficult to be aware of an object's aware of an object's statestate, , (private (private attributes and methods can be affected with simple attributes and methods can be affected with simple getters and setters.) getters and setters.)

Visibility and State are Visibility and State are iimportant for class and mportant for class and cluster testing. cluster testing. A class's or other class's methods A class's or other class's methods may change an object's may change an object's state state

State unobservableState unobservable: If part of that state cannot be : If part of that state cannot be observed, then it will be difficult to observed, then it will be difficult to designdesign test cases and test cases and observeobserve test results. test results.

14

Encapsulation – more Encapsulation – more IssuesIssues

ForFor Objects strongly encapsulated: Objects strongly encapsulated: must find a way must find a way to to ensure private fieldsensure private fields are correct if they are are correct if they are modified by modified by other classes. other classes.

Initial state?Initial state?: Often initial state cannot be : Often initial state cannot be determined. determined. Thus Thus ability to control a test's inputsability to control a test's inputs may be difficultmay be difficult ((This might mean This might mean creating new creating new methods methods to display a class's to display a class's state which may or may state which may or may not go against the class goals as not go against the class goals as designed.) designed.)

15

Inheritance Inheritance (1 of 4)(1 of 4)

InheritanceInheritance is a mechanism for is a mechanism for sharingsharing attributes/behaviors attributes/behaviors from pre-existing classes to other subclasses. from pre-existing classes to other subclasses.

Only 'near' comparison in Traditional Testing: method or Only 'near' comparison in Traditional Testing: method or structure reuse.structure reuse.

IssueIssue: One class a subclass of another, not ==> all of the : One class a subclass of another, not ==> all of the inherited methods are correct inherited methods are correct ifif they have been verified in the they have been verified in the super classsuper class

IssueIssue: Superclass well tested does not ==> classes that inherit : Superclass well tested does not ==> classes that inherit it will be correct.it will be correct.

16

Inheritance - Complexity Inheritance - Complexity Metric:Metric:

Depth of Inheritance Tree Depth of Inheritance Tree (DIT)(DIT)

DIT = path length of class to root class in inheritance treeDIT = path length of class to root class in inheritance tree. .

FactFact: Deeper the class in tree, larger number of methods that can be : Deeper the class in tree, larger number of methods that can be inherited. inherited. FactFact: High DIT ==> : High DIT ==> behaviorbehavior much more complicated, much more complicated, FactFact: More difficult to : More difficult to predict behaviorspredict behaviors, , FactFact: Much more difficult to : Much more difficult to design test cases!design test cases!

IssueIssue: Cannot test a child class child without its parent class : Cannot test a child class child without its parent class BehavioralBehavioral Errors might Errors might propagatepropagate down the inheritance tree.) down the inheritance tree.)

IssueIssue: What about an inherited method changed in the subclass but the : What about an inherited method changed in the subclass but the subclass has an untouched, inherited method that uses the changed method? subclass has an untouched, inherited method that uses the changed method?

Overridden method and the untouched method both need to be testedOverridden method and the untouched method both need to be tested17

Inheritance - Complexity MetricInheritance - Complexity MetricNumber of Children (NOC) Number of Children (NOC) (3 of (3 of

4) 4)

NOC: Number of Children a class has. NOC: Number of Children a class has.

NOC – Number of NOC – Number of immediateimmediate subclassessubclasses in an in an inheritance hierarchy. inheritance hierarchy. ConsideredConsidered a a measure of the influence measure of the influence one class may have one class may have over the system as a over the system as a whole. whole.

IssueIssue: best tested in system and cluster testing : best tested in system and cluster testing UUsed to determine sed to determine how much emphasis how much emphasis should should be placed on be placed on testing a specific class. testing a specific class.

18

Metrics: DIT and NOC DIT and NOC (4 of 4)(4 of 4)

DIT and NOC often used to determine DIT and NOC often used to determine testingtesting overheadoverhead

DIT AdviceDIT Advice: Researcher advice: if DIT > 6 : Researcher advice: if DIT > 6

design complexity is high and testing overhead can be large.design complexity is high and testing overhead can be large.

NOC Advice:NOC Advice: If NOC is similarly high, If NOC is similarly high,

design of abstract classes is considered diluted design of abstract classes is considered diluted

Design of abstract classes is not utilized or Design of abstract classes is not utilized or Design Design of abstract classes are too general of abstract classes are too general

Difficult to assign Difficult to assign measurable metrics in inheritance measurable metrics in inheritance Inheritance Inheritance has many forms and has many forms and inheritance trees can become terribly inheritance trees can become terribly complex. complex.

SoSo,, Keep inheritance as simple as possible. Keep inheritance as simple as possible.

19

Polymorphism Polymorphism (1 of 2)(1 of 2)

NNo real analog to procedural development. o real analog to procedural development.

Polymorphism allows attributes to take on many forms or Polymorphism allows attributes to take on many forms or data types. data types.

Operations may Operations may 1. 1. returnreturn more than one data type or more than one data type or 2. 2. acceptaccept more than one data type for parameters. more than one data type for parameters.

Polymorphism: crucial to OO programming; makes it versatile and Polymorphism: crucial to OO programming; makes it versatile and reusablereusable. . But presents But presents great complexity great complexity in test design and can be a in test design and can be a huge task. huge task.

AdviceAdvice: A Class or a group of classes should be designed well enough so : A Class or a group of classes should be designed well enough so that the overhead to design a test is low that the overhead to design a test is low

Note: Note: notnot suggestingsuggesting polymorphismpolymorphism should not be used, but should not be used, but attributes of a class should be attributes of a class should be more limitedmore limited and and wellwell--defineddefined in in regards to both design and testing. regards to both design and testing.

20

Polymorphism Polymorphism (2 of 2)(2 of 2)

Two kinds of polymorphism: ad hoc polymorphism and universal Two kinds of polymorphism: ad hoc polymorphism and universal polymorphism polymorphism Ad Hoc Ad Hoc polymorphism is polymorphism is sharing names and coercion in expressionssharing names and coercion in expressions UniversalUniversal is considered true polymorphism; Is discussed in is considered true polymorphism; Is discussed in dynamic dynamic binding.binding.

High cost in testing if true polymorphic types are used in design and High cost in testing if true polymorphic types are used in design and implementation. implementation. Again, this is discussed in dynamic binding ahead.Again, this is discussed in dynamic binding ahead.

Understanding interactions resulting from the polymorphic nature of some Understanding interactions resulting from the polymorphic nature of some objects can be very daunting (but necessary) when developing test cases. objects can be very daunting (but necessary) when developing test cases.

TestingTesting Polymorphism AdvicePolymorphism Advice: : Ad hoc testing is less testing intense because the tests Ad hoc testing is less testing intense because the tests can be derived can be derived staticallystatically Universal polymorphism is Universal polymorphism is much more difficult to test because the many forms much more difficult to test because the many forms an entity an entity can take on during run time; This may be very wide-ranging. . can take on during run time; This may be very wide-ranging. .

21

Cohesion Cohesion (1 of 3)(1 of 3)

Cohesion: a Cohesion: a measuremeasure by which the methods of a class by which the methods of a class create a single, well-defined class. create a single, well-defined class.

Traditional paradigm: cohesion: Traditional paradigm: cohesion: how well module of code hangs together how well module of code hangs together as as a unit.a unit.

OO paradigm: talk about cohesiom in terms of its OO paradigm: talk about cohesiom in terms of its instanceinstance variablesvariables. .

Procedural programs do not have instance variablesProcedural programs do not have instance variables; ; Information is passed via parameters. Information is passed via parameters. Cohesion is concerned with Cohesion is concerned with methods with similar parameters and methods with similar parameters and functionality. functionality.

OO programsOO programs, class is cohesive if , class is cohesive if methods methods contributecontribute to the class as a single unit, and to the class as a single unit, and the class is more the class is more reusable, more reliable, and more easily understood. reusable, more reliable, and more easily understood.

22

Cohesion Metric: Lack of Cohesion Cohesion Metric: Lack of Cohesion MetricMetric

(LOCM) (LOCM) (2 of 3)(2 of 3)

LOCM: the LOCM: the mathematical difference mathematical difference between the between the number of methods whose number of methods whose instance variablesinstance variables are are completely dissimilar completely dissimilar compared to number of compared to number of methods where methods where instance variablesinstance variables are are sharedshared. .

ResearchersResearchers: LCOM and lines of code (LOC) are : LCOM and lines of code (LOC) are the most the most predictive methods predictive methods over over DIT, NOC, and CBO DIT, NOC, and CBO (Coupling Between (Coupling Between Objects - Objects - ahead).ahead).

23

Cohesion vs. Coupling in classes Cohesion vs. Coupling in classes (3 of 3)(3 of 3)

High cohesion (desirable) & low coupling (desirable) and vice High cohesion (desirable) & low coupling (desirable) and vice versa. versa.

High cohesion ==> methods similar in variables used and tasks High cohesion ==> methods similar in variables used and tasks performed. performed.

Data are more easily developed and understood.Data are more easily developed and understood.

Low cohesion ==> more different types of data and methods are foundLow cohesion ==> more different types of data and methods are found

Low cohesion leads to Low cohesion leads to higher testing costshigher testing costs and and error-prone testingerror-prone testing

Cohesion AdviceCohesion Advice: high cohesion is highly desirable!: high cohesion is highly desirable!24

Coupling Coupling (1 of 3)(1 of 3)

CCoupling: a measure of the oupling: a measure of the dependencydependency between between modulesmodules..

Strong coupling Strong coupling (undesirable) (undesirable) preventsprevents independentindependent testingtesting and and changing of variables and methods changing of variables and methods

Strong coupling ==>many or all of the coupled methods need to Strong coupling ==>many or all of the coupled methods need to be be understoodunderstood as a setas a set rather than each class operating as its rather than each class operating as its own unit. own unit.

Strong coupling negativelyStrong coupling negatively impactsimpacts testing testing

Strong couplingStrong coupling implies implies Unit Testing Unit Testing cannot be done cannot be done independently usually forcing independently usually forcing Cluster Testing. Cluster Testing.

25

Coupling Metric - Coupling Between Coupling Metric - Coupling Between Objects Objects

(CBO) (CBO) (2 of 3)(2 of 3)

CBO is a count of the CBO is a count of the number of classes number of classes to which a class is to which a class is coupled. ==> another measure of complexity. coupled. ==> another measure of complexity.

High CBO High CBO poor reliability and higher interoperability between classes. poor reliability and higher interoperability between classes.

NoteNote: Some interoperability between classes is generally required as objects need : Some interoperability between classes is generally required as objects need to to communicate in some way.communicate in some way.

High CBO High CBO unit testing unit testing may be difficult and may require may be difficult and may require cluster testingcluster testing..

Coupling AdviceCoupling Advice: G: Good design ==> ood design ==> minimizeminimize coupling especially coupling especially in large complex systems where strong coupling could force cluster and in large complex systems where strong coupling could force cluster and system testing absorbing system testing absorbing most of the testing resourcesmost of the testing resources. .

26

Other metrics for measuring coupling include efferent coupling and afferent coupling

Efferent coupling: measure of coupling between packages and refers to external packages a package depends upon.

Afferent coupling measure of coupling between packages where classes external to a package are dependent upon the classes inside the package.

27

(3 of (3 of 3)3)

Dynamic Binding Dynamic Binding (1 of 3)(1 of 3)

Consider Java: method Consider Java: method return typereturn type and/or and/or attributeattribute typestypes of some attributes can be of some attributes can be determined at determined at runrun timetime vice at vice at compilecompile timetime..

Dynamic binding feature : concerns when Dynamic binding feature : concerns when designingdesigning testtest cases cases Exact data types of Exact data types of attributes cannot be known attributes cannot be known staticallystatically..

IssueIssue: : Dynamic binding issues may arise from Dynamic binding issues may arise from universal polymorphism (as opposed to ad hoc universal polymorphism (as opposed to ad hoc poly) in some languages.poly) in some languages.

28

Dynamic Binding Dynamic Binding (2 of 3)(2 of 3)•Universal Polymorphism Universal Polymorphism has two subtypes: has two subtypes: inclusive inclusive polymorphism and parametric polymorphismpolymorphism and parametric polymorphism

In Parametric polymorphismIn Parametric polymorphism a method/object may be written in a a method/object may be written in a genericgeneric mannermanner through parameters and through parameters and given a class value when the object isgiven a class value when the object is instantiatedinstantiated. .

There is There is very high overheadvery high overhead in testing when we have parametric polymorphism in testing when we have parametric polymorphism

So in So in designing test cases designing test cases with parameter polymorphism, it may only be with parameter polymorphism, it may only be necessary to necessary to test based on test based on how other classes in the application will instantiate a how other classes in the application will instantiate a class. class.

Serious IssuesSerious Issues: This would be a part of cluster or system testing. : This would be a part of cluster or system testing. Unit testing / class testing is difficult because it may Unit testing / class testing is difficult because it may not be known without looking not be known without looking at the whole system, how a class might be at the whole system, how a class might be instantiated. instantiated.

Thus class and unit testing should not be done in the case of dynamic Thus class and unit testing should not be done in the case of dynamic binding binding due to its complexity as a unit. due to its complexity as a unit.

29

Dynamic Binding Dynamic Binding (3 of 3)(3 of 3)

In Inclusive polymorphismIn Inclusive polymorphism a subclass can be used in a subclass can be used in place of a superclass place of a superclass

Inclusive polymorphismInclusive polymorphism may be an easier form of may be an easier form of dynamic binding to test. dynamic binding to test. This is because it is often known This is because it is often known what classes inherit a what classes inherit a superclass. superclass.

In this way it can be known what kind of objects In this way it can be known what kind of objects may may be bound at runtime. be bound at runtime.

All combinations of these dynamic bindings must be run All combinations of these dynamic bindings must be run in a test case. in a test case.

30

Abstraction Abstraction (1 of 1)(1 of 1)

An Abstract Class: An Abstract Class: Cannot be instantiated; Cannot be instantiated;

Can serve (oftentimes serve) as an interface for other Can serve (oftentimes serve) as an interface for other classes.classes.

If class is an abstract class and If class is an abstract class and notnot an interface, it provides useful an interface, it provides useful methods and useful attributes. methods and useful attributes.

IssueIssue: : Independent TestingIndependent Testing. . Cannot Cannot instantiateinstantiate an abstract class or an interface. an abstract class or an interface.

This ==> This ==> defineddefined methodsmethods cannot be tested independently cannot be tested independently and and Analysis must thus done from their subclasses. Analysis must thus done from their subclasses.

IssueIssue: : InheritanceInheritance. IF an abstract class is . IF an abstract class is inheritedinherited by more than one by more than one class, how many of these child classes should be tested? class, how many of these child classes should be tested?

Even if the child classes have not overridden the inherited methods, Even if the child classes have not overridden the inherited methods, AllAll methodsmethods would need to be tested because the abstract class would need to be tested because the abstract class by itself cannot be directly tested. by itself cannot be directly tested. 31

Conclusions Conclusions (1 of 3)(1 of 3)

Key: OOT is based Key: OOT is based not only not only on both the on both the I/OI/O of an object’s of an object’s methods, but also how that input and output may influence methods, but also how that input and output may influence the the object’s stateobject’s state. . Great features of OO paradigm may: Great features of OO paradigm may: CCreate greatly reate greatly increase in program complexity increase in program complexity (sometimes (sometimes

in unintended and unseen ways) in unintended and unseen ways) oftenoften negatively impacting negatively impacting test effort and timetest effort and time. .

Cohesion Cohesion reducesreduces amount of testing, while encapsulation, amount of testing, while encapsulation, inheritance, polymorphism, coupling, dynamic binding, and inheritance, polymorphism, coupling, dynamic binding, and abstraction abstraction addadd complexity and cost in test design. complexity and cost in test design.

32

Conclusions (2 of 3)(2 of 3)

Traditional testingTraditional testing: view input and output of a program in a procedural : view input and output of a program in a procedural manner. Test cases tend to be manner. Test cases tend to be one dimensionalone dimensional. Traditional testing . Traditional testing involves both unit and system testing. involves both unit and system testing.

Object-oriented testingObject-oriented testing: test cases are : test cases are two dimensionaltwo dimensional, because , because changes in an object’s state must be considered. changes in an object’s state must be considered.

Object-oriented testing Object-oriented testing also requires also requires Class testing Class testing (for how the methods (for how the methods of of a single object a single object

work together) and work together) and

Cluster testing Cluster testing (for how coupled objects change each (for how coupled objects change each other’s’ states). other’s’ states).

Verification testing Verification testing (the testing done by the developers) has been truly (the testing done by the developers) has been truly changed by the object-oriented paradigm, while changed by the object-oriented paradigm, while ValidationValidation testingtesting (that (that done by the end-user) has not.done by the end-user) has not.

Conclusions (3 of 3)

There has been There has been no major consensusno major consensus as to what the best as to what the best way to test is or what factors are most important in way to test is or what factors are most important in testing. testing.

Most assert: Most assert: testingtesting OO-modules is not as OO-modules is not as definitedefinite as in as in traditional programs, where the order of tests follows a traditional programs, where the order of tests follows a procedural path. procedural path.

Clearly in OOT, testing rises to a new level of concerns Clearly in OOT, testing rises to a new level of concerns as we reap the benefits of the OO paradigm.as we reap the benefits of the OO paradigm.