patterns: from system design to software testing

Innovations Syst Softw Eng (2008) 4:71–85DOI 10.1007/s11334-007-0042-z

ORIGINAL PAPER

Patterns: from system design to software testing

Neelam Soundarajan · Jason O. Hallstrom ·Guoqiang Shu · Adem Delibas

Received: 16 December 2007 / Accepted: 19 December 2007 / Published online: 12 January 2008© Springer-Verlag London Limited 2008

Abstract Design patterns are used extensively in the designof software systems. Patterns codify effective solutions forrecurring design problems and allow software engineers toreuse these solutions, tailoring them appropriately to theirparticular applications, rather than reinventing them fromscratch. In this paper, we consider the following question:How can system designers and implementers test whethertheir systems, as implemented, are faithful to the require-ments of the patterns used in their design? A key considera-tion underlying our work is that the testing approach shouldenable us, in testing whether a particular pattern P has beencorrectly implemented in different systems designed usingP , to reuse the common parts of this effort rather than hav-ing to do it from scratch for each system. Thus in the approachwe present, corresponding to each pattern P , there is a setof pattern test case templates (PTCTs). A PTCT codifies areusable test case structure designed to identify defects asso-ciated with applications of P in all systems designed usingP . Next we present a process using which, given a systemdesigned using P , the system tester can generate a test suitefrom the PTCTs for P that can be used to test the particularsystem for bugs in the implementation of P in that system.This allows the tester to tailor the PTCTs for P to the needs

N. Soundarajan (B) · G. Shu · A. DelibasComputer Science and Engineering, Ohio State University,Columbus, OH 43210, USAe-mail: [email protected]

G. Shue-mail: [email protected]

A. Delibase-mail: [email protected]

J. O. HallstromSchool of Computing, Clemson University,Clemson, SC 29634, USAe-mail: [email protected]

of the particular system by specifying a set of specializa-tion rules that are designed to reflect the scenarios in whichthe defects codified in this set of PTCTs are likely to man-ifest themselves in the particular system. We illustrate theapproach using the Observer pattern.

Keywords Design patterns · Contracts · Testing

1 Introduction

Design patterns [4,11,24] have had a profound impact onhow software systems are built. This is not surprising sincepatterns capture the distilled wisdom of the software com-munity, and provide effective solutions to recurring designproblems. They allow software engineers to reuse these solu-tions, tailoring them to the needs of their particular systems,rather than reinventing them from scratch. Patterns are usedprimarily during the design phase of software systems. Inthis paper, we focus on later stages of the software life-cycle,the testing and maintenance phases. Specifically, the ques-tion we are interested in is, how can system designers andimplementers test whether their systems, as implemented, arefaithful to the requirements of the underlying patterns? Thisquestion is especially important for large systems since suchsystems tend to have correspondingly large software teamsand hence a correspondingly greater likelihood that differ-ent members of the team may have subtly different inter-pretations of the patterns underlying the system’s design.Hence the team members may implement their respectiveparts of the system in ways such that when these parts are puttogether, the resulting overall system is not compatible withthe patterns’ intent. Design-related bugs of this kind may notmanifest themselves in test cases that are developed usingstandard approaches such as testing against the functionalrequirements or even code-based criteria such as statement,

123

72 N. Soundarajan et al.

or branch, or path coverage [3]. The testing techniques andsupporting tools discussed in this paper will be of value inreducing the likelihood of such bugs going undetected, andwill aid in their localization.

The problem of ensuring that a system is faithful to thepatterns underlying its design is even more serious duringthe maintenance phase since system maintenance may bethe responsibility of team members who were not involvedwith the original design or implementation of the system—and consequently have only a superficial understanding ofits design. As a result, over time, the design integrity of thesystem is likely to erode. The approach we develop in thispaper can help address this aspect of the problem as well.This is because the test suite used to test whether the systemis faithful to its underlying patterns can be included as akey part of the system’s design documentation in the samemanner as standard unit tests or other functionality tests. Asthe system evolves during maintenance, the design test suitewill help system maintainers identify and localize changesthat may conflict with the original design. If one of these testswere to fail, that doesn’t necessarily mean that the changesin question are flawed. It may be that the design needs tobe modified; but this decision should be made consciouslywith the design documentation being appropriately updated,rather than simply modifying the code. We will return to thispoint in the final section of the paper.

In standard software testing, we test software against itsspecifications. These specifications must be formal since,otherwise, there is no way to conclude whether a test was suc-cessful or not1. Similarly, to test whether a system is faithfulto the intent of the patterns underlying its design, we needformal specifications of the patterns.

In previous work [12,26,31], we presented an approach toproviding precise specifications for patterns and their appli-cations in the form of pattern contracts and subcontracts. Itis against these contracts and subcontracts that we wish totest individual systems to validate their design correctness.

An important aspect of this approach to pattern speci-fication is that the requirements and behavioral guaranteesspecified in a pattern contract apply to all uses of the pattern;a subcontract characterizes how the pattern is specialized ina particular application. Thus the wisdom of the communitycaptured in a design pattern is reflected in the correspondingpattern contract; it can be reused by every team that uses thepattern in the design of its system. The individual team need

1 When is a test successful? One definition in the testing literature [3]says that a test is successful when it shows that the software does notmeet its specification, since the purpose of the test is to find flaws inthe software. Others [20] use the convention that a test succeeds, or thatthe software “passes” the test if, during the test execution, the softwarebehaves according to its specification. We will use this latter conventionin this paper.

only create the subcontract that specifies how the pattern isspecialized for use in a particular application.

A similar consideration is a primary motivation for theapproach we present in this paper. That is, a key goal of ourwork is that the testing approach should enable us, in test-ing whether a particular pattern P has been correctly imple-mented in different systems designed using P , to reuse thecommon parts of this effort rather than having to implement itfrom scratch for each of these systems. Thus in the approachwe develop, corresponding to each pattern P , there is a setof what we will call pattern test case templates (PTCTs). APTCT codifies a reusable test case structure designed to iden-tify defects associated with applications of P in all systemsdesigned using P . Next we present a process using which,given any system designed using P , the system tester cangenerate a test suite from the PTCTs for P that can be usedto test the particular system for bugs in the implementationof P . The process allows the system tester to tailor the setof PTCTs corresponding to P to the needs of the particularsystem by specifying a set of specialization rules that aredesigned to reflect the structure and the scenarios in whichthe defects codified in the set of PTCTs for P are likely tomanifest themselves.

An example will help illustrate our approach. Considerthe classic Observer pattern which we will use as our pri-mary case study throughout the paper. According to the stan-dard description [4,11] of the pattern, participating objectsplay one of two roles: Subject or Observer. The intent ofthe pattern is to keep multiple observers consistent withthe state of a single subject. When an object wishes tobecome an observer of a subject, it invokes the subject’sAttach() method. When it is no longer interested, it invokesthe Detach() method. In addition, Subject includes aNotify() method which, according to the intended usage ofthe pattern, is required to be called whenever the state of thesubject is modified.Notify() is required to invokeUpdate()

on each attached observer which must, in turn, update thestate of that observer to make it consistent2 with the currentsubject state.

The standard UML diagram for the pattern appears inFig. 1. The subject maintains a set of references to thecurrently attached observers in the variable _observers.According to the standard description, when Attach() isinvoked, it adds a reference to the attaching observer to_observers. But if this was all Attach() did, the intent of thepattern would be violated; this is because the newly attachedobserver’s state may not be consistent with the current stateof the subject – it will remain inconsistent until the next

2 What precisely terms such as “modified” and “consistent” mean is notclear from such informal descriptions. The formal contract for Observerwhich we will see in the next section will resolve this and other ambi-guities in the informal description.

123

Patterns: from system design to software testing 73

Observer

+Update()

o.Update()

ConcreteSubject

−subjectState

*

ConcreteObserver

−observerState

+Update()

*

1

1

+Attach(in Observer)

+Detach(in Observer)

Subject

+Notify()

_observers

for all o in _observers

Fig. 1 Observer pattern

notification cycle. To address this problem, Attach() mustinvoke Update() on the newly attached observer. Failure todo this is a common bug that appears in systems designedusing the Observer pattern. It is also a bug that can be missedduring standard (functional) testing since the inconsistencyof the newly attached observer may last only for a shorttime, depending on when exactly the next modification inthe subject state takes place, resulting in a notification cyclewhich will remove the inconsistency.

In our approach, given that this problem is common to sys-tems designed using the Observer pattern, one of the PTCTsfor this pattern, as we will see, will correspond to this poten-tial bug. When testing a system designed using Observer,the system tester will specialize this PTCT appropriately togenerate a set of test cases that will help test the particularsystem for the presence of this bug. These tests will confirmthat when a new observer in this system attaches to the sub-

ject, it is appropriately updated. In other words, these testswill help ensure that the system is faithful to (this particu-lar aspect of) the intent of the Observer pattern; and if thesystem were to be modified during system evolution, the testcases and the associated documentation will help the systemmaintainers test for this bug and to identify any violationsof the intent of the underlying pattern that may be insertedduring this phase. Thus, the approach will help preserve thedesign integrity of the system as it evolves.

Although some approaches to formalizing patterns havebeen proposed [8,10,19,23] (in addition to our work citedabove), we are not aware of any other work focused onthe question of testing a system to see whether it correctlyimplements the patterns underlying its design. Given thewidely recognized importance of design patterns and theiruse in designing and implementing large software systems,we believe that this is an important concern and believe thatthe approach we present will prove valuable for this task.

Paper organization. Section 2 summarizes our approach topattern contracts and their relation to PTCTs. Section 3 devel-ops the essential ideas of PTCTs and how they are definedbased on pattern contracts. A number of PTCTs for Observerare presented based on the contract for the pattern. Section4 considers how test cases, for a particular system designedusing a given pattern, may be generated from the PTCTs forthe pattern; a simple system, the simulation of a Hospital, isused for illustration. Issues of coverage (and the associatedmetrics) in the context of pattern-centric testing are also dis-cussed. Section 5 briefly surveys elements of related work.Section 6 concludes with a summary of the approach, and itsrelation to test-driven design (TDD).

2 Pattern contracts

The contract for a design pattern P consists of a set of role-contracts, one corresponding to each role of P , and a portionthat corresponds to the pattern as a whole. The role-contractfor a role R lists the state components of R, and specifies, instandard pre-/post-condition format, requirements that mustbe satisfied by the various methods of R. In a system designedusing P , a class C playing the role R will typically pro-vide other methods in addition to those “named” in R. Ifthese “other” methods were to behave arbitrarily, then theintent of the pattern would be violated even if the methodscorresponding to the named methods behaved according totheir specifications in the role-contract. In order to elimi-nate this possibility, R’s role-contract will also include anothers specification that must be satisfied by all methodsof C , except those corresponding to the ones named in therole-contract. The portion of the contract that correspondsto the pattern as a whole consists of an invariant over the(role) states of the various objects enrolled, at runtime, in aninstance of the pattern. The invariant will be satisfied when-ever control is not inside any of the methods of any of theparticipating objects.

One potential problem with formalizing a design patternis that its flexibility might be reduced or even eliminated.A number of features of our formalism help guard againstthis, the most important of these being the notion of an aux-iliary concept. An auxiliary concept is a relation over oneor more states of one or more objects interacting with eachother according to the pattern. Auxiliary concepts are usedin the role-contracts and the pattern invariant, but their defin-ition are not part of the pattern contract. Instead, definitionstailored to particular systems, are provided in the subcon-tracts corresponding to systems. The subcontract for aparticular system also includes a set of role-maps, onecorresponding to each class C playing a role R in the pattern,as applied in this system. The C-R-role-map specifies howthe state, i.e., variables, of C map to the state of R (listed in its

123


role-contract), which methods of C correspond to each of thenamed methods of R, etc.

As noted in Sect. 1, the pattern contract applies to allsystems designed using the pattern; the subcontract speci-fies how the pattern is specialized for use in a given system.Correspondingly, the pattern test case templates that we willdevelop in the next section and will apply to all systemsdesigned using the pattern, will be based on the pattern’s con-tract. The test cases for a given system will be obtained byspecializing the templates using the information in the sub-contract and accounting for additional application-dependentfactors, as we will see later.

Consider the Observer pattern. The intent of the pattern, asnoted earlier, is to ensure that when the state of the subject ismodified, the observers “attached” to the subject are appro-priately updated so that their states become consistent withthe current state of the subject. “Modified” does not, how-ever, mean a change in any arbitrary bit or byte of the sub-

ject state; rather, some modifications are important enough torequire updates of the attached observers and others are not.At the same time, which modifications are important enoughin this sense and which are not varies from system to system.Hence, we use an auxiliary concept, Modified(), a relationover two states of the subject, to distinguish “important”changes in the subject state from “unimportant” ones. Sim-ilarly, what it means for an observer state to be “consistent”with the subject state varies from system to system; and,indeed, from one kind of observer that may be attached tothe subject to another that may be attached to the same sub-

ject. Hence we use a second auxiliary concept, Consistent(),a relation over an observer state and a subject state, to rep-resent this notion. The contract for Observer, a portion ofwhich appears in Fig. 2, is written using these concepts andspecifies the requirements that apply to all systems designedusing this pattern.

Many patterns are concerned with the sequences of inter-actions between various objects, i.e., the sequences of methodcalls invoked on the various objects involved in the pattern.Thus, as we noted in Sect. 1, the Notify() method of theSubject role is required to call Update() on each of theobservers currently attached to the subject. Such require-ments are expressed in the pattern contract using the trace τ ,a ghost variable [15] provided by the formalism. In effect,when a method m() starts execution, a corresponding traceτ , initialized to the empty sequence, is automatically cre-ated. Each call that m() makes to a named method duringits execution is recorded as an element of the sequence, andincludes information about the object on which the methodwas invoked, the name of the method, and information aboutthe associated arguments and return values. A number ofmathematical functions, some of which we will introduceas needed, simplify trace manipulation, access to individualtrace elements, etc. Contract requirements concerning object

Fig. 2 Observer pattern contract (partial)

interactions are then expressed in the form of appropriateconditions on the trace variables. As we will see in the nextsection, PTCTs will be designed to test whether the sequenceof interactions that occur in a system designed using the pat-tern satisfy these conditions.

The contract for Observer starts by listing the auxiliaryconcepts. Next (lines 6–8), we have a constraint that mustbe satisfied by the definitions of these concepts in any sub-contract of Observer; we will consider this after consider-ing the rest of the contract. Next (lines 10–11), we havethe pattern invariant, the reason (or the “reward” [26]) forusing the pattern, that is guaranteed provided we meet all therequirements specified in the pattern contract. players[] isthe array of objects enrolled in the pattern instance in ques-tion and is another ghost variable of the contract formalism.In the case of an instance of Observer, the first player toenroll3 in any instance of the pattern is the subject; hence

3 The complete pattern contract will also include clauses that specifyhow an object enrolls to play a given role in a pattern instance, howa pattern instance is created, etc. We omit the details of these here;see [12,26]. When testing a system designed using the pattern, it is,of course, necessary that the requirements specified in these clausesof the pattern contract (specialized for the particular system) are satis-fied. Hence appropriate PTCTs must be defined, or these tests must beincluded as part of other PTCTs.

123


(a reference to) this object will be in players[0]. The restof the objects in players[] will be the objects that enroll toplay the Observer role (by invoking the Attach() method,this being the action required for this purpose as specifiedin the (elided from Fig. 2) enrollment clause for this role).Thus the invariant (lines 10–11) states that the state of thefirst object in players[], i.e., the state of the subject in anygiven instance of this pattern, is Consistent() with the statesof each of the observers (currently) enrolled in this instance.

Next we have (part of) the Subject role contract. First wespecify the state of the subject as consisting of obs, used tostore references to the currently attached observers. Next wehave the specification ofAttach(), one of the named methodsof this role. The pre-condition requires that the attachingobserver not already be attached, i.e., not have a referenceto it in obs. The post-condition states4 that a reference to theobject is added toobs; that the subject itself is not modified,in the sense of the auxiliary concept Modified(), and that onecall to a named method of the pattern has been made duringthe execution ofAttach(), this being to theUpdate() methodinvoked on the attaching observer.

The others specification (lines 20–23) states that anyother method of the class playing the Subject role shouldmake no change in obs, and should either not modify (againin the sense of Modified()) the subject state, or must invokethe Notify() method (on the current subject). The (elided)specification of Notify() states that this method invokesUpdate() on each observer in obs.

The Observer role contract states that the state of thisrole consists of the variable sub (that holds a reference tothe subject to which this observer is attached); that theUpdate() method does not change the value of sub, andmakes the state of the observer to be consistent (in the senseof the auxiliary concept Consistent()) with that of the sub-

ject. The others specification also states that it not modifysub, and leave the observer in a state that is consistent withthat of the subject5.

Let us now consider the constraint specified in the con-tract (lines 6–8). Although the definitions of the auxiliaryconcepts, tailored to the needs of a particular application, willbe provided by the corresponding subcontract, these defini-tions cannot be completely arbitrary. For example, suppose,

4 We use the “#” notation in the post-condition to refer to the pre-condition value of the variable, i.e., its value when the method startedexecution.5 Standard informal descriptions [11,25] of the pattern seem to suggestthat the other methods of this role should not make any changes inthe state of the observer. But this is unnecessarily restrictive. As thepattern contract states, all we need is that any changes in the observerbe such that they leave the observer state consistent with the sub-ject state. This is an example [12,26] of how our pattern contracts canidentify dimensions of flexibility that may be missing in the standardinformal descriptions.

in a particular application, that s1, s2 are two states of thesubject, and o1 is the state of an observer. Suppose that thedefinitions of Modified() and Consistent() for this applicationare such that each of the following is true:

Consistent (s1,o1), ¬Modified (s1,s2), and¬Consistent (s2,o1)

Suppose, at some point in the execution of this system, thesubject is in the state s1 and the observer in the state o1.Suppose, finally, that the subject state changes from s1 tos2. Then, according to the others specification in the Sub-

ject role, the Notify() method will not be invoked and henceUpdate() will not be invoked on this observer and its statewould remain as o1. At this point, the current state, o1, of theobserver will be inconsistent with the current state, s2, ofthe subject, and the invariant that the pattern was intendedto guarantee will be violated although the individual meth-ods in the individual roles, as actually implemented in theparticular application, satisfy their respective specifications.The problem is that we have conflicting notions of what asufficiently serious modification in the subject state is thatrequires the observers to be updated on the one hand, versuswhat it means for the subject state to be consistent with theobserver state, on the other. The constraint specified (lines6–8) in the contract ensures that this problem doesn’t arise.Thus while the designers of a particular system are free totailor the definitions of the auxiliary concepts to their partic-ular needs, they must ensure that these definitions satisfy theconstraint specified in the contract.

How realistic is this problem? That is, how likely is itthat the software team responsible for an actual applicationdesigned using the Observer pattern would base their sys-tem on such mutually conflicting notions of Modified() andConsistent()? It depends on the size of the team in question;as the team grows larger, the likelihood of such problemsseem to also grow. But even for relatively small systems, theproblem can creep in during system evolution; we will seean example of this later in the paper.

Unfortunately, however, specific PTCTs (or, more pre-cisely, test cases obtained by specializing them for a partic-ular application) cannot test these constraints because theyare general conditions typically involving universal quanti-fiers over the states of the various objects enrolled in thepattern instance. Instead, violations of these constraints mayshow up as violations of, for example, the pattern invariant—although the individual methods invoked as part of the testcase may satisfy their own specifications. We will see this inthe discussion of the example.

3 Pattern test case templates (PTCTs)

Given the requirements captured in a pattern contract, howdo we develop appropriate PTCTs that can be used to test

123


Fig. 3 Pattern-centric testing

whether systems implemented using the pattern satisfy theserequirements? It is important to stress that, given a patterncontract, construction of the corresponding PTCTs is not amechanical activity. The purpose of the PTCTs is to help testfor common mistakes in implementing the particular pattern.Clearly, which mistakes are common and which are not is aquestion that can be answered only on the basis of experiencewith systems designed using the pattern. In this section, wewill present three PTCTs, the first two corresponding to theObserver pattern and the last one corresponding to the Com-posite pattern. These are based on our assessment, based onour own experience with implementing systems and basedon the reports in the literature, of the most common mis-takes in implementing this pattern. PTCTs corresponding toother patterns will similarly be based on an assessment ofcommonly reported mistakes made in implementing them.But these are, of course, not cast in stone. Indeed, with addi-tional information about such mistakes, these PTCTs will berefined and additional PTCTs introduced. The main contribu-tion of this paper, then, is the approach presented for testingsystems to ensure the correct implementation of their under-lying patterns—not the particular PTCTs presented. We willreturn to this point in the final section.

Figure 3 schematically illustrates our approach to test-ing systems against the patterns underlying their designs.The pattern contract formally captures the requirements ofthe design pattern, the subcontract specifies how the pat-tern is specialized in the particular system. The PTCTs aredesigned based on the most common mistakes that are madein applying the pattern and are designed to check whether or

not the contract requirements are satisfied in those commonsituations. These templates must then be specialized to obtainthe actual test cases corresponding to a particular system. Thesystem tester does this specialization, as we will see in thenext section, with help from JDUnit6, a testing framework/-tool that we are in the process of building, by combiningthe information in the contract, in the subcontract and thePTCTs, and taking account of any relevant details about theparticular system.

The language for defining PTCTs is intended to mirrorstandard Java programming syntax to support rapid practi-tioner adoption. Before looking at specific PTCTs, we sum-marize the syntax of the structure of a PTCT. A PTCT isrepresented as a segment of stylized Java code with inter-spersed asserts that, based on the pattern contract that thePTCT corresponds to, are expected to be satisfied at thosepoints. The asserts will be based on the pre- and post-conditions of the named methods as well as the other spec-ifications in the various role contracts of the pattern contract.In addition, some of the asserts may also be based onthe pattern invariant. Since the PTCT is intended to apply toall applications of the corresponding pattern, the construc-tor and method calls used in its definition must conform tothe signatures specified in the relevant role contracts. Whenthe actual test cases corresponding to a given application aregenerated from the PTCT, information in the correspondingsubcontract about the mapping of named methods of theroles to particular methods of the corresponding classes andthe mappings of the role-method signatures to the class-levelmethod signatures will be used to translate the statements inthe PTCT to appropriate code in the test cases.

A PTCT begins with an initialization portion that createsinstances of one or more role types. named methods maybe invoked to bring the new player objects into states appro-priate for the test case family being developed. The PTCTdeveloper may optionally include calls to other methods,which serve as placeholders for class methods, which maybe introduced at the point of test case generation. Since theprecise number of other method calls required to bring anobject into the state appropriate for testing a particular systemvaries, the PTCT syntax allows developers to specify that asingle other method may be instantiated by multiple classmethods, supplied at the point of test case generation. Theinitialization section typically concludes with a PTCT pre-condition. This assertion, expressed as a standard booleanexpression, captures properties assumed by the PTCT body.The check ensures that the class-level constructor and method

6 The name of the framework, currently under construction, is intendedto draw a parallel to JUnit, the popular unit testing framework for Java;the additional D emphasizes that the framework is used to test units ofdesign.

123


calls introduced at the point of PTCT instantiation satisfy thispre-condition.

Next, a PTCT defines a body. The body portion gener-ally follows a standard testing idiom: First, local variablesare used to store the pre-conditional states of the playersparticipating in the PTCT. Next, invocations are placed tothe methods of interest, i.e., those relevant to the designpattern requirements being tested. Finally, the PTCT con-cludes with one or more assertions used to check that therelevant pattern requirements are satisfied. These assertions,as noted above, are based on the method and invariant spec-ifications in the pattern contract and generally involve thepost-conditional states of the participating players, as wellas the pre-conditional states saved at the start of the body.

There is, however, one important deviation from the stan-dard testing idiom: Since key parts of the pattern contractinvolve the trace τ of method calls, the assertions containedin a PTCT will correspondingly include conditions on a spe-cial variable tau, used to record the trace of invocations orig-inating from the PTCT. Note that the PTCT is not allowedto contain any code that directly updates this variable; themanagement of tau and updating it to reflect the methodinvocations as they happen is the responsibility of JDUnit.The one exception to this rule is that a PTCT may clear tau

when the earlier invocations recorded in it are no longer ofinterest. In addition, JDUnit provides a number of helperfunctions that we may use in a PTCT; these functions allowus to obtain information about the current contents of tau.We will see several of these in the PTCTs that we discussnext.

There are some important differences in detail betweenthe trace τ used in the specifications in the pattern contractand the trace tau maintained by JDUnit. The primary reasonfor these differences is to enable us to construct more usefulPTCTs. In particular, suppose we have a PTCT that includesa call to a method m1() followed by a call to method m2().Thus the invocation of m1() completes before the invocationof m2() begins; i.e., the execution of neither is nested insidethe other. Suppose the trace of method calls that takes placeduring the execution of m1() is t1 and that during m2() ist2. The trace tau maintained by JDUnit allows us to write anassert following the completion of m2() that, for example,imposes conditions on t2 based on the value of t1 or vice-versa. We will see these details shortly.

The main portion of a PTCT intended to test the behav-ior of the Attach() operation of the Subject role of theObserver pattern appears in Fig. 4. As discussed earlier, acommon problem in using this pattern is not updating anobserver when it initially attaches to a subject. In the PTCT,we begin by creating a subject s (line 1), and an observer o(line 2). Next, we use a “getter” method provided by JDUnit,to retrieve the pre-conditional value of s.obsbefore the call toAttach() (line 7), and to store this value in pre_obs. Note

Fig. 4 PTCT for subject.attach()

that JDUnit provides appropriate getter methods for eachrole field, including private fields.) Next, we add the attach-ing observer to pre_obs (line 4) to simplify the expressionof the assertion check corresponding to the post-condition ofAttach() (lines 9–10). We additionally store the current stateof s (line 5), since this is required to check that Attach() notmodify s according to the applicable definition of Modified().

A couple of points should be noted here. The getter meth-ods are more involved than simply returning the current valueof the field in question. In an actual system designed using thepattern, the class C playing a particular role R may providea different set of fields from those specified in R’s role con-tract. As we will see in the next section, the subcontract forthe system will specify the mapping from C’s fields to thoseof R. The getter methods provided by JDUnit will make useof this mapping to translate the values of C’s fields to thecorresponding values of R’s fields. Similarly, the clone()operation applied to s (line 5) will create an object of what-ever class plays the Subject role in the particular application.Finally, JDUnit will use the auxiliary concept definitionsprovided in the subcontract to evaluate assertions involvingconcept references, such as the one involving Modified()(line 9). We will return to these points in the next section.

Let us now consider contract requirements involvingtraces. As we saw, in the contract formalism, each methodm() has an associated trace τ , initialized to the emptysequence when the method starts execution. Informationabout calls made by m() to named methods are recorded inτ , with m()’s post-condition imposing necessary conditionson τ . In those cases where a PTCT includes calls to methodswith trace requirements, the PTCT will include assert state-ments that check relevant conditions on the associated tracevariables. Indeed, a PTCT may include assert statementsthat require particular relations to hold across multiple tracevariables, each associated with methods preceding the asser-tion check. To enable easy expression of such asserts, we usethe following approach. When the instantiated PTCT beginsexecution, the testing framework creates a trace object tau,initialized to the empty sequence. Each named method

123


invoked during the execution of the PTCT is recorded asan element of tau, and includes information about the tar-get object, the identity of the method invoked, etc. In addi-tion, and this is the key difference from the use of τ in thecontract formalism, each element of tau itself includes thetrace of methods executed during the associated call’s exe-cution. Thus, while there is only a single tau object, it main-tains a branching structure corresponding to the computationtree rooted at the PTCT. The traces contained in the individ-ual elements of τ may be extracted using a simple accessor(“tr()”) function provide by JDUnit.

Immediately prior to the call to Attach(), we clear tau

(line 6). This removes the trace entries associated with thepreceding constructor calls (lines 1–2), as well as any calls tonamedmethods introduced during the specialization processin going from the PTCT to actual test cases; in the nextsection we will see how the specialization may introducesuch calls. Hence, when control returns following the call toAttach(), tau contains a single entry, corresponding tothis call. The trace of method calls that occurred during theexecution of this Attach() is extracted and stored in t1

(line 8). The assert that follows imposes appropriate condi-tions on this trace, based on the requirements specified in thepattern contract. The first two clauses require that s._obs beappropriately updated (line 9), and that s not be Modified()(line 10). The last three clauses (lines 11–13) address traceconditions imposed on Attach(). Together, these clausesrequire that Attach() invoke exactly one named method,and that this call be to the Update() method of the attach-ing observer7.

The assert statement is, of course, based on the specifi-cation of Attach() included in the Subject role contractshown in Fig. 2. The effort involved in writing the PTCTscorresponding to a pattern is primarily in identifying thecommon mistakes that are made in applying the pattern andcoming up with appropriate test case templates based on this.Once that has been done, the appropriate asserts to includeare determined by the pattern contract. But even here, thePTCT designer may choose not to include all of the clausesspecified in the role contract as the post-condition of theparticular method. Even more importantly, at what pointsto include checks of the pattern invariant and assertions thatrequire particular relations between the traces associated withtwo or more method calls appearing earlier in the PTCT, areall decisions the PTCT designer must make. And these deci-

7 We obtain the identify of the method invoked and the target objectby using the helper functions mt() and ob() provided by the JDUnitframework. These functions may be applied to individual elements of atrace in which case they return the identify of the method/target objectinvolved in that element. Or they may be applied to a trace, in which casethey return the sequence of method identities/target objects involved inthe various elements of the trace. We such usage in the pattern contractin the last section.

Fig. 5 PTCT for subject.other()

sions will be based primarily on the common mistakes thatpractitioners make in using the given pattern.

Note also the use of appropriate temporary objects tocapture pre-conditional values referenced in various post-conditions; this is common in specification-based testinggiven that post-conditions refer to both the state at the end ofthe method and at its start. The pattern invariant (lines 6–7,Fig. 2) could also have been checked as part of the assert

statement. This would amount to checking that Attach() notonly invokes Update() on the attaching observer, but alsothat the latter method appropriately updates the observer’sstate to be Consistent() with the subject.

Next consider the PTCT shown in Fig. 5, intended to testthe behavior of Subject’s other methods. We begin by cre-ating a subject (s) and two observers (o1, o2) (lines 1–3),and attach both observers to the subject (line 4). We thensave the pre-conditional value of s.obs (line 5) and the valueof s as a whole (line 6). tau is then cleared, and some other

method is invoked on s (line 7). Finally, the trace associ-ated with the other call is saved (line 8), and the behaviorof the method is checked against the requirements specifiedin the pattern contract (lines 9–10). The assert statementrequires that s.obs be unchanged, and that both o1 and o2

be Consistent() with s.In generating test cases for a particular system from this

PTCT, the s.other() call will have to be replaced by calls toappropriate methods of the class playing the Subject role.Some of these methods may require additional arguments. Inthe next section, we will consider the problem of generatingsuitable values for these arguments. At this point, however,the more important issue is the mismatch between the assert

statement and the requirements specified in the Observer con-tract. According to the contract (Fig. 2), when s.other()

terminates, either the state of s must not be Modified(), orthe Notify() method must have been invoked. As we havealready seen, this method will in turn invoke Update() oneach attached observer, bringing the objects into states con-sistent with the new state of the subject. Hence, the assert

included in the PTCT is a test of the expected net behaviorof the participating objects, i.e., it is a more “global” test ofthe system. There is, however, a risk in using such a PTCT,especially if these were the only ones used. If, for instance,

123


an other method were to modify the state of the subject,neglect a call to Notify(), but by chance leave s in a stateConsistent() with the states of o1 and o2, the design defectwould go undetected.

In [27], we present a scenario in which this problem mani-fests itself during system evolution; we summarize the exam-ple here. A class S1 plays the Subject role, and provides twofields, f1 and f2. A change in either field is considered to bea modification of the subject according to the definition ofModified() supplied in the system’s subcontract. A class O1

plays the Observer role. In an initial version of the system,O1 is interested only in the value of f1; changes in f2 areignored. The Update() method of O1 uses an appropriategetter method to retrieve the value of f1, and then updatesthe observer’s state to become Consistent() with the state ofthe subject. The Consistent() concept is defined suitably inthe subcontract. The S1 class includes a bug that omits a callto Notify() when a change in f2 occurs. A PTCT similar tothe one above will not detect this defect since each observer

will remain Consistent() with the subject when the other

method terminates — even if f2 has changed without a cor-responding call to Notify().

During system evolution, O1 is modified so that the valueof f2 becomes significant. Instances of O1 record the cur-rent value of f2 associated with their corresponding subject.Hence, the definition of Consistent() is suitably modified, asis the implementation of O1’s Update() method. The newimplementation retrieves the values of both f1 and f2, andupdates the state of theobserver appropriately. When the testcase derived from the PTCT shown in Fig. 5 is executed, theassert statement will generate an error; the clause involv-ing the Consistent() concept will be violated. The naturalassumption is that the fault lies in an area affected by sys-tem evolution. In fact the defect lies in the original S1 class— in particular, the failure of its other() method to invokeNotify() when the value of f2 changes.

The solution is to revise the PTCT to fully test the interac-tion requirements specified in the pattern contract, rather thantesting for net effects. More precisely, the last two clauses ofthe assert in Fig. 5 (line 10) should be replaced with thefollowing conditions:

8 assert( … clauses from Fig. 5 …9 && (! Modified(pre_s ,s) ||

10 ((t1.length ()==1)&&( t1.m()==" Notify "))

Before concluding this section, we will consider anotherPTCT, this one for the Composite pattern. Composite is astructural pattern [11] intended to compose objects into atree structure to represent part-whole hierarchies and to allowclients to treat individual objects and composite objects in auniform manner. The pattern has three roles, Component,Leaf, and Composite. Component is an abstract role withLeaf and Composite being the two kinds of components.

Fig. 6 Composite role contract (partial)

Patterns such as Composite, although classified as structural,also have behavioral aspects to them. Indeed, the main pur-pose of the Composite pattern is to ensure that each operationof a composite object is implemented by invoking the cor-responding operation on each of its components and com-bining the results returned by these invocations appropriatelyto obtain the result of the original operation invocation.

A small portion of the Composite pattern contract, in par-ticular a part of the Composite role contract, appears inFig. 6. The specification of any operation() of this rolestates that when this method finishes, the set of childrenis unchanged; that during the execution of this method, thereshould have been a call to the corresponding method on eachchild c; and that the result returned by this method should beequal to the value obtained by appropriately combining theresults of these method calls invoked on the children. Sincethe details of precisely how these results are combined toobtain the result of the Operation() applied to the com-

posite will vary from one application designed using thispattern to another, we use an auxiliary concept, Glue() torepresent this. Thus the final clause of the post-conditionof Operation() requires that the result returned bythe original call is equal to the value of Glue() applied tothe sequence of results in the method calls recorded in τ ,this being obtained by means of the helper function,resultSeq().

Figure 7 presents a PTCT to test that the requirementsspecified in Fig. 6 are satisfied. We create two composites,c1andc2, a leafl, and create a suitable composite structure. Wethen invokeoperation()on the top level composite struc-ture and, assert, after the call returns that the trace createdduring the execution of this method is equal to the number ofchildren of this structure, and that the result returned matchesthat obtained by applying the auxiliary concept Glue() onthe results returned by the calls recorded in that trace. Thedefinition of Glue() will, of course, be given in the sub-contract corresponding to a particular system, for which wewill specialize this PTCT to produce actual test cases. It maybe worth noting that this PTCT does not completely test thespecification in Fig. 6; in particular, while it does test that the

123


Fig. 7 PTCT for composite.operation())

number of method calls recorded in the trace t1 is what itshould be according to the specification, it does not test whichmethods were invoked in these calls, nor on what objects. It isstraightforward to modify the PTCT to include these checksand we omit the details.

4 Generating test cases

Given PTCTs such as those in the last section, how do wegenerate actual test cases corresponding to a given system Sdesigned using a particular pattern? Consider, for example,line 7 of Fig. 4. The methodAttach() that is being invokedin this line may have an entirely different name in S. Indeed,even the classes playing the Subject and Observer roleswill have their own names appropriate for the application.To illustrate these and other issues involved in generatingactual test cases, we will use a simple Hospital Simulationsystem, parts of which appear in Fig. 8, designed using theObserver pattern.

The Hospital system consists of three main classes,Patient, Nurse and Doctor. Instances of the Patient classplay the Subject role; instances of the other two play theObserver role. Zero or one doctor object and zero or morenurse objects are assigned to observe patient. The variables_nurses and _doctor in the Patient class are used to keeptrack of the assigned doctor/nurses. A nurse object tracksthe temperature (_temp) of the patient it is observing; adoctor object tracks the heart-rate (_hrtRt). The checkVi-

tals() method updates _temp and _hrtRt as needed, andinvokes notify() if needed. The Nurse and Doctor classesinclude an update() method to help keep track of the dataabout the patients being observed. They also provide a get-

Status()method that returns information about thepatients.Before considering how test cases corresponding to the

Hospital system may be generated from the PTCTs of thelast section, we have to consider the subcontract that specifieshow the Observer pattern is specialized in this system; themain part of this subcontract appears in Fig. 9.

Fig. 8 Hospital simulation code

The role map for Patient as Subject specifies how the_obs of this patient viewed as as subject may be obtainedfrom the state of the patient. Next, the method maps specifythat addNurse() and setDoctor play the role of Attach()

and that notify() plays the role of Notify(). The role mapfor Nurse as Observer illustrates a technical problem: sincea nurse may be observing multiple patients, each of whichis the subject in a distinct instance of Observer, how dowe identify the particular patient involved with the currentpattern instance? The contract formalism [12,26] providesthe notion of lead object to address this. The lead object ina pattern instance is the first object that enrolls in a patterninstance, i.e., the one that is in players[0]. The method mapfor Nurse maps its update() (applied to the lead object) toUpdate() of the Observer role. The role map for Doctor

as Observer is analogous.Next we have the definitions of the auxiliary concepts.

According to the definition of Modified() (lines 25–27), thestate of apatient is considered to be modified if either _temp

value or the _hrtRt value has changed. The definition in lines28–29 state that a nurse n’s state is consistent with that of thepatient p if n has the correct information about p._temp.The definition (lines 30–31) of consistency of a doctor d

and patient p is similar.

123


Fig. 9 Hospital-observer subcontract

Let us now turn to the main task of this section, generat-ing, from the PTCTs for a pattern, test cases for a particu-lar system designed using the pattern. For any such PTCT,a (large) number of test cases may be generated. Each testcase would be obtained by replacing each role object in thePTCT, such as s, o1, and o2 in the case of the PTCT inFig. 5, by corresponding application objects, as determinedby the subcontract, such as a patient, a nurse, and a doc-

tor, respectively, in the case of the Hospital system; replacingthe calls to role methods in the PTCT, such as s.Attach(o1)

and s.Attach(o2), by calls to the appropriate applicationmethods, again as determined by the subcontract, such ass.addNurse(o1) and s.setDoctor(o2), respectively; etc.

While that much is straightforward, there are some impor-tant additional issues to consider. For example, it may be thatthe constructor functions (or other methods, for that matter)of some of the application classes require additional argu-ments. What values do we supply for these? The PTCT, ofcourse, imposes no requirements on these. As far as the PTCTand the pattern are concerned, any values may be used forthese additional arguments. This means that the number ofpossible test cases we can generate will be very large. Whichof these test cases will actually be useful will depend onthe system, in particular its implementation details. Hencethe human tester using our approach will be required to

suitably specialize the PTCT by defining appropriate test casespecializers.

A key purpose of a test case specializer (TCS) is to limitthe set of generated test cases, i.e., to specialize the PTCTin such a manner that the test cases that are generated arethe ones most appropriate to the system under test. Thereare a number of dimensions in which this specialization hasto be performed. First, the objects constructed in setting upthe scenario for a PTCT are role objects. If R is a role ofthis pattern and, in the PTCT, we construct an object ob oftype R, and if, according to the subcontract for the systemin question the class C1 and C2 play the role R, then in anactual test case, ob may be either a C1 object or a C2 object;the TCS has to specify which it is.

Second, calls in the PTCT to named role methods (such asAttach()) have to be replaced by calls to the correspondingclass methods, as specified in the appropriate role maps ofthe subcontract. This process, by itself, does not require anyadditional information from the TCS – unless more than oneclass method plays the role method in question or, as is morefrequently the case, the class method in question requiresadditional arguments beyond the ones expected by the rolemethod. These values will, of course, have to be provided bythe TCS. There are two important considerations to accountfor here. First, the application class may specify, as part ofthe pre-condition of the corresponding class method, someconditions that must be satisfied by these argument values. Ifsome set of argument values generated according to the spe-cialization in the TCS do not satisfy these conditions, thattest case cannot be used. For example, in the Hospital sys-tem, we are not allowed to attach two doctor objects at thesame time to a given patient object. In other words, anattempt to call addDoctor() on a patient object thatalready has a doctor will violate the pre-condition of thismethod and hence the method may behave in whatever wayit chooses and this is not a bug in the system, despite the factthat, because of such behavior, an assert in the test casemay fail. At the same time, we cannot always eliminate thistype of problem when the test case is generated because someof these assertions may be checkable only during executionof the test case. In order to address this type of problem,JDUnit inserts additional asserts in the test cases it gen-erates immediately prior to calls to any methods. Each ofthese asserts checks that the pre-condition of the methodbeing called is satisfied. If, when a test case is executed,such an assertion is not satisfied, that is recorded in the logmaintained by the system; but the assertion failure does notindicate a failure of the test case8. The second issue that the

8 This is a distinction from the JUnit framework where every assertionviolation is reported as a failure. In order to allow a different treat-ment for pre-condition violations, in our design of JDUnit, we use anadditional method, preAssert, to specify such assertions.

123


tester designing the TCS has to be concerned about in decid-ing on the argument values is producing a large enough set oftest cases with different values for these arguments to ensureadequate coverage of the system under test.

Next, each other() call that appears in the PTCT mustbe converted to a call to one of the methods of the appro-priate application class, other than the ones mapped to thevarious named role methods. In addition, as in the case con-sidered above, if the method in question requires additionalarguments, the TCS must provide these as well. And, as inthe case considered above, we must account for any pre-conditions that may be imposed by the application class inquestion.

The final dimension of specialization of the PTCT that theTCS must account for is concerned with a construct that mayappear in a PTCT but, as it turns out, was not not used inany of the PTCTs in the last section. This is the “other*();”construct, which denotes a sequence of zero or more callsto other methods. This construct may appear in a PTCTat points where the PTCT designer wants to allow for thepossibility that a number of additional calls to other methodsmay be needed in order to get the various objects into suitablestates for continuing the test. The TCS will have to specifythe sequence of method calls that must be made in place ofthe other*() construct.

Let us now see how these considerations play out whenapplied to some of the PTCTs (for the Observer pattern) whenwe specialize them to test the Hospital system. Consider thePTCT in Fig. 4. Line 2 of this PTCT requires an observer

object to be created. Suppose the TCS designer decides thatthis object should be a doctor. In the notation for TCS thatwe are designing concomitantly with the design of JDUnit,this may be expressed as:

8 Observer () {@2} = Doctor(/*args*/);

On the other hand, given that both Doctor and Nurse playthe Observer role in this application, the TCS designer maywant to create two test cases, one corresponding to each ofthese possibilities. This is expressed in our notation as:

8 Observer () {@2} = Doctor(/*args*/)9 || Nurse(/*args*/);

Let us consider another statement that appears in the samePTCT, the call to Attach() in line 7 of the PTCT in Fig. 4.The following specializes that call so that it is replaced by acall to the setDoctor() method:

8 Attach(o) {@7} =9 {setDoctor(o,/* additional args*/)}

If, as we considered above, the observer that was con-structed earlier (line 2 of the PTCT) could be either a doctor

or a nurse, our notation for TCS allows the tester to spec-ify which method must be used when replacing the call toAttach() in line 7:

8 Attach(o) {@7} =

9 {Doctor(o): setDoctor(o,/* addnl args*/)10 ||Nurse(o): addNurse(o,/* addnl args*/)};

Strictly speaking, it should not be necessary for the tester toprovide this information since it is available from the sub-contract (Fig. 9). But in those cases where multiple methodsof the same class may play the role of a named method, theTCS will have to specify which of those must be used. Thisis, of course, very much needed in dealing with other() callsthat appear in the PTCT since, in general, there will be manyclass methods that qualify as other methods. In the case ofother*() calls, we must provide the sequence of methodcalls that should replace it and the arguments for each callin the sequence. For example, suppose the other() call inline 5 of the PTCT in Fig. 5 was an other*() call. Then wemay have the following:

8 other *() {@7} =9 {checkVitals (); checkVitals ();}

Here the other*() call is replaced by a sequence of two callsto checkVitals(). We can also use the “||” notation to allowfor alternative sequences of calls.

The detailed syntax for TCS is very much a work inprogress. For example, in the situation we just considered,if the tester wanted to test a range of different sequencesof other calls, listing them all explicitly would be unde-sirable. It would be more convenient if we could use regu-lar expression-like notations for such situations. We intendto apply our approach to a number of patterns and systemsdesigned using the patterns to determine the exact notationalfacilities that TCS must include. In addition, practitioner-friendliness will be a key consideration since it is the testerswho are part of individual software teams who will be respon-sible for defining the TCSs for their particular systems.

Before we conclude this section, one final point about theHospital system is worth briefly discussing. Suppose, dur-ing system evolution, it is decided that the nurse objectsobserving a given patient object should keep track also ofthe medicine-level, i.e., the value of the _medLvl variableof the Patient class. It would be easy enough to modify theNurse class to do so, perhaps by introducing a new variable,_medLevels, similar to the Nurse._vitals; and to reviseNurse.update() so that the value of _medLevels as wellas _vitals is appropriately updated.

While this seems simple and reasonable, if this is all wedo, there will be a bug in this system! The problem is that, asit is implemented, the Patient class methods do not invokenotify() if only the _medLvl value changes. So in those sit-uations where only this variable changes, the attached nurse

objects will have incorrect information about medLvl (untilthe next time the patient’s _temp or _hrtRt changes, trig-gering a call to notify()).

123


Unfortunately, however, the test cases generated using thePTCTs we have seen will not catch this error. The problemis that, according to the definition of the auxiliary conceptsin Fig. 6, if only the value of _medLvl changes, the stateof the patient is not considered to be modified; moreover,the state of nurse is considered to be consistent with thatof the patient as long as the former has the correct infor-mation about the value of the _temp of the latter. Thus anytest cases generated from the PTCT in Fig. 5 will succeedwhen executed. If we were to change the definition of oneof these concepts but not the other, the constraint specifiedin the original pattern contract (lines 6–8 of Fig. 2) will beviolated. Since such a problem could easily arise during sys-tem evolution, it is important to ensure that our PTCTs checkthat constraints on the auxiliary concepts are satisfied. At thesame time, the fact that these constraints are typically uni-versally quantified over the states of various roles makes thisa challenge. This is one of the questions we intend to addressin future work.

5 Related work

The PTCT testing approach builds upon our prior work inpattern formalization [12,26,31]. The key conceptual ele-ments of this work were summarized in Section 2. Severalother groups have also considered improving the precision ofpattern descriptions. Some have described techniques basedon diagrammatic notations that extend standard UML(-like)syntax [7,21,32]. Closer to our specification approach is thatof Eden et al. [9,10]; the authors describe an approach to cap-turing the structural properties of patterns using a logic nota-tion. Patterns are expressed as logic formulae that specifyparticipating classes, methods, inheritance hierarchies, andthe structural relations among them. Mikkonen [19] describesa behavioral specification approach based on DisCo [14], aUNITY-like [5] action system notation. Data tuples representobjects, and guarded actions model their interactions. Helmet al. [13] describe a contract formalism that bears similarityto ours. In particular, the formalism has constructs analogousto our role contracts, auxiliary concepts, and a limited form oftrace conditions. The formalism additionally provides somesupport for contract specialization and system-level reason-ing. However, despite the effort invested in documenting pat-terns precisely, there appears to be relatively little prior workin testing the correctness of systems constructed accordingto particular pattern descriptions, either formal or informal.By contrast, this has been the focus of our work.

Some readers may take note of the fact that the phrase“test(ing) patterns” appears throughout the literature.Indeed, there is at least one workshop series devoted to testingpatterns [28–30], a corresponding online repository [16], andwidely referenced books that detail the subject [3,18]. So how

can the work described here be new? The standard meaning oftesting pattern refers to a pattern used to address a recurringproblem in the context of software testing. Indeed, a test(ing)pattern is simply a special type of design pattern [17], oneintended to guide the design of testing code. Our work, how-ever, is focused on validating the correct usage of designpatterns through software testing. We note that other authorshave also considered applying design patterns as part of thesystem design process to improve testability [1,6]. Again,however, our focus has been on ensuring the correct applica-tion of patterns.

It is important to note that McGregor [17,18] extends thetraditional notion of a test(ing) pattern to patterns intendedfor use in validating design pattern implementations. Theconcept is similar to our use of PTCTs in that each testpattern specifies a particular scenario of application objectsthat interact in ways intended to check pattern requirements.However, the scenarios are not based on precise pattern spec-ifications. Equally important, McGregor does not considertest suite specialization, nor automated generation. Still, thesimilarities are important.

Finally, it may be useful to mention the SABER systemdescribed by Reimer et al. [22]. The system is designed todetect defects in large Java applications using static analy-sis; SABER operates on systems designed using particularframeworks. Correct use of these frameworks requires thatcertain methods be invoked in certain orders, that others notbe invoked at certain points, etc. SABER attempts to iden-tify violations of such requirements based on call sequencerules specified using an XML dialect. Given the importanceof call sequence requirements in pattern-based systems,elements of SABER may be useful in detecting patternviolations statically, prior to pattern testing. We intend toexplore this possibility as part of future work.

6 Conclusion

We began with the observation that design patterns playa central role in software practice, influencing the designof most major software systems and class libraries. Conse-quently, techniques for ensuring the correctness of patternimplementations are likely to have a profound impact on thereliability of real-world systems. To this end, we focusedon an approach to testing pattern implementations basedon precise pattern specifications. Whereas our prior workprovides the specification foundation, the work reportedhere provides the foundation for specification-based testingof pattern implementations.

In developing the testing approach, we observed that pat-tern implementations follow a similar structure. This is notsurprising. After all, for a designer to claim that a particu-lar group of classes have been designed according to a given

123


pattern, she must have satisfied certain requirements—namely, those common to all applications of the pattern inquestion. It then follows that the test cases used to vali-date the pattern’s implementation share a common structureacross systems designed using the pattern. The central moti-vation of our testing approach is that this common structure,applicable to all systems designed using the pattern, shouldbe reusable. In defining the test cases for a particular system,testers should only be required to invest the incremental effortnecessary to specialize the generalized structure as appropri-ate to the system under test. Moreover, the ability to definereusable test case structures enables the software engineer-ing community to encode test case families that are likely toreveal the most common pattern implementation errors.

We described a testing approach that satisfies these goals,and the design of a software tool used to assist in test case gen-eration. In our approach, pattern test case templates (PTCTs)are used to define generalized test case structures reusableacross applications of particular patterns. Each template isexpressed using a Java-like syntax amenable to practitioneradoption; the code is written in terms of the role types and rolemethods specified by the corresponding pattern specification.To test the behavior of a system implemented using a partic-ular pattern, the corresponding PTCTs must be instantiatedto generate executable test cases. The instantiation processis guided by a specification of the mappings between pat-tern roles and system classes, as well as rules that governsystem-specific specializations. JDUnit assists in the instan-tiation process by guiding developers in selecting the appro-priate PTCTs and pattern specifications, and in introducingapplication-specific objects, method calls, and arguments.The output of the tool is a set of JUnit test cases used tovalidate the correctness of the relevant patterns as applied inthe system under test9.

We conclude with a short comment on the relationshipbetween our approach to pattern-centric testing (PCT), andtest-driven-development (TDD) [2]. In TDD, test cases thatcapture the expected behavior of a component are writtenbefore the component is implemented. Hence, TDD dictateswhen test cases must be designed. On the other hand, PCT isconcerned with what must be tested. One could, as dictated byTDD, develop our test cases before a system is implemented.(Of course, relevant portions of the design must be completesince otherwise we would not know which PTCTs to instan-tiate.) Alternatively, following a more conventional process,we may design and implement the system first, and then,

9 A key guideline for using the JUnit framework is that the tests shouldbe designed to test behavior rather than individual methods [20]. Thisalso applies to our approach, except that the behaviors in question arethose corresponding to the use of a particular pattern. Hence, the PTCTstypically involve multiple roles. Consequently, the instantiated test casesinvolve multiple system classes, by contrast to more traditional unittests.

during testing, use PCT to generate appropriate test cases.Hence, PCT and TDD are fully compatible, but orthogonaltesting principles.

References

1. Baudry B, Le Sunyé Y, Jézéquel J (2001) Towards a ‘safe’ use ofdesign patterns to improve OO software testability. In: The 12thinternational symposium on software reliability engineering, IEEEComputer Society, Washington, DC, pp 324–329

2. Beck K, Gamma E (1998) Test infected: programmers love writingtests. Java Rep 3(7): 37–50

3. Binder R (1999) Testing object-oriented systems. Addison-Wesley, Menlo Park

4. Buschmann F, Meunier R, Rohnert H, Sommerlad P, Stal M(1996) Pattern-oriented software architecture: a system of patterns.Wiley, New York

5. Chandy K, Misra J (1988) Parallel program design. Addison-Wesley, Menlo Park

6. Dasiewicz P (2005) Design patterns and object-oriented softwaretesting. In: The 2005 Canadian conference on electrical and com-puter engineering, IEEE Canada, Dundas

7. Dong J (2002) UML extenstions for design pattern compositions.In: Mingins C (ed) Proceedings of TOOLS, in special issue ofjournal of object technology, vol 1, issue 3, pp 149–161

8. Dong J, Alencar P, Cowan D (2001) A behavioral analysisapproach to pattern-based composition. In: Proceedings of the 7thinternational conference on object-oriented information systems,Springer, pp 540–549

9. Eden A (2001) Formal specification of object-oriented design. In:Proceedings of the international conference on multidisciplinarydesign in engineering

10. Eden A (2002) LePUS: a visual formalism for object-orientedarchitectures. In: Proceedings of the 6th world conference on inte-grated design and process technology, IEEE Computer Society,pp 149–159

11. Gamma E, Helm R, Johnson R, Vlissides J (1995) Design patterns:elements of Reusable OO Software. Addison-Wesley, Menlo Park

12. Hallstrom J, Soundarajan N, Tyler B (2006) Amplifying the ben-efits of design patterns. In: Aagedal J, Baresi L (eds) Proceedingsof the 9th international conference on fundamental approaches tosoftware engineering (FASE), Springer, pp 214–229

13. Helm R, Holland I, Gangopadhyay D (1990) Contracts: specifyingbehavioral compositions in object-oriented systems. In: OOPSLA-ECOOP, pp 169–180

14. Järvinen H, Kurki-Suonio R (1991) Disco specification language:marriage of actions and objects. In: Proceedings of the 11th inter-national conference on distributed computing systems, IEEE Com-puter Society, Los Alamitos, pp 142–151

15. Jones C (1990) Systematic software development using VDM.Prentice-Hall, Englewood Cliffs, New York

16. Marick B (2007 (date of last access)) Test patterns repository/software testing patterns. http://www.testing.com/test-patterns/patterns/

17. McGregor J (1999) Testpatterns: please stand by. J Object-orientedProgram 12:14–19

18. McGregor J, Sykes D (2001) A practical guide to testing object-oriented software. Addison-Wesley, Menlo Park

19. Mikkonen T (1998) Formalizing design patterns. In: Proceedingsof 20th ICSE, IEEE Computer Society Press, pp 115–124

20. Rainsberger J (2005) JUnit recipes. Manning21. Reenskaug T (1996) Working with objects. Prentice-Hall, Engle-

wood Cliffs

123

http://www.testing.com/test-patterns/patterns/

http://www.testing.com/test-patterns/patterns/


22. Reimer D, Schonberg E, Srinivas K, Srinivasan H, Alpern B,Johnson R, Kershenbaum A, Koved L (2004) “saber”: smart analy-sis based error reduction. In: Proceedings of “ISSTA ’04”, ACMPress, pp 243–251

23. Riehle D (1997) Composite design patterns. In: Proceedings ofOOPSLA, ACM, pp 218–228

24. Schmidt D, Stal M, Rohnert H, Buschmann F (1996) Pattern-oriented software architecture: patterns for concurrent and net-worked objects. Wiley, New York

25. Shalloway A, Trott J (2002) Design patterns explained. Addison-Wesley, Menlo Park

26. Soundarajan N, Hallstrom J (2004) Responsibilities and rewards:specifying design patterns. In: Finkelstein A, Estublier J,Rosenblum D (eds) Proceedings of 26th international conferenceon software engineering (ICSE), IEEE Computer Society, pp 666–675

27. Soundarajan N, Hallstrom J (2006) Pattern-based system evolu-tion: a case-study. In: Zhang K, Spanoudakis G, Visaggio G (eds)Proceedings of 18th international conference on software engineer-ing and knowledge engineering (SEKE 2006), Knowledge SystemsInstitute, pp 321–326

28. Testing (2001) Patterns of software testing 129. Testing (2001) Patterns of software testing 230. Testing (2001) Patterns of software testing 331. Tyler B, Hallstrom J, Soundarajan N (2006) A comparative study of

monitoring tools for pattern-centric behavior. In: Hinchey M (ed)Procedings of 30th IEEE/NASA sofware engineering workshop(SEW-30), IEEE-Computer Society

32. Vlissides J (1998) Notation, notation, notation. C++ Report

123

patterns: from system design to software testing

Documents