1 levi lúcio © a test selection language for co-opn specifications levi lúcio, luis pedro and...
TRANSCRIPT
1 Levi Lúcio©
A Test Selection Language for CO-OPN Specifications
Levi Lúcio, Luis Pedro
and Didier Buchs
University of Geneva
2 Levi Lúcio©
Presentation Plan
The CO-OPN Modeling Language CO-OPN as a prototyping tool
Testing Theory Stating hypotheses about the SUT for test selection
A Test Selection Language for CO-OPN Specifications Language Structure Hypotheses
• On Paths
• On Values
Conclusion
3 Levi Lúcio©
Presentation Plan
The CO-OPN Modeling Language CO-OPN as a prototyping tool
Testing Theory Stating hypotheses about the SUT for test selection
A Test Selection Language for CO-OPN Specifications Language Structure Hypotheses
• On Paths
• On Values
Conclusion
4 Levi Lúcio©
The CO-OPN Modeling Language
CO-OPN (Concurrent Object-Oriented Petri Nets) is a formal specification language, with clearly stated semantics. Based on:
Algebraic Data Types: data and associated operations Petri Nets: behavior
It includes object based concepts: Class
• Attributes: places in a Petri Net
• Methods: guarded transitions in a Petri Net
Powerful synchronization mechanism Coordination of the components of a specification
• Objects
• Contexts (stateless coordination units) “All or nothing” semantics (sequence / parallel / choice)
5 Levi Lúcio©
Structure of a CO-OPN Specification
Contexts
(instances of) Classes
Algebraic Data Types
Coordinate
Use
Coordinate
Coordinate
6 Levi Lúcio©
Example specification in CO-OPN
A (simple) mobile phone is composed by: A card containing (among other things) the personal
identification number (PIN) to activate the phone; A phone controller holding (among other things) the state of
the phone, which can be:• Off
• Standby (waiting for the correct PIN)
• Idle (ready to receive or make calls)
In our specification the user can perform two actions: Turn on the phone; Insert a PIN number
7 Levi Lúcio©
Outer ContextCoordinates the Phone Controller and the Card
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
gate (output event)
method(inputevent)
8 Levi Lúcio©
Class PhoneController
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Algebraic Data TypesTyped values (+ operations) inside the Petri Net places…
9 Levi Lúcio©
CO-OPN as a Prototyping Tool
Our tool (CoopnBuilder) is able to produce Java code from a CO-OPN specification. We can then execute the specification…
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
10 Levi Lúcio©
SUT (System Under Test) Verification
We can “run” the specification to make sure the prototype will act as expected, but…
What if the prototype is changed / extended, or simply implemented in a language other than Java?
There is no way of keeping a record of previously tested behaviors (for example for regression testing…)
Solution: devise a testing language to include test modules in CO-OPN specifications:
The test module will use the specification to generate syntactically and semantically meaningful tests;
Prototype execution provides an easy way of deciding what should be the result of a given test case when it is applied to the SUT (System Under Test).
11 Levi Lúcio©
Presentation Plan
The CO-OPN Modeling Language CO-OPN as a prototyping tool
Testing Theory Stating hypotheses about the SUT for test selection
A Test Selection Language for CO-OPN Specifications Language Structure Hypotheses
• On Paths
• On Values
Conclusion
12 Levi Lúcio©
Presentation Plan
The CO-OPN Modeling Language CO-OPN as a prototyping tool
Testing Theory Stating hypotheses about the SUT for test selection
A Test Selection Language for CO-OPN Specifications Language Structure Hypotheses
• On Paths
• On Values
Conclusion
13 Levi Lúcio©
Testing Theory
Problem: How do we select “interesting” tests? In the general case, the amount of tests for a given system
is infinite: Infinite domains for operation’s domains; Infinite paths of execution in the SUT’s state space.
It is necessary to restrict the “exhaustive” test set to a practicable one. This is done by providing hypotheses about the functioning of the SUT:
Reducetestset
Increasehypothesis
strength
H0
H1…
Hk-1Hk
T0
T1…
Tk-1Tk
14 Levi Lúcio©
Testing Process
Does P satisfy SP?
P SP
Test Selection(Hypotheses H Application)
Execution of P using T
OracleP satisfies, or not, T!
P does not satisfy SP Undefined P satisfies SP!
P satisfies, or not, H
Test Set T
no inconclusive yes
no yes
Correction of P
TestProcedure
15 Levi Lúcio©
Presentation Plan
The CO-OPN Modeling Language CO-OPN as a prototyping tool
Testing Theory Stating hypotheses about the SUT for test selection
A Test Selection Language for CO-OPN Specifications Language Structure Hypotheses
• On Paths
• On Values
Conclusion
16 Levi Lúcio©
Presentation Plan
The CO-OPN Modeling Language CO-OPN as a prototyping tool
Testing Theory Stating hypotheses about the SUT for test selection
A Test Selection Language for CO-OPN Specifications Language Structure Hypotheses
• On Paths
• On Values
Conclusion
17 Levi Lúcio©
Test Language Structure
CO-OPN event pairsinput event (method), output event (gate)
(black-box approach)
HML (Hennessy-Milner Logic)next <_> operatornot, and operators
Hypothesesconstraints over theexhaustive test set
18 Levi Lúcio©
Mobile Phone Example Revisited
Let us create a test module for an extended version of the Mobile phone CO-OPN specification:
A (simple) mobile phone is composed by:• A card containing (among other things) the personal
identification number (PIN) to activate the phone. The card gets blocked after 3 wrong attempts at inserting a PIN.
• A phone controller holding (among other things) the state of the phone, which can be:
– Off
– Standby (waiting for the correct PIN)
– Idle (ready to receive or make calls) In our specification the user can perform two actions:
• Turn on the phone;
• Insert a PIN number,
• Reset the card when it becomes blocked
19 Levi Lúcio©
ConstraintSet NatelBehaviors;
Interface
Constraints insertPins; block; reachOn;
Body
Constraints nWrongPins;
Use Boolean; Pin;
Axiom [] in nWrongPins; f in nWrongPins => f . HML(<insertPin(newPin(1 1 1 1)),pinResult(false)>) in nWrongPins; f in nWrongPins, depth(f) <= 3 => HML(<turnOn,null>) . f . HML(<m,g>) in insertPins; depth(f) <= 3 => HML(<turnOn,null>) . f . HML(<resetCard,null>) in block; depth(f) <= 3 => HML(<turnOn,null>) . f . HML(<insertPin(newPin(1 2 3 4)),phoneReady>) in reachOn;
Variables f : HML; m : method; g : gate;
End NatelBehaviors;
Test Module for the Mobile Phone
Exported constraints
Locally defined constraints
Imported Algebraic Data Types
Variables used in constraint definition Constraints’Semantics
20 Levi Lúcio©
Hypotheses on Paths
The language allows defining the shape of sequences of input/output pairs that will be used to generate tests:
[] in nWrongPins;f in nWrongPins => f . HML(<insertPin(newPin(1 1 1 1)),pinResult(false)>) in nWrongPins;f in nWrongPins, depth(f) <= 3 => HML(<turnOn,null>)} . f . HML(<m,g>) in insertPins;
nWrongPins (local constraint) defines an (infinite) sequence of insertPin inputs with a wrong PIN;
insertPins defines a sequence of operations starting by turnOn, followed by a sequence of at most three insertPin inputs with a wrong PIN and finishing by any input/output pair.
HML(<turnOn,null>,<insertPin(newPin(1 1 1 1)),pinResult(false)>,<resetCard,null>),falseHML(<turnOn,null>,<insertPin(newPin(1 1 1 1)),pinResult(false)>, <insertPin(newPin(1 1 1 1)), pinResult(false)>,<insertPin(newPin(1 1 1 1),pinResult(false)>,<resetCard,null>),trueHML(<turnOn,null>,<insertPin(newPin(1 2 3 4)),pinResult(true)>,true...
Examples of tests generated by the insertPin constraint:
21 Levi Lúcio©
Hypotheses on Values
By default, the tests generated by the pinTest will correspond to all values of type Pin - large amount
Solution 1:
Axioms HML(<turnOn,null>,<insertPin(p), g>) in pinTest;
Variables p : Pin; g : gate;
uniformity(p) => HML(<turnOn,null>,<insertPin(p), g>) in pinTest
Choose one value from the domain - equivalent to random testing…
Corresponds to the hypotheses: “if insertPin works well for one PIN, it will work well for all PINs”.
22 Levi Lúcio©
Hypotheses on Values (2)
Solution 2:
subUniformity(p) => HML(<turnOn,null>,<insertPin(p), g>) in pinTest
Choose one value from each equivalence class of the behavior of the insertPin operation.
insertPin(p)P is the good PIN => Send pinResult(true)
P is a wrong PIN => Send pinResult(false)
In order to decompose the behavior of an operation we need to analyze its choice points…
This is done by analyzing the CO-OPN specification. Feature under development, using Prolog for symbolic execution and constraint
resolution.
23 Levi Lúcio©
Managing Non-Determinism in the SUT
We would like to manage behaviors of the SUT which cannot be calculated in test generation time…
Example: an e-banking application proposes a challenge as part of the authentication mechanism. How do we calculate the reply to the challenge in order to login successfully during the test?
Solution: Introduce the possibility to have “external” functions that will perform calculations in testing time.
In the e-banking case there would be a function to calculate the answer to the challenge in function of the proposed value.
24 Levi Lúcio©
Conclusion
We have presented a test selection language intended at testing software (and possibly hardware) prototypes specified in CO-OPN;
The test selection is done by the test engineer, assisted by the tool while decomposing the behavior of the operations in a specification;
The tool is currently half implemented. We envisage on the short term a case study to fully validate our ideas, also in terms of scalability.