![Page 1: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/1.jpg)
LSRTOBIAS
and some ASE challenges
Y. Ledru
P. Bontron, O. Maury,
L. du Bousquet, M.L. Potet,
C. Oriat , S. Beghdadi, H. Bouldjedri
LSR/IMAG – Grenoble
ASE Workshop at Irvine, June 2002
![Page 2: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/2.jpg)
LSR
Overview• Two challenges of ASE
• TOBIAS
• How TOBIAS is confronted to these challenges
![Page 3: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/3.jpg)
LSR
OverviewTwo challenges of ASE
• TOBIAS
• How TOBIAS is confronted to these challenges
![Page 4: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/4.jpg)
LSR
Signature of ASE tools
Formal or structured document
Formal or structured document
ASEtool
Source code, tests,formal specifications,
UML diagrams,XML documents
![Page 5: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/5.jpg)
LSR
1st challenge: process documents
• Complexity of input/output documents• Algorithmic complexity of processing:
– Combinatorial explosion– Undecidable problems
Formal or structured document
Formal or structured document
ASEtool
Challenge 1 : Process
documents
Domain specific
tools
![Page 6: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/6.jpg)
LSR
2nd challenge: produce documents
• Software engineers only like to write code! [XP]• Millions of amateur software engineers…• … how can we learn them category theory?
Formal or structured document
Formal or structured document
ASEtool
Captureinformation
Challenge 1 : Process
documents
Challenge 2 : Produce
documents
![Page 7: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/7.jpg)
LSR
Elements of solutions
• Don’t underestimate software engineers: provide useful/efficient/scalable tools!• Provide easier formalisms (graphical formalisms, GUI tools,…)• Integrate your tools in the process of the company!
Formal or structured document
Formal or structured document
ASEtool
Captureinformation
Challenge 1 : Process
documents
Challenge 2 : Produce
documents
costs vs b
enefits
![Page 8: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/8.jpg)
LSR
OverviewTwo challenges of ASE
TOBIAS
• How TOBIAS is confronted to these challenges
![Page 9: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/9.jpg)
LSR
TOBIAS
• Generates a large number of tests…• … from a generative pattern (test schema)• Should be used in combination with tools that
provide the test oracle (VDMTools, TGV).
Test schemas
Test data/Test cases/
Test purposesTOBIAS
Captureinformation
Class signatures
![Page 10: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/10.jpg)
LSR
A simple example
• An integer set classwith two operations
• « manual » tests:– Add(0)– Add(0),Add(1),Add(2)– Add(0),Add(0)– Add(0),Remove(0)– Add(0),Add(1),Remove(0)– Add(0),Remove(1)
Add(v:int)Remove(v:int)
IntegerSet
![Page 11: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/11.jpg)
LSR
Add(0)Add(1)Add(2)Add(3)
• Add(x)^1..3;Remove(y)^0..2• Where • x : {0,1,2,3}• y : {0,1,2}
Add(0),Add(0) Add(2),Add(0)Add(0),Add(1) Add(2),Add(1)Add(0),Add(2) Add(2),Add(2)Add(0),Add(3) Add(2),Add(3)
Add(1),Add(0) Add(3),Add(0)Add(1),Add(1) Add(3),Add(1)Add(1),Add(2) Add(3),Add(2)Add(1),Add(3) Add(3),Add(3)
A TOBIAS test schema
…Add(0),Add(1),Add(2)…Add(2),Add(1),Add(0)… Add(0),Remove(0)
Add(0),Remove(1)…
…Add(0),Add(1),Remove(0)…
• Add(x1)• Add(x1),Add(x2)• Add(x1),Add(x2),Add(x3)• Add(x1),Remove(y1)• Add(x1),Add(x2),Remove(y1)• Add(x1),Add(x2),Add(x3), Remove(y1)• Add(x1),Remove(y1),Remove(y2)• Add(x1),Add(x2),Remove(y1),Remove(y2)• Add(x1),Add(x2),Add(x3), Remove(y1),Remove(y2)
(4+4*4+4*4*4)*(1+3+3*3) = 1092 sequences
![Page 12: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/12.jpg)
LSR
Results of TOBIAS• TOBIAS generates the manual tests
+ lots of similar tests
for the same design cost
• A recent experiment:– 45 manual tests– 4320 TOBIAS tests– same test design effort– 2 new errors detectedTOBIAS helps detecting more errors!
![Page 13: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/13.jpg)
LSR
OverviewTwo challenges of ASE
TOBIAS
How TOBIAS is confronted to these challenges
![Page 14: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/14.jpg)
LSR
1st challenge complexity• Intrinsic to the tool:
– generates a large number of tests– because IUTs are complex
• Still too many tests are useless:– Require untractable resources– May not detect more errors
Keep the number of tests small enough:• Avoid redundant test cases
Extend the test schemas (constraints, hypotheses)
• Remove non-conform test cases Verify vs specification
![Page 15: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/15.jpg)
LSR
Test schemas
TOBIAS and complexity
Test data/Test cases/
Test purposesTOBIAS
Captureinformation
Class signaturesSolutions to the
first challenge…
… make the second one more complex!
ExtendedTest
schemas
Specification(UML diagrams,
Pre/post cond., …)
![Page 16: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/16.jpg)
LSR
Test schemas are simple and short !
TOBIAS provides a GUI for test schemas
Specificationscan be used in
conjunction with generated tests(oracle, TGV,…)
Specificationsare not neededto generate the first set of tests!
Class signaturescan be extracted
from existing documents
Capturing knowledge for TOBIAS
Test schemas
Test data/Test cases/
Test purposesTOBIAS
Captureinformation
Class signatures
Specification(UML diagrams,
Pre/post cond., …)
UML class diag.Or
Java Programextract
![Page 17: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/17.jpg)
LSR
Conclusion• TOBIAS is confronted to both ASE
challenges– complexity– information acquisition
• Complexity addressed by additional knowledge
• Information acquired progressively:– Provide first benefits at low cost– Motivate efforts for specification– Multiple use of specifications (oracle, TGV)– Fits into a classical process
![Page 18: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/18.jpg)
LSR
THANKS !
![Page 19: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/19.jpg)
LSR
Redundant test cases• The implementation of the integer set
may be independent from the actual values of integers
• Lots of tests generated by TOBIAS can be considered redundant
• This corresponds to the expression of test hypotheses.
Add(0)Add(1)Add(2)Add(3)
…Add(0),Add(1),Add(2)…Add(2),Add(1),Add(0)…
Add(0),Add(0) Add(2),Add(0)Add(0),Add(1) Add(2),Add(1)Add(0),Add(2) Add(2),Add(2)Add(0),Add(3) Add(2),Add(3)
Add(1),Add(0) Add(3),Add(0)Add(1),Add(1) Add(3),Add(1)Add(1),Add(2) Add(3),Add(2)Add(1),Add(3) Add(3),Add(3)
![Page 20: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/20.jpg)
LSR Add(x)^1..3;Remove(y)^0..2Where x : {0,1,2,3}y : {0,1,2}Forall x_i,x_j. i<j => x_i<=x_j
Add(0),Add(0) Add(2),Add(0)Add(0),Add(1) Add(2),Add(1)Add(0),Add(2) Add(2),Add(2)Add(0),Add(3) Add(2),Add(3)
Add(1),Add(0) Add(3),Add(0)Add(1),Add(1) Add(3),Add(1)Add(1),Add(2) Add(3),Add(2)Add(1),Add(3) Add(3),Add(3)
…Add(0),Add(1),Add(2)…Add(2),Add(1),Add(0)…
![Page 21: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/21.jpg)
LSR
Overview
• Context: the COTE project (COmponent TEsting)
• Why TOBIAS ?
• TOBIAS: description of the tool– Test purpose generation– Test case generation with VDM– Smarter generation
![Page 22: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/22.jpg)
LSR
Overview
• Context: the COTE project (COmponent TEsting)
• Why TOBIAS ?
• TOBIAS: description of the tool– Test purpose generation– Test case generation with VDM– Smarter generation
![Page 23: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/23.jpg)
LSR
The COTE project
• A RNTL Project
• Partners – SOFTEAM (tool objecteering) leader– IRISA (tools UMLAUT/TGV)
– GEMPLUS (smart cards/software components)
– FT R&D (OO technologies/components)
– LSR-IMAG (Software engineering)
![Page 24: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/24.jpg)
LSR
The context of COTE
• Conformance testing of software components– The component is tested against its UML
specification
– Black box testing: the internal behaviour of the product under test is hidden
![Page 25: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/25.jpg)
LSR
Aim of the COTE project
corba ejb .net
UMLTest casesAutomatic
generationof executable tests
• UML sequence diagrams provide an abstraction for tests cases
• provide tools for automatic executable test generation from UML test cases
• target three component technologies
![Page 26: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/26.jpg)
LSR
Aim of the COTE project
Testsynthesis
• Assist the user in the specification of tests
• UML specification of tests
corba ejb .net
UMLTest casesAutomatic
generationof executable tests
test purpose
![Page 27: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/27.jpg)
LSR
Test case / Test purposeAn example :
A coffee vending machine
!sugar
?coffee+sugar ?coffee+sugar
?coin?coin
!coffee
!sugar
A test purposea sequence of transitions to reach
?tea
?coffee
?tea+sugar
?coffee+sugar
?coin
?coin
!coffee !coffee
!tea
!sugar
!tea
!sugar
?i : input of the specification
!o : output of the specification
A test casea complete path in the behavioural specification
![Page 28: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/28.jpg)
LSR
General structure of COTE
UMLTest purposes
Behaviour computationUMLAUT/TGV
UMLTest cases
corba ejb .net
UML
Specification
Testsynthesis
Automatictests
generation
![Page 29: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/29.jpg)
LSR
Overview
• Context: the COTE project (COmponent TEsting)
• Why TOBIAS ?
• TOBIAS: description of the tool– Test purpose generation– Test case generation with VDM– Smarter generation
![Page 30: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/30.jpg)
LSR
Why TOBIAS ?Behavioural testing requires the specification of
many tests.– Test campaign => (potentially) thousands of test
cases– UMLAUT/TGV:
one test purpose and one execution = one test case
– New values => creation of new test purposes
Hugues MARTIN’s doctoral thesis (GEMPLUS): « Une méthodologie de génération automatique
de suites de tests pour applets java-card. » A methodology for automatic generation of
test suites for java-card applets.
![Page 31: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/31.jpg)
LSR
Generation of test purposes
• Idea : notion of high level test purpose
(test pattern)
=> abstraction of test purposes on objects, methods or values…
• A test pattern => a set of test purposes => a set of test cases
Significantly reduces the repetitive tasks of test definitions
![Page 32: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/32.jpg)
LSR
TOBIAS
UML
Specification
UMLTest purposes
Behaviour computationUMLAUT/TGV
Test synthesis
UMLTest cases
TOBIAS
Test pattern
![Page 33: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/33.jpg)
LSR
Overview
• Context: the COTE project (COmponent TEsting)
• Why TOBIAS ?
• TOBIAS: description of the tool– Test purpose generation– Test case generation with VDM– Smarter generation
![Page 34: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/34.jpg)
LSR
Principles of TOBIAS• TOBIAS provides:
– help for test purposes design– test pattern descriptions
• Idea:– instantiation of test patterns according to
different criteria (objects, parameters …)
![Page 35: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/35.jpg)
LSR
TOBIAS inputs
test patterns
UML specification
class diagram
deployment diagram
Test purposein UML
TOBIAS
![Page 36: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/36.jpg)
LSR
A little example (1)•UML class diagram
Actor
IAdministrator IUser HCI
*1
1
1
1
VendingMachine
Additionnal Drink
Cancel
Button
NumButton : integer
PushButton()
insertCoin(Coin p)
GiveChange()
1..5
1
0..1
![Page 37: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/37.jpg)
LSR
A little example (2)• UML deployment diagram (one for each
test campaign)
TestConfiguration:
DrinkMachine:VendingMachine
Testor:Actor
Front:HCI
Coffee:Drink
Tea:Drink
Cappucino:Drink
Chocolate:Drink
Reset:Cancel
![Page 38: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/38.jpg)
LSR
Some test purposes
For:- 4 different drinks- 1 cancel touch (Reset.GiveChange)- 5 different coins = {0.1, 0.2, 0.5, 1, 2}
The number of different test purposes is around:(5+5²) x (4 + 1) = 150 test purposes
Coffee.ClickButton()Chocolate.ClickButton()Tea.ClickButton()Cappucino.ClickButton()Reset.GiveChange()
insertCoin(0.2€);insertCoin(0.1€);…
1. insertCoin(2€);2. insertCoin(0.5€);3. insertCoin(1€);4. insertCoin(0.1€);5. insertCoin(0.2€);
5 test purposes :
![Page 39: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/39.jpg)
LSR
Coffee.ClickButton()Chocolate.ClickButton()Tea.ClickButton()Cappucino.ClickButton()Reset.GiveChange()
insertCoin(0.2€);insertCoin(0.1€);…
1. insertCoin(2€);2. insertCoin(0.5€);3. insertCoin(1€);4. insertCoin(0.1€);5. insertCoin(0.2€);
Principle of test patterns• Provide the generation of test purposes
according to different criteria:– the method parameters– the objects– the groups– the number of group or method calls
![Page 40: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/40.jpg)
LSR
Principle of test patterns• Provide the generation of test purposes
according to different criteria:– the method parameters– the objects– the groups– the number of group or method calls
Coffee.ClickButton()Chocolate.ClickButton()Tea.ClickButton()Cappucino.ClickButton()Reset.GiveChange()
insertCoin(0.2€);insertCoin(0.1€);…
1. insertCoin(2€);2. insertCoin(0.5€);3. insertCoin(1€);4. insertCoin(0.1€);5. insertCoin(0.2€);
![Page 41: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/41.jpg)
LSR
Coffee.ClickButton()Chocolate.ClickButton()Tea.ClickButton()Cappucino.ClickButton()Reset.GiveChange()
insertCoin(0.2€);insertCoin(0.1€);…
1. insertCoin(2€);2. insertCoin(0.5€);3. insertCoin(1€);4. insertCoin(0.1€);5. insertCoin(0.2€);
Principle of test patterns• Provide the generation of test purposes
according to different criteria:– the method parameters– the objects– the groups– the number of group or method calls
![Page 42: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/42.jpg)
LSR
Principle of test patterns• Provide the generation of test purposes
according to different criteria:– the method parameters– the objects– the groups– the number of group or method calls
Coffee.ClickButton()Chocolate.ClickButton()Tea.ClickButton()Cappucino.ClickButton()Reset.GiveChange()
insertCoin(0.2€);insertCoin(0.1€);…
1. insertCoin(2€);2. insertCoin(0.5€);3. insertCoin(1€);4. insertCoin(0.1€);5. insertCoin(0.2€);
![Page 43: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/43.jpg)
LSR
Coffee.ClickButton()Chocolate.ClickButton()Tea.ClickButton()Cappucino.ClickButton()Reset.GiveChange()
insertCoin(0.2€);insertCoin(0.1€);…
1. insertCoin(2€);2. insertCoin(0.5€);3. insertCoin(1€);4. insertCoin(0.1€);5. insertCoin(0.2€);
Principle of test patterns• Provide the generation of test purposes
according to different criteria:– the method parameters– the objects– the groups– the number of group or method calls
![Page 44: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/44.jpg)
LSR
An example
GroupAction = { Drink.ClickButton(), Cancel.GiveChange()}
Using the following test pattern:
insertCoin(x)^1..2 ; GroupAction
TOBIAS generates the 150 test purposes previously showed
![Page 45: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/45.jpg)
LSR
Advantages of using TOBIAS
• TOBIAS is useful to generate many test purposes
generating many test cases using UMLAUT/TGV
• Generating test purposes for new values is easier
• TOBIAS can be used to generate test suites for VDM in combination with VDMTools
![Page 46: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/46.jpg)
LSR
Using TOBIAS with VDM
• VDMtools provide an environment to specify an application, to execute test suites and to evaluate the test results
• The output of TOBIAS is directly used as a test suite.
Using TOBIAS for the automatic generation of VDM test cases - VDM Workshop 2002 at Floc02
![Page 47: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/47.jpg)
LSR
Using TOBIAS with VDM
TOBIAS
VDMTools
UML specification
Test patter
n
VDM test suites
Results: • fault detection• code coverage
VDM specification
![Page 48: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/48.jpg)
LSR
Using TOBIAS with VDM
• We did the following experiment:– specify an application: managing groups of
students– manually specify a test suite
(about 15 test cases)– generate a test suite using TOBIAS
(about 4300 test cases)
![Page 49: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/49.jpg)
LSR
Using TOBIAS with VDM• We obtained the following results:
– the generated and the manually written test suite have the same code coverage
– The development effort is similar
– the generated test suite detect more errors• one unknown error• one known error (from reading the specification)• new activations of a known error
– the generated test suite takes significantly more time to be executed:
• TOBIAS generates non conforming test cases• TOBIAS generates redundant test cases
![Page 50: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/50.jpg)
LSR
Smarter generation• Preventing non conforming test
purposes / test cases– using more elements from the
specification• relations between classes• pre/post conditions• state diagrams
• Reducing the test suite size– eliminating redundant test purposes / test
cases– defining constraints on the schemas
![Page 51: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/51.jpg)
LSR
Using more elementsTOBIAS extract classes from the UML class diagram.
Class A
Class B Class D
Class C
A!C C!A B!C C!B A!B B!A
A!D D!A B!D D!B C!D D!C
A!A B!B C!C D!D
the class A calls the methods of the class C
![Page 52: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/52.jpg)
LSR
Using more elementsRelations between classes
A!C C!A B!C xC!B xA!B xB!A x
Class A
Class B Class D
Class C0..1
0..1
1
1
0..1
1
A!D xD!A B!D D!B C!D xD!C x
A!A xB!B xC!C xD!D x
![Page 53: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/53.jpg)
LSR
Using more elements
Deployment diagram that gives a fixed state of the application
a:A
b:B d:D
c:C
A!C a!cC!A c!aB!C xC!B xA!B xB!A x
A!D xD!A d!a xB!D b!dD!B d!bC!D xD!C x
A!A xB!B xC!C xD!D x
Deployment diagram
![Page 54: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/54.jpg)
LSR
Using more elements
Deployment diagram that gives the initial state of the application
a:A
b:B d:D
c:C
A!C a!cC!A c!aB!C xC!B xA!B xB!A x
A!D xD!A d!aB!D b!dD!B d!bC!D xD!C x
A!A xB!B xC!C xD!D x
![Page 55: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/55.jpg)
LSR
Adding constraints
• The user can associate a constraint to each test pattern
• Constraints deal with parameters
• Constraints are used to filter test purposes / test cases
![Page 56: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/56.jpg)
LSR
Adding constraints
)1_2_)6.02_1_(()2_var( xxxxx
A test pattern:
insertCoin(x)^1..2 ; GroupAction
GroupAction = { Drink.ClickButton(), Cancel.GiveChange()}
A constraint:
Number of selected test purposes:
(5+9)x(4+1) = 70 test purposes
- insertCoin(x_1) ; GroupAction- insertCoin(x_1) ; insertCoin(x_2) ; GroupAction
![Page 57: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/57.jpg)
LSR
Conclusion• TOBIAS provides a new abstract level for test design: the
test patterns
• A test pattern => a set of similar test cases
• To avoid combinatorial explosion, the user can associate
constraints to the schemas
• TOBIAS can be used to generate test sequences
• TOBIAS will be used and validated on an industrial test case
in the COTE project
![Page 58: LSR TOBIAS and some ASE challenges Y. Ledru P. Bontron, O. Maury, L. du Bousquet, M.L. Potet, C. Oriat, S. Beghdadi, H. Bouldjedri LSR/IMAG – Grenoble](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d565503460f94a34b6d/html5/thumbnails/58.jpg)
LSR
Prospects
• Refining the test generation
– Deleting redundant
– Using pre/post conditions on operation to eliminate
non-conforming sequences
• Helping the decision to stop test campaign:
– Specification coverage by test patterns using more elements from the
specification