verification & validation verification –from latin veritas meaning truth. –building the...
TRANSCRIPT
![Page 1: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/1.jpg)
Verification & Validation
• Verification– from Latin veritas meaning truth.– Building the product right.
• Validation– from Latin Valere meaning to be worth.– Building the right product.
![Page 2: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/2.jpg)
Tools for V & V• Must be able to support
– modelling
– simulation
– state-space search
– testing
• Must be scalable
• Must be able to operate on different platforms– Unix or Linux
– Windows NT
– Windows 95,98
![Page 3: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/3.jpg)
Example: Telelogic Tau• Modelling
– MSC(Message Sequence Chart)• consists of entities and events occurring in time
between those entities.
– SDL(Specification and Description Language)• consists of a system defining the boundaries of the
overall task.
• the system consists of components called blocks which are connected by channels.
• the blocks can consist of blocks and/or processes.
• the processes are the actors of the system and are series of actions, changes or functions that bring about a particular result.
![Page 4: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/4.jpg)
– TTCN(Tree and Tabular Combined Notation)• A tree contained in a table.
• Tree is composed of texts having different indentations. Texts having the same indentation have the same height in the tree.
• Each text is preceded by “?” (incoming event) or “!”(outgoing event).
– State Space Coverage• A graph that shows all the different combinations of
inputs and outputs possible and their resulting transitions.
• It can be used to check for deadlocks and unspecified receptions.
![Page 5: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/5.jpg)
• Simulation– Tau contains a simulator that allows the interactive
inspection and checking of the system’s behavior.
• State-space search– a bit-state or a random walk can be done with the
validator in Tau. These will explore the state space of the SDL system to find the most possible run-time errors.
• Testing– TTCN link– Autolink
![Page 6: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/6.jpg)
Case study: Egg timer system
• An egg timer which boils an egg for the amount of time specified by the user.
![Page 7: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/7.jpg)
SDLSystem Diagram
• A system defines a set of blocks and a set of channels through which interaction occurs between the environment and the system.
• Signals, signal lists and types are also defined.
![Page 8: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/8.jpg)
SDL• Specification and Description Language
– Initially built for communicating systems– Divides the specification in
• Environment
• System(s)SD T rw /a/insite/users/isales/sdl/E ggB oiler/egg_ boiler.sdt
rw /a/insite/users/isales/sdl/E ggB oiler/
A nalysis M odel
U sed F iles
SD L System Structu re
E ggB oiler [unconnected]
D ecT im e rw D ecT im e.spdD isplaying rw D isplaying.sprI n terfacing rw I n terfacing.sprT im ing rw T im ing.spr
E g g T im er R W E ggB oilerC ontroller.sbk
E g g T im erS y stem R W E ggT im erSystem .ssy
T T C N T est Specification
O ther D ocum ents
The Environment Border
![Page 9: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/9.jpg)
Into the blocks
• Blocks are composed of processes or of other blocks.
• Each block eventually contains processes.
• The processes communicate with the environment of the block via signal routes.
![Page 10: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/10.jpg)
Signals and Signal lists
• Signals are the means through which processes interact.
• Signals to and from the environment are declared at the system level.
• Signal lists group signals that are used in the same grouping at several places.
![Page 11: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/11.jpg)
SDL• A Whole system
SD T rw /a/insite/users/isales/sdl/E ggB oiler/egg_ boiler.sdt
rw /a/insite/users/isales/sdl/E ggB oiler/
A nalysis M odel
U sed F iles
SD L System Structu re
E ggB oiler [unconnected]
D ecT im e rw D ecT im e.spdD isplaying rw D isplaying.sprI n terfacing rw I n terfacing.sprT im ing rw T im ing.spr
E g g T im er R W E ggB oilerC ontroller.sbk
E g g T im erS y stem R W E ggT im erSystem .ssy
T T C N T est Specification
O ther D ocum ents
SystemBlock
Process
Procedure
![Page 12: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/12.jpg)
Inside an SDL System
S ys tem E g g T im erS ystem 1(1)
S IG N A L O neM inuteB tn, S tartB tn , R esetS to p B tn;
S IG N A LL IS T B utto ns = O neM inuteB tn, S tartB tn , R esetS to p B tn;
S IG N A L D isp lay(M inuteD ig it, S eco nd D ig it1, S eco nd D ig it2), B ell;
S IG N A LL IS T To U ser = D isp lay, B ell;
S IG N A L S tartB o iling , S to p B o iling ;
S Y N TY P E M inuteD ig it = Integ er C O N S TA N TS 0:5E N D S Y N TY P E ;
S Y N TY P E S eco nd D ig it1 = Integ er C O N S TA N TS 0:5E N D S Y N TY P E ;
S Y N TY P E S eco nd D ig it2 = Integ er C O N S TA N TS 0:9E N D S Y N TY P E ;
E g g T im er
E g g B o iler
U serInterface(To U ser)
(B utto ns )
InternalS tartB o iling ,S to p B o iling
Channel (and channel name)System name
Signals (Messages)Declarations
Signalist (set of messages)
Newtype definitions
![Page 13: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/13.jpg)
SDL System Structure
E g g T im er
E g g B o iler
U serInterface(To U ser)
(B utto ns )
InternalS tartB o iling ,S to p B o iling
Signals are shown in square brackets,separated by a comma
System blocks
Channel name
Signallists are declared like signals,but enclosed in parenthesis
Environment of the system
![Page 14: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/14.jpg)
The EggTimer SDL Block
B lo ck E g g T im er 1(1)
S IG N A L IncM inute, S tartT im er, R esetT im er;
S IG N A L D isp layT im e(M inuteD ig it,S eco nd D ig it1,S eco nd D ig it2), R ing TheB ell;
D ecT im e
Interfac ing (1,1)
T im ing (1,1) D isp laying (1,1)
U serInterface
InterfaceIn
(B utto ns )
U serInterface
To T im er
IncM inute,S tartT im er,R esetT im er
In terfaceO ut
D isp lay,B ell
In ternalTo B o iler
S tartB o iling ,S to p B o iling
To D isp lay
D isp layT im e,R ing TheB ell
Connection points (system channel names)
Procedure declaration Process declarationsGeneral declarations (signals, types, vars)
routes
Signals
![Page 15: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/15.jpg)
The Egg-Timer SDL Block Structure
B lo ck E ggT im er 1(1)
Interfac ing (1,1)
D ecT im e
Tim ing (1,1) D isp laying (1,1)
U serInterface
InterfaceIn(B utto ns )
U serInterface
To T im erIncM inute,S tartT im er,R esetT im er
InterfaceO ut
D isp lay,B ell
InternalTo B o iler
S tartB o iling ,S to p B o iling
To D isp lay
D isp layT im e,R ing TheB ell
ProcessesAt least 1 instance and at most1 instance of this process
Procedure Reference
Displaying sends to the environmentand receives from Timingetc...
![Page 16: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/16.jpg)
SDL Process exampleP ro cess T im ing 1(1)
D C L m M inuteD ig it := 0, s1 S eco nd D ig it1 := 0, s2 S eco nd D ig it2 := 0;
T im er O neS eco nd ;
S etting T im e
IncM inute
m < 5
m := m + 1
D isp layT im e(m ,s1,s2)
S etting T im e
S tartT im er
S et(N O W + 1,O neS eco nd )
S tartB o iling
T im ing
O neS eco nd
D ecT im e(m ,s1,s2)
D isp layT im e(m ,s1,s2)
m = 0 ands1 = 0 and
s2 = 0
S to p B o iling
R ing TheB ell
S etting T im e
S et(N O W + 1,O neS eco nd )
-
R esetT im er
m := 0;s1 := 0;s2 := 0;
R eset(O neS eco nd )
S etting T im e
R esetT im er
m := 0;s1 := 0;s2 := 0;
S etting T im e
true
false
true false
State
Initial stateInitial transition
Declarations
Input
Output
If
Procedure call
Task
Timer Input(or timeout)
![Page 17: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/17.jpg)
SDL Declaration example
D C L m M inuteD ig it := 0, s1 S eco nd D ig it1 := 0, s2 S eco nd D ig it2 := 0;
T im er O neS eco nd ;
To say you’re declaring something
Identifier(instance name)
Instance type
InitializationTimer declaration
Timer Identifier
![Page 18: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/18.jpg)
SDL Transition exampleS etting T im e
IncM inute
m < 5
m := m + 1
D isp layT im e(m ,s1,s2)
S etting T im e
true
false
FROM
WHEN
ACTIONS
SEND
TO
Local variableParameters to send
![Page 19: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/19.jpg)
SDL Tasks
m := m + 1
S et(N O W + 1,O neS eco nd )
D ecT im e(m ,s1,s2)
m := 0;s1 := 0;s2 := 0;
m := 0;s1 := 0;s2 := 0;
Incrementing or attributing a value
Making a procedure call
Setting the timer
Set([NOW+]<Duration>, <TimerName>)<variable> := <expression>[; <another task]
![Page 20: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/20.jpg)
SDL Procedures(and operators)
;F PA R IN /O U T m M inuteD ig it, IN /O U T s1 S eco nd D ig it1, IN /O U T s2 S eco nd D ig it2;
P ro ced ure D ecT im e 1(1)
s2 = 0
s1 = 0
m = 0
s1 := 5;s2 := 9;
m := m -1;
s2 := 9;s1 := s1 - 1;
s2 := s2 - 1;
true
true
truefalse
false
false
Formal Parameters
Param modifiername
and type
ProcedureStart
Procedure return
If first ‘seconds’digit is 0 thengo down, elsego left
Modifiers:IN - Parameter is an Input parameterOUT - Parameter is an Output parameterIN/OUT - Parameter is both IN and OUT
![Page 21: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/21.jpg)
SDL Message HandlingEach process
•has one and only one message queue•sends (by default) a message to the closest receiver (if the process itself is capable of receiving such message, it will send to itself)•the default sending procedure can be modified by
•VIA:
•TO:
<Message_name> VIA <channel_name>
<Message_name> TO <process_id>
![Page 22: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/22.jpg)
T im ing (1,1)
To T im erIncM inute,S tartT im er,R esetT im er
To D isp lay
D isp layT im e,R ing TheB ell
SDL Message Handling
This process queue has four possible messages•IncMinute•StartTimer•ResetTimer•OneSecond
The first three are external messages where OneSecond is an internal message, indicating when the clock has decreased one second.
There could be several other messages comingfrom other processes or other blocks
![Page 23: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/23.jpg)
Simulation
• Once a complete SDL system has been designed, the Tau tool allows us to simulate the system i.e. to inspect and check its behavior.
• After having generated an executable simulator, a simulator user interface can be started. It looks like the following:
![Page 24: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/24.jpg)
![Page 25: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/25.jpg)
Validation
• The validator is used to explore the state space to find errors or inconsistencies in the system.
• It can also be used to verify the system against requirements by running system level MSCs in the validator.
• An executable validator must be generated and the validator user interface can then be started.
![Page 26: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/26.jpg)
![Page 27: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/27.jpg)
Principles of Validation
i) Design– User-centred view
• scenarios
• use-cases
• MSCs
• functional, blackbox, providing a service (or NOT), end to end
– guided execution in SDL high-level design– refined design, re-run validation scenarios
![Page 28: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/28.jpg)
Next i:Tr. Start
SDL Guided execution• Using the Navigator in a SDL Validator
tool.Up 1
Next 1:Signal:
OneMinuteBtn
Next 2:Signal: StartBtn
Next 2:Signal:
ResetStopBtnNext 1:Tr. Start
...
Next 1:...
Input: DisplayTimeSender: Timing:1Parameter(s):1,0,0
By clicking on theNext, we go derivinga scenario by followingthe state-space tree
![Page 29: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/29.jpg)
SDL Guided Execution(in MSC form)
M S C Valid ato rTrace
env_ 0
Interfac ing
Interfac ing _ 1
T im ing
T im ing _ 2
D isp laying
D isp laying _ 3
Id le
S etting T im e
D isp laying
O neM inuteB tn
IncM inute
S etting T im er
D isp layT im e
1, 0, 0
S etting T im e
D isp lay
1, 0, 0
D isp laying
Valid ato r traceg enerated b yS D T Valid ato r 2.3
![Page 30: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/30.jpg)
Functional MSC for the previous EggTimer scenario
M S C Validato rTrace
E nviro nm ent E g g T im erS ys tem
O neM inuteB tn
D isp lay
1, 0, 0
Valid ato r traceg enerated b yS D T Valid ato r 2.3
And executing this MSC in the Validator Tool,by clicking on “Verify MSC” and selectingthe proper file, we obtain (as the system has been correctly specified) 1 report
1 MSC VerificationMSC ValidatorTraceverifiedDepth: 7
![Page 31: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/31.jpg)
Principles of Validationii) Test
• functional user interface tests in TTCN
S1
S2 S3
c ab
ab
+Test NameEvent a Event b pass
Tree and TabularCombined Notation
•TTCN standard for formal test case descriptions•Facilitate sharing of tests•Eliminates Ambiguities•UNIX-based TTCN Workbench toolkit•TTCN used for X.25, ISDN, CCS7, and many other protocols•All PDU parameters specified•All tests are formally assigned verdicts
![Page 32: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/32.jpg)
Principles of Validation
ii) Test (cont’d)
• basic syntax & semantics of TTCN (TREE and TABLE Combined Notation)– Test Case Dynamic Behaviour
Test Case Name: Group:Purpose:Default:
Behavior Constr. VerLabel Description Reference dict Remarks
A!X A?Y A!J A?K Pass
A?Z Fail Wrong Resp.
A?Otherwise Inc. Test body not reached
Extended Comments:
![Page 33: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/33.jpg)
Principles of Verification
i) Design– State-space search
Process Chart
Transition Chart
State Space
-transitions
reduced state space
![Page 34: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/34.jpg)
Principles of Verification
i) Design (cont’d)– Possible outcomes
• Unspecified reception
• queue overflow
• deadlock
• infinite livelock (no real progress)
– Random walk
![Page 35: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/35.jpg)
Principles of Verification
ii) Test– Conformance Testing (TTCN)
• preamble
• test step
• verification sequence
• postamble
![Page 36: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/36.jpg)
Principles of VerificationStructure of a Transition Test Case
for Conformance Testing
PREAMBLE: Initializationto Preamble State
TEST BODY: Application ofTest Stimulus and Confirmation of
System Response
VERIFICATION: Confirmationof Post-Transition State
POSTAMBLE: Reset to StableState for Next Test
![Page 37: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/37.jpg)
Principles of VerificationRepresent S1 S3 S4a/X b/Y
As a sequence of user actions and system responses, starting in state 1 and finishing in state 4.
In TTCN
+Preamble(S1) !a ?X !b ?Y +Verify(S4) PASS * FAIL * Inconclusive
![Page 38: Verification & Validation Verification –from Latin veritas meaning truth. –Building the product right. Validation –from Latin Valere meaning to be worth](https://reader034.vdocument.in/reader034/viewer/2022052510/56649e7b5503460f94b7ce67/html5/thumbnails/38.jpg)
Industrial Strength Tool Support
• Value added– MSC coverage and test coverage.– auto test generation.– reverse engineering can be done to a certain
extent.– provides graphics editor(MSC & SDL).– provides editor for TTCN(ITEX).