a model-driven approach to trace checking of pattern-based temporal properties
Post on 22-Jan-2018
186 Views
Preview:
TRANSCRIPT
.lusoftware verification & validationVVS
A Model-Driven Approach to Trace Checking of
Pattern-based Temporal PropertiesWei Dou, Domenico Bianculli, Lionel Briand
SnT Centre - University of Luxembourg
.lusoftware verification & validationVVS
A Model-Driven Approach to Trace Checking of
Pattern-based Temporal PropertiesWei Dou, Domenico Bianculli, Lionel Briand
SnT Centre - University of Luxembourg
Model-driven Engineering at CTIE
6
Creation Date: 16/10/2010 Last Modification: 29/11/2011 (LMA) Version: 0.6 email: bpmo.info@ctie.etat.luAuthor: LAN
Based on some data, a decision will be made as for what to do next. Each branch is annotated with the condition under which the branch is taken. All conditions are exclusive, so that only one branch is taken. If all possible cases are not covered by explicit conditions, a default branch must be present. When merging, each incoming flow directly activates the outgoing flow without synchronization.
With a splitting inclusive gateway, each outgoing branch should show the condition under which the branch is taken. When the gate is activated, the conditions of all branches are evaluated and one or more branch(es) will be taken.When joining, inclusive gateways aresynchronizing : therefore, the branch coming out of the gateway is only activated when all active incoming branches have completed.
The Event-Based Gateway represents a branching point in the Process where the alternative paths that follow the Gateway are based on events that occur (message reception, timeout, signal, etc).When the first event is triggered, the path that follows that event will be used and all the remaining paths will no longer be valid.
The error end event indicates that a named error is generated and that all currently active threads in the particular subprocess are terminated.The error must be caught by using a catch error intermediate event on the boundary of the nearest enclosing parent activity.
COMMON PATTERNS PROCESS MODELLING METHOD
RULES and GUIDELINES
Data based exclusive decision
Data based inclusive decision
Event based decision
Error handling
1. All flow objects other than start events, boundary events, and compensating activities must have an incoming sequence flow.
2. All flow objects other than end events and compensating activities must have an outgoing sequence flow.
3. Message flows can only be used between pools.Sequence flows can only be used within a pool.
4. Only message events, pools and send/receive/user/service tasks can be used with messages flows.
5. A start event cannot have incoming sequence flow or outgoing message flow, but can have incoming message flow.
6. An end event cannot have incoming message flow or outgoing sequence flow, but can have outgoing message flow.
7. Each white box pool must have at least one start event and one end event. Multiple start events should be used sparingly.
8. A gateway is either « diverging» (one input, multiple outputs) or « converging » (multiple inputs, one output).
9. An activity must have exactly one input sequence flow and one output sequence flow.
10. Activity names start with a verb and contain a noun.
11. All tasks should have an activity type.
12. Only one outgoing path from exclusive, inclusive and complex gateways may be left unlabeled.
Level%1
1a.%Process%MapA"Process"Map"provides"a"structured"overview"of"all"processes. Level%1
1b.%Function%TreeA"Function"Tree"provides"a"structured"inventory"of"all"activities,"as"well"as"a"view"of"each"function’s"hierarchy.
3a.%BPMN%Function%Allocation%DiagramA"BPMN"FAD"can"be"used"to"describe"various"elements"of"the"process’"context"such"as"risks,costs,"assisting"roles,"KPIs,"etc.
2.%Value%Added%ChainOne"or"two"levels"of"Value"Added"Chain"can"be"used"to"link"processes"together"or"to"break"down"the"most"complex"processes.
Level%3 3b.%BPMN%Collaboration%diagramsProcesses"are"described"using"layers"of"BPMN"diagrams."BPMN"Collaboration"diagrams"are"used"to"describe"interactions"between"participants"and"for"highIlevel"and"intermediary"processes.
Level%4
4.%BPMN%Process%diagramsDetails"of"activities"performed"by"a"single"participant"are"described"at"lowIlevel"by"BPMN"process"diagrams."For"this"type"of"diagram,"all"elements"of"the"Collaboration"diagrams"can"be"used,"except"pools"and"message"flows"(as"only"one"participant"is"involved).
Level%2
BPMN%Conventions
Level%3
Credit: Centre des Technologies de l'Information de l'Etat
Compliance of a Business Process with respect to its Requirements
• Model checking
• Run-time monitoring and verification
• Offline trace checking
9
Compliance of a Business Process with Respect to its Requirements
Offline trace checking as a first step, and possible development,
towards run-time checking
10
Goal of the project
To develop a practical and scalable solution for offline trace checking of the requirements of an
eGovernment business process
11
Requirements (1)
• To be viable in the long term, any procedure shall rely on standard MDE technology (tools implementing OMG specifications)
• Why?
• our industrial partner has adopted MDE in practice
• requires any software solution added to the development process to adhere to OMG specifications and rely on the corresponding tools, for ease of integration with existing tools and solutions
• It can be generalized to other contexts in which solutions have to be engineered by using standard MDE technologies
12
Requirements (2)
• Any procedure shall:
• scale linearly with respect to the length of the trace
• complete within practical time limits
• e.g., a trace with millions of events should be checked within seconds
• to enable real-time log analysis, to promptly detect violations
13
Our contributions
• A scalable and practical solution, based on standard MDE technologies, for offline trace checking of temporal properties
• Tool: TemPsy-Check
http://weidou.github.io/TemPSy-Check/
14
Main Ingredients of Offline Trace Checking
15
2.2. Definition of TemPsy
hTemPsyBlocki ::= hTemPsyExpressioni+hTemPsyExpressioni ::= [‘temporal’ hIdi ‘:’]
hScopei hPatternihScopei ::= ‘globally’
| ‘before’ hBoundary1i| ‘after’ hBoundary1i| ‘between’ hBoundary2i
‘and’ hBoundary2i| ‘after’ hBoundary2i
‘until’ hBoundary2ihPatterni ::= ‘always’ hEventi
| ‘eventually’ hRepeatableEventExpi| ‘never’ [‘exactly’ hInti] hEventi| hEventChainExpi ‘preceding’
[hTimeDistanceExpi] hEventChainExpi| hEventChainExpi ‘responding’
[hTimeDistanceExpi] hEventChainExpihBoundary1i ::= [hInti] hEventi [hTimeDistanceExpi]hBoundary2i ::= [hInti] hEventi [‘at least’ hInti ‘tu’]
hEventChainExpi ::= hEventi(‘,’ [‘#’ hTimeDistanceExpi] hEventi)*
hTimeDistanceExpi ::= hComparingOpi hInti ‘tu’
hRepeatableEventExpi ::= [hComparingOpi hInti] hEventihComparingOpi ::= ‘at least’ | ‘at most’ | ‘exactly’
hEventi ::= hIdihIdi ::= hIdStartChari hIdChari*
| hIdi (hIdConnectori hIdi)*hIdStartChari ::= [A-Z] | ‘_’ | [a-z]
hIdChari ::= hIdStartChari | [0-9]
hIdConnectori ::= ‘.’ | ‘::’
hInti ::= [1-9] ([0-9])*
Figure 2.2: Syntax of TemPsy
boundaries (expressed with hBoundary1i or hBoundary2i) that denote a specific occurrence of anevent as a boundary, possibly with a time distance; notice that hBoundary2i represents a syntacticrestriction of hBoundary1i. Chains of events, used in precedence and response patterns, are definedas hEventChainExpi, which denotes a comma-separated list of events, possibly with a time distance(hTimeDistanceExpi) between each pair of events (denoted with the ‘#‘ symbol). Time distances areexpressed with an integer value, followed by the ‘tu’ keyword, which represents a generic time unit(i.e., any denomination of time).
11
specification language
offline trace checking procedure
Requirements Analysis of eGovernment Business Processes
• Analysis of the requirement documents of eGovernment business processes
• The majority of the requirements could be expressed as temporal properties
17
Specification of Temporal Properties
18
globally at most 2 P before R at most 2 P after Q at most 2 P between Q and R at most 2 P after Q until R at most 2 P
Temporal Logic
Domain-specific Language (DSL)
Excerpt from TemPsy Grammar
25
2.2. Definition of TemPsy
hTemPsyBlocki ::= hTemPsyExpressioni+hTemPsyExpressioni ::= [‘temporal’ hIdi ‘:’]
hScopei hPatternihScopei ::= ‘globally’
| ‘before’ hBoundary1i| ‘after’ hBoundary1i| ‘between’ hBoundary2i
‘and’ hBoundary2i| ‘after’ hBoundary2i
‘until’ hBoundary2ihPatterni ::= ‘always’ hEventi
| ‘eventually’ hRepeatableEventExpi| ‘never’ [‘exactly’ hInti] hEventi| hEventChainExpi ‘preceding’
[hTimeDistanceExpi] hEventChainExpi| hEventChainExpi ‘responding’
[hTimeDistanceExpi] hEventChainExpihBoundary1i ::= [hInti] hEventi [hTimeDistanceExpi]hBoundary2i ::= [hInti] hEventi [‘at least’ hInti ‘tu’]
hEventChainExpi ::= hEventi(‘,’ [‘#’ hTimeDistanceExpi] hEventi)*
hTimeDistanceExpi ::= hComparingOpi hInti ‘tu’
hRepeatableEventExpi ::= [hComparingOpi hInti] hEventihComparingOpi ::= ‘at least’ | ‘at most’ | ‘exactly’
hEventi ::= hIdihIdi ::= hIdStartChari hIdChari*
| hIdi (hIdConnectori hIdi)*hIdStartChari ::= [A-Z] | ‘_’ | [a-z]
hIdChari ::= hIdStartChari | [0-9]
hIdConnectori ::= ‘.’ | ‘::’
hInti ::= [1-9] ([0-9])*
Figure 2.2: Syntax of TemPsy
boundaries (expressed with hBoundary1i or hBoundary2i) that denote a specific occurrence of anevent as a boundary, possibly with a time distance; notice that hBoundary2i represents a syntacticrestriction of hBoundary1i. Chains of events, used in precedence and response patterns, are definedas hEventChainExpi, which denotes a comma-separated list of events, possibly with a time distance(hTimeDistanceExpi) between each pair of events (denoted with the ‘#‘ symbol). Time distances areexpressed with an integer value, followed by the ‘tu’ keyword, which represents a generic time unit(i.e., any denomination of time).
11
Specific Occurrence of an Event in a Scope Boundary
“A password reset email will be sent to the user after the third incorrect login attempt.”
27
Constraint on the Time Distance from a Scope Boundary
“The card requester has to show up for the collection of her physical information within five days from the first notification.”
28
Constraint on the Time Distance between Two Events in Precedence and Response Patterns
29
“If an ID card is reported as lost, then a new card will be issued within five days.”
Constraint on the Number of Occurrences of the Events in Existence and Absence Patterns
30
“The notification of renewing an ID card should be sent at least twice before it expires.”
Between-and Scope
between [n1-th] E1 [distance constraint] and [n2-th] E2 [distance constraint]
32
specific occurrence of an event in a scope boundary
Constraint on the time distance from a scope boundary
Example - Request
36
“Once a card request is approved, the applicant is notified within three days; this notification has to occur before the
production of the card is started.”
(Before + Response)
37
temporal R1:
before ICM.issueCard
ICM.notifyApproval responding at most 3*24*3600 tu ICM.approveRequest
“Once a card request is approved, the applicant is notified within three days; this notification has to occur before the
production of the card is started”
Example - Request
38
temporal R1:
before ICM.issueCard
ICM.notifyApproval responding at most 3*24*3600 tu ICM.approveRequest
“Once a card request is approved, the applicant is notified within three days; this notification has to occur before the
production of the card is started”
Example - Request
Approach Based on Off-the-shelf Trace Checker for Temporal Logic
41
Trace
TemPsy Properties
Off-the-shelf trace checker for
temporal logic
MonPoly, SOLOIST-ZOT
Approach Based on Off-the-shelf Trace Checker for Temporal Logic
42
Trace
TemPsy Properties
Temporal Logic MonPoly,
SOLOIST-ZOT
Off-the-shelf trace checker for
temporal logic
from the project requirements: “…solution based on OMG standards and stable MDE technology”
Our Approach for Offline Trace Checking of TemPsy Properties
Reducing offline trace checking of TemPsy properties to the evaluation of OCL constraints (semantically-equivalent to TemPsy properties) on an instance of a trace model
43
Trace Meta-Model
44
Trace TraceElementevent: String timestamp: Integer*properties:
Elist<TemPsyExpression>
traceElements
TemPsy Meta-Model
45
Chapter 3. Model-driven Offline Trace Checking of Temporal Properties
traceElements*
Trace
properties : EList<TemPsyExpression>
applyScopeGlobally(Scope):Elist<EList<TraceElement>>applyScopeBefore(Scope):Elist<EList<TraceElement>>applyScopeAfter(Scope):Elist<EList<TraceElement>>applyScopeBetweenAnd(Scope):Elist<EList<TraceElement>>applyScopeAfterUntil(Scope):Elist<EList<TraceElement>>checkPatternUniversality(EList<TraceElement>, Pattern):BooleancheckPatternExistence(EList<TraceElement>, Pattern):BooleancheckPatternAbsence(EList<TraceElement>, Pattern):BooleancheckPatternPrecedence(EList<TraceElement>, Pattern):BooleancheckPatternResponse(EList<TraceElement>, Pattern):Boolean
TraceElement
event : EStringtimestamp : EInt
Figure 3.1: Conceptual model for execution traces
*temPsyExpressions
1scope 1pattern
boundary1 boundaryBegin
1
1
boundaryEnd
0..1
timeDistance
1 event
timeDistance
0..1block21..*1..*block1
0..1
timeDistance
1
event
1event
TemPsyBlock
TemPsyExpression
Scope
type:ScopeType
⌧enumeration�ScopeType
GLOBALBEFOREAFTERBETWEENANDAFTERUNTIL
Global
UniScope
BiScopeBoundary
ordinal:EInt[0..1]
Pattern
type:PatternType
Universality
OccurrencePattern
comparisonOperator:ComparisonOperator[0..1]times:EInt[0..1]
OrderPattern
EventChainElement EventRepresentation
name:EStrison
TimeDistance
comparisonOperator:ComparisonOperatorquantity:EInt
⌧enumeration�PatternType
UNIVERSALITYEXISTENCEABSENCEPRECEDENCERESPONSE
⌧enumeration�ComparisonOperator
ATLEASTATMOSTEXACTLY
Figure 3.2: TemPsy model
type string, which represents the actual event recorded in the trace, and an attribute timestamp oftype integer, which indicates the time at which the event occurred. Class Trace contains also an at-tribute properties, which is a collection of TemPsyExpressions1, representing the propertiesto be checked on the trace.
We have defined some side-effect-free operations in OCL for the Trace class; these operationsconsist of two types of functions. The first type, of the form applyScope*S*, are named afterthe different types of scope (e.g., applyScopeBefore, applyScopeBetweenAnd) and returnsegment(s) of a trace (i.e., sub-traces) as determined by the parameters of the scope provided ininput. The second type, of the form checkPattern*P*, are named after the different types ofpattern (e.g., checkPatternExistence, checkPatternPrecedence) and check whether
1Class TemPsyExpression belongs to the meta-model of the language (not shown here for space reasons) andrepresents objects corresponding to the non-terminal hTemPsyExpressioni of the grammar shown in Figure 2.2.
28
Model-driven Offline Trace Checking of TemPsy Properties
47
Eclipse OCL
Trace
TemPsy Properties
Trace Instance
TemPsyExpression
Instances
OCL Constraints on Trace Class
TemPsy-Check
Simplified Template of OCL Constraints
context Trace
inv: let subtraces=applyScope*S*(scope) in
subtraces->forAll(subtrace |
checkPattern*P*(subtrace, pattern))
48
Simplified Template of OCL Constraints
context Trace
inv: let subtraces=applyScope*S*(scope) in
subtraces->forAll(subtrace |
checkPattern*P*(subtrace, pattern))
49
Simplified Template of OCL Constraints
context Trace
inv: let subtraces=applyScope*S*(scope) in
subtraces->forAll(subtrace |
checkPattern*P*(subtrace, pattern))
50
IterationapplyOriginalBetweenAnd(scope)
53
Find the next occurrence of event A
If event A has occurred, find the next occurrence of event B
Select the trace segment between the occurrences of event A and B
Append the trace segment to the output
Apply the time distance constraints on the boundaries, if either of them is not empty
IterationcheckPrecedenceAtMost(subtrace, pattern)
56
Find the latest occurrence of block1
If the first event of block2 occurs:
If t(firstOfBlock2) - t(lastOfBlock1) > T, report a violation if size(block2) = 1 or postpone until the entire block2 is found; otherwise ignore and continue the iteration
Or else if the next event of block2 occurs, report a violation if the entire block2 is found
TemPsy-Checkhttp://weidou.github.io/
TemPSy-Check/
57
Evaluation
• Real properties checked on synthesized traces
• Focus on scalability
• Comparison with MonPoly from ETHZ
59
Why Synthesized Traces?
1. Real traces are often inadequate to cover a large range of trace lengths and a variety of properties
2. We want to be able to control the diversity of occurrences of patterns in the traces
3. Real traces are not suitable for scalability evaluation, since they may contain faults
4. Real traces could not be shared for forming a public benchmark
60
Why MonPoly?
• Winner of the first International Competition on Software for Runtime Verification (CSRV 2014)
• Caveat: MonPoly supports MFOTL, which is more expressive than TemPsy
61
Properties (globally scope)
62
•P1: globally always A
•P2: globally never B
•P3: globally eventually at least 2 A
•P4: globally eventually at most 3 A
•P5: globally A responding at most 1000 tu B
•P6: globally A responding exactly 1000 tu B
•P7: globally A preceding at most 6000 tu B
•P8: globally A preceding at least 100 tu B
•P9: globally A preceding exactly 100 tu B
•P10: globally A, B preceding at least 1000 tu C, D
•P11: globally A responding at least 1000 tu B, C
•P12: globally A responding B
Research Questions (globally scope)
RQ-G1) What is the relation between the execution time of TemPsy-Check and the length of a trace?
RQ-G2) What are the types of pattern most taxing on the execution time?
RQ-G3) How does TemPsy-Check compare with MonPoly in terms of execution time?
63
Experiment Results (globally scope)
64
Aver
age e
xecu
tion
time
1s
2s
3s
Trace length100k 200k 300k 400k 500k 600k 700k 800k 900k 1.000k
P1 P2 P3 P4 P5 P6 P7 P8P9 P10 P11 P12
Answers to Research Questions RQ-G1 and RQ-G2
RQ-G1 TemPsy-Check scales linearly with respect to the length of a trace
RQ-G2 The precedence and response patterns that contain distance constraints are most taxing, e.g., P5, P6, P7, P9, P11
65
Aver
age e
xecu
tion
time
1s
2s
3s
Trace length200k 400k 600k 800k 1.000k
P1 P2 P3 P4 P5P6 P7 P8 P9 P10P11 P12
Comparison with MonPoly for Batch Checking of P3-P12
66
Aver
age e
xecu
tion
time
60s
120s
180s
240s
300s
360s
Trace length1M 2M 3M 4M 5M 6M 7M 8M 9M 10M
TemPsy-Check MonPoly
Comparison with MonPoly for Batch Checking of P3-P12
67
Aver
age e
xecu
tion
time
90s
180s
270s
360s
Trace length1M 2M 3M 4M 5M 6M 7M 8M 9M10M
TemPsy-CheckMonPoly
•TemPsy-Check performs similarly to or better than MonPoly in terms of batch execution time
Future Work
• Run-time verification of temporal properties
• More informative output when violations are detected in a trace
70
top related