alignment based precision checking
DESCRIPTION
This slide tells you how to measure precision between process model and its recorded execution. Presented at BPI 2012, Tallinn, Estonia, by Arya Adriansyah. The technique is implemented in ProM 6.x, package ETConformance (see http://processmining.org).TRANSCRIPT
![Page 1: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/1.jpg)
Alignment-based
Precision CheckingA. Adriansyah1, J. Munoz Gamma2,
J. Carmona2, B.F. van Dongen1,
W.M.P. van der Aalst1
Tallinn, 3 September 2012
1) Department of Mathematics and Computer Science,
Eindhoven University of Technology, The Netherlands
2) Software Department, Universitat Politècnica de
Catalunya, Spain
![Page 2: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/2.jpg)
Model needs to be precise
1
Event Log
Process Models
a d
a b
b
cend
c
a b
d
end
aabd
abd
acd
aabd
c
a d
![Page 3: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/3.jpg)
2
Event Log
Process Modelsc
a b
d
end
aabd
abd
acd
aabd
c
ad
Unfitting
d
a d
a b
b
cend
a
b
c
b
a
d
14 4
2
d
1
1
1d
2 2
0
0
b
c
0
d0
d
c0…
…
…
…
0a
0c
…
Prefix automaton
Reference: J. Muñoz-Gama, J. Carmona (2010). A
Fresh Look at Precision in Process Conformance.
8th International Conference of Business Process
Management (BPM): 211-226.
How to measure precision for unfitting
event logs?
![Page 4: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/4.jpg)
Overview
3
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabd
aabd
acd
…
Fitting TracesAlignment
Precision
Metrics
![Page 5: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/5.jpg)
Optimal alignment: least #deviations
4
a d
a b
b
cend
Trace adab
aAlignment
d b
a a b
a
d
Process Model
aabd Reference: W.M.P. van der Aalst, A.
Adriansyah, B.F. van Dongen (2012).
Replaying history on process models
for conformance checking and
performance analysis. WIREs Data
Mining and Knowledge
Discovery, 2(2), 182-192.
![Page 6: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/6.jpg)
Optimal alignments can be many
5
a d
a b
b
cend
Trace ad
Alignment 1 Alignment 2
Process Model
ba
a d
d ca
a d
d
Both alignments are optimal
abd or acd
![Page 7: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/7.jpg)
Overview
6
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments Precision
Metrics
aabd
aabd
acd
…
Fitting TracesAlignment
![Page 8: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/8.jpg)
a22 3
db1.51.5 11
b1a
d
3
All-Alignment Automaton (Prefix-based)
7
c
Fitting Traces
a d
a b
b
cend
Process Model
All-Alignments Automaton
aabd
abd
abd or acd
aabd
1 1
0.5d
0.5
4 4
2 2 2
![Page 9: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/9.jpg)
a
b
b
a
d
24 4
2 2 2
d2
aabd
abd
abd or acd
aabd
1-Alignment automaton (Prefix-based)
8
Fitting Traces
a d
a b
b
cend
Process Model
1-Align Automaton
aabd
abd
abd
aabd
![Page 10: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/10.jpg)
b d
b
a
ad
4 4
2 2 2
1-Alignment automaton (Prefix-based)
9
0c
Fitting Traces
a d
a b
b
cend
Process Model
1-Align Automaton
aabd
abd
abd
aabd
Imprecision
2 2
![Page 11: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/11.jpg)
1-Alignment
Comparison of log automata
10
aabd
abd
ad
adab
Event Log
a da b
bc
end
Process Model
Without alignment
All-Alignments
Pre
fix-b
ased
Lo
g A
uto
mata
aabd
abd
abd/acd
aabd
Fitting Traces
![Page 12: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/12.jpg)
Overview
11
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabd
aabd
acd
…
Fitting TracesAlignment
Precision
Metrics
Reference: J. Muñoz-Gama, J. Carmona (2010). A Fresh Look at Precision in
Process Conformance. 8th International Conference of Business Process
Management (BPM): 211-226.
![Page 13: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/13.jpg)
Overview
12
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabd
aabd
acd
…
Fitting Traces
Precision
Metrics
Alignment
![Page 14: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/14.jpg)
Implementation in ProM 6
• Package: “ETConformance”
13
![Page 15: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/15.jpg)
Experiment settings
14
Event Log
Flower (F)
Precise (P)
Flower’ (F’)
Precise’ (P’)
Duplicate Log Combined Log
Precise-Precise (PP’)
Precise-Flower
(PF’)
Flower -Precise
(FP’)
Flower-Flower
(FF’)
![Page 16: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/16.jpg)
Experiment 1: Measuring precision stability
15
Perfectly fitting logs and models Non-fitting logs and models
P FP’ PF’ PP’P FP’ PF’ PP’
![Page 17: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/17.jpg)
Experiment 2: Sensitivity to unfitting trace
•
16
Model P
0.80
0.85
0.90
0.95
1.00
0 1 2 3 4
0.00
0.20
0.40
0.60
0.80
1.00
0 1 2 3 4
Pre
cisi
on
Number of Removed Events
Noise Sensitivity - P
ETC
1-Align
All-Align
Pre
cis
ion
#Removed events
0.00
0.20
0.40
0.60
0.80
1.00
0 1 2 3 4
Model PF’
Pre
cis
ion
#Removed events
0.00
0.10
0.20
0.30
0.40
0 1 2 3 4
Pre
cis
ion
#Removed events
Model FP’
0.90
0.93
0.95
0.98
1.00
0 1 2 3 4
Pre
cis
ion
#Removed events
Model PP’
![Page 18: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/18.jpg)
Experiment 3: Real life cases
•
17
![Page 19: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/19.jpg)
Overview
18
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabd
aabd
acd
…
Fitting Traces
Precision
Metrics
Alignment
![Page 20: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/20.jpg)
Conclusion and Future Work
•
19
![Page 21: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/21.jpg)
Questions
20
![Page 22: Alignment based Precision Checking](https://reader031.vdocument.in/reader031/viewer/2022020218/559708df1a28ab9f2f8b474c/html5/thumbnails/22.jpg)
Thanks!
21