phd defense november 2015
TRANSCRIPT
1/15www.janclaes.info
Jan ClaesSupervisors UGent : Geert Poels & Frederik Gailly
Supervisors TU/e : Paul Grefen & Irene Vanderfeesten
Investigating the process of process modeling and its relation to modeling quality
The Role of Structure Serialization
2/15www.janclaes.info
Study 1: visualization
Research Objective 1Build knowledge about
how people create models
Overall objectiveCuriosity-driven
Build knowledge about PPM
Research Objective 2Build knowledge about
relation with quality
Research Objective 3Build knowledge about
structured modeling
Study 3: theorizationStudy 2: exploration
Research Objectives
3/15www.janclaes.info
CREATE_ACTIVITYCREATE_START_EVENTCREATE_END_EVENTCREATE_ANDCREATE_XORCREATE_EDGE
Study 1 – Visualization
MOVE_ACTIVITYMOVE_START_EVENTMOVE_END_EVENTMOVE_ANDMOVE_XOR
DELETE_ACTIVITYDELETE_START_EVENTDELETE-END_EVENTDELETE_ANDDELETE_XORDELETE_EDGE
NAME_ACTIVITYRENAME_ACTIVITYNAME_EDGERENAME_EDGE
4/15www.janclaes.info
CREATE_ACTIVITY CREATE_START_EVENT CREATE_END_EVENT CREATE_AND CREATE_XOR CREATE_EDGE MOVE_ACTIVITY MOVE_START_EVENT MOVE_END_EVENT MOVE_AND MOVE_XOR DELETE_ACTIVITY DELETE_START_EVENT DELETE-END_EVENT DELETE_AND DELETE_XOR DELETE_EDGE NAME_ACTIVITY RENAME_ACTIVITY NAME_EDGE RENAME_EDGE
Study 1 – VisualizationPPMChart
time
mod
el e
lem
ents
5/15www.janclaes.info
FastmodelingSlow
modelingInitialdelayMany
pauzesFewelementsMany
elements No(separate)lay-outing
Quicklay-outingDedicated
lay-outingphase
Continuouslay-outingUnpaired
eventcreation
Pairedevent
creationNo pauzes
SerializationPaired
gatewaycreation
Delayededge
creationChunked
modeling
Study 1 – Visualization
Based on dataset of 357 unique modeling executions
6/15www.janclaes.info
Study 1 – VisualizationEvaluation
Sample of intended users (6 academic researchers) Five extreme examples in PPMChart or Dotted Chart Observe and measure amount, quality, and timing of
insights gained through the visualization Observe and ask about perceived usefulness
Results Perceived as useful More cognitive effective than Dotted Chart
7/15www.janclaes.info
Study 2 – Exploration
Based on dataset of 40 unique modeling executions
Fastmodeling
Slowmodeling
Initialdelay
Manypauzes
Fewelements
Manyelements
No(separate)lay-outing
Quicklay-outing
Dedicatedlay-outing
phase
Continuouslay-outing
Unpairedevent
creation
Pairedevent
creation
No pauzesSerializationPairedgatewaycreation
Delayededge
creation
Chunkedmodeling
8/15www.janclaes.info
Study 2 – ExplorationStructuredness Movement Speed
Based on dataset of 40 unique modeling executions
Fastmodeling
Slowmodeling
Quicklay-outing
Dedicatedlay-outing
phase
Continuouslay-outing
Serialization
Chunkedmodeling
CONJECTURE 1
CONJECTURE 2
CONJECTURE 3
9/15www.janclaes.info
Study 2 – ExplorationConjecture 1: Structured modeling
results in more understandable models
Conjecture 2: A high number of move operations results in less understandable models
Conjecture 3: Slow modeling results in less understandable models
10/15www.janclaes.info
Study 2 – Exploration
Based on dataset of 103 unique modeling executions
T-testt=-2,231
(p=0,028)
T-testt=2,199
(p=0,030)
CONJECTURE 1structuredness
T-testt=-1,984
(p=0,049)
T-testt=0,457
(p=0,648)
T-testt=-2,183
(p=0,031)
T-testt=2,505
(p=0,014)
CONJECTURE 2movement
CONJECTURE 3speed
11/15www.janclaes.info
Study 3 – Theorization
Based on dataset of 103 unique modeling executions
T-testt=-2,231
(p=0,028)
T-testt=2,199
(p=0,030)
CONJECTURE 1structuredness
T-testt=-1,984
(p=0,049)
T-testt=0,457
(p=0,648)
T-testt=-2,183
(p=0,031)
T-testt=2,505
(p=0,014)
CONJECTURE 2movement
CONJECTURE 3speed
WHY?
12/15www.janclaes.info
Study 3 – Theorization
Combined
Flow-oriented Aspect-oriented
Undirected
Based on dataset of 118 unique modeling executions
13/15www.janclaes.info
Study 3 – Theorization
A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B
learning style
degree of serialization
adopted serialization style
field-dependency need for structure
– +
course of intrinsic cognitive load for process modeling phases
course of intrinsic cognitive load for aggregation phases
course of cognitive overload
course of intrinsic cognitive load for strategy building phases
+ + +
serialization style fitstructuredness of serialization– – – –
1 2 3
14/15www.janclaes.info
Study 3 – Theorization
Evaluation of utilityNovelty
(uses existing theories in fundamental new way)Parsimony
(11 constructs, 15 associations)Consistency
(can explain additional observations)Plausibility
(accurate and profound explanation)Credibility
(building blocks are established theories)Transferability
(problem solving in general)
Consistency based on dataset of 143 unique modeling executions
15/15www.janclaes.info
FUTURE WORK
Knowledge gaps
Knowledge contribution Research instrument
Knowledge gap 4How to change one’s modeling strategy?
Knowledge gap 3How should one model
in a specific context?
Study 1: visualizationContributions
A – PPMChartB – 22 patternsC – 13 observations
Study 2: explorationContributionsD – 8 patternsE – 3 conjecturesF – 1 metric
Knowledge gap 1How do people
currently model?
Knowledge gap 2Which strategy is
intrinsically better?
A BC
D E F
H HG I
Study 3: theorizationContributions
G – 5 stylesH1 – 6 observationsH2 – 3 impressionsI – SPMT
I