Experimental Comparison of Sequence and Collaboration Diagrams
in Different Application Domains
Chanan Glezer, Mark Last, Efrat Nachmany, Peretz Shoval
Dept. of Information Systems EngineeringBen-Gurion University, Beer-Sheva, Israel
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Introduction UML is a collection of overlapping modeling techniques
Difficulty in establishing practical guidelines; especially for interaction diagrams: sequence and collaboration
Attempts have been made to evaluate the comprehensibility of sequence and collaboration diagrams did not address the quality of diagrams created by analysts
This work reports the findings from a comparative experiment on comprehensibility and quality of sequence and collaboration diagrams in two domain: Management Information System (MIS) Real-Time system (RT)
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Sequence diagrams
Introduced into OO modeling by Objectory (Jacobsen et al. 1992)
Depict an explicit sequence of stimuli messages exchanged among object instances participating in the interaction
Include lifelines for the instances used to portray the temporal dimension of the modeled pattern
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Borrowing Desk:
Title: Borrower: Order: Borrow:
Locate Title)(
Locate )Name(
Locate Item)(
Identify Borrower )(
Locate Title )Name(
Locate )Item(
Locate )Borrower(
<<Erase<< )Order Exists(
<<New<< )Item, Borrower(
Item:
Librarian
Sequence Diagram – Library MIS
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
:Siren Control Box:
Control Detector: Breach
Corridor:Breach DetectorSecurity
Guard:Activate)(
Detector_operating=:Is_activated)(
Control Detector: Fire Corridor:Fire
Detector
Burglar:Monitoring)(
Monitoring )(
Until [Detector_operating= True] *
Activate)(
Breach_occurs)(
Detector_operating=:Is_activated)(
Until [Detector_operating= True]*
Activate )Alarm_Period= Burglar(
{15 Minutes}
Activate )Alarm_Period= Burglar(
{30 minutes}
Activate )Alarm_Period= Breach(
Fire_occurs )(
Sequence Diagram – Security Monitoring System
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Collaboration diagrams
Have been introduced by Wirfs-Brock et al. (1990)
Directed graph where nodes represent communicating entities and edges representing communications
Edges are numbered to represent the order of communications
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Borrowing Desk
:Title
Borrower:
Order
Borrow:
Item:
1 .Locate Title)(
3. Identify Borrower )( . 1.1 Locate )name(
2.1 Locate Title )name(
. 3.1 Locate )name(
מצא )מידע על משאיל( 3.2
[ישנה הזמנה] >>מחק<< 3.3
3.4 .)Borrower( <New Item>
. 3.2 Locate )Borrower(
3.3 .[Order Exists]>Erase<
2. Locate Item )(
Librarian
Collaboration Diagram- Library MIS
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Security
guard:
Control Box:
Corridor:
Breach Detector
Control Detector: Fire
Corridor:
Fire Detector
1 .turn on)(
1.1 Monitor A)( 1.1 Monitor B)(
2.1Breach A)(
3.1 Fire ignition)(
1.1.1 Detector_operating:= AIs_activated)(
Until [Detector_operating=True]*
1.2 Activate )time=bip(
2.2 Activate A )time=breach( {15 minutes}
3.2 Activate B )time=fire({30 minutes}
2 .Activate)(
Control Detector: Breach
Siren:
1.1.1 Detector_operating:= BIs_activated)(
Until [Detector_operating=True]*
Collaboration Diagram – Security Monitoring System
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Collaboration vs. Sequence Diagrams
• Both express similar information – Informational equivalence
• But they are not Computational equivalent
• Sequence diagrams emphasize the time dimension; assumed to depict better the order of events.
• Collaboration diagrams assumed to portray better the interactions among objects.
• Collaboration diagrams are claimed to enable better modeling of complex branching and concurrent activities of multiple processes.
• But collaboration diagrams do not capture the temporal dimension; the relative order of messages exchanged between objects needs to be enumerated.
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Related Experimental Comparisons Otero and Dolado (2002)Otero and Dolado (2002)
Subjects: 18 students of Informatics Task: analyzed three types of diagrams:
Sequence, Collaboration and State-charts Domains:
Telephone - real-time non-reactive (synchronous) system Library - Management Information System (MIS) Digital Dictaphone - real time reactive (asynchronous)
system Depended variables:
Comprehensibility: number of correct answers on multiple-choice questions
Time to answer the questions
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Otero and Dolado (2004)Otero and Dolado (2004) First part - repeating the early study from 2002
same basic design, diagrams types & design documents added independent variable “Group” to eliminated the
effect of learning caused in the 1st experiment.
Main Results: Real-Time reactive and non-reactive system:
sequence diagramsequence diagram are easier to comprehend than collaboration diagram
MIS – no difference
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Otero and Dolado (2004) – cont.
Second part: Comprehension of pairs of diagrams Task: Analyzed comprehension of pairs:
collaboration & state-chart sequence & state-chart sequence & collaboration
Domains: Parking system - Real-time non-reactive (synchronous) Insurance system - MIS
Main Results:
Regardless to the application domain, the most comprehensive combination is sequence diagrams + state-charts.
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Otero and Dolado (2002 & 2004) - Summary
MISLibrarySeque.StateCollab.
InsuranceSeque.-State =Collab.-StateSeque.-Collab.
TelephoneCollab.SequenceState
ParkingCollab.-State =Seque.-StateSeque.-Collab.
Real TimeDictaphoneStateSequenceCollab.
comprehension+ -
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Research ObjectiveResearch Objective
To fill the research gaps on these interaction diagrams: compare both Comprehension and Quality, i.e.
correctness of diagrams constructed by analysts. Quality was not addressed by Otero & Tolado.
The research questions:
Are there differences between sequence and collaboration diagrams, in terms of quality and comprehension, and with respect to RT and MIS systems?
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Hypotheses- 2 types of diagrams x 2 types of systems x 3 variables = 12 hypotheses/tests- For each test, the Null-hypothesis is equality (no difference)
Dependent VariableMIS RT system
CScore for diagram comprehension
Ha0:µc, MIS, col=µc, MIS,
seq Ha1:µc, MIS, col=µc, MIS,
seq
Hb0:µc, RT, col=µc, RT,
seq Hb1:µc, RT, col=µc, RT,
seq
QScore for quality of diagram constructed
Hc0:µq, MIS, col=µq, MIS,
seq
Hc1:µq, MIS, col=µq, MIS,
seq
Hd0:µq, RT, col=µq, RT,
seq Hd1:µq, RT, seq=µq, RT,
col
TcTime (minutes) spent on diagram comprehension
He0:µtc, MIS, col=µtc,
MIS, seq He1:µtc, MIS, col=µtc,
MIS, seq
Hf0:µtc, RT, col=µtc, RT,
seq Hf1:µtc, RT, seq=µtc, RT,
col
TqTime spent on diagram construction
Hg0:µtq, MIS, col=µtq,
MIS, seq Hg1:µtq, MIS, col=µtq,
MIS, seq
Hh0:µtq, RT, col=µtq, RT,
seq Hh1:µtq, RT, seq=µtq,
RT, col
PcPerceived comprehensibility of each diagram type
Hi0:µpc, MIS, col=µpc,
MIS, seq
Hi1:µpc, MIS, col=µpc,
MIS, seq
Hj0:µpc, RT, col=µpc, RT,
seq
Hj1:µpc, RT, seq=µpc, RT,
col
PqPerceived ease of construction of each certain diagram type
Hk0:µpq, MIS, col=µpq,
MIS, seq
Hk1:µpq, MIS, col=µpq,
MIS, seq
Hl0:µpq, RT, col=µpq, RT,
seq
Hl1:µpq, RT, seq=µpq, RT,
col
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Experimental Design
Interaction Diagrams
Sequence Collaboration
Application Domains
MIS - LibraryReal Time - Security Monitoring System
Tasks
Construction
Subjects
G1 G2 G3 G4
Tim
e to
C
ompr
ehen
d
Per
ceiv
ed
Com
preh
ensi
bili
tyC
ompr
ehen
sibi
lity
S
core
C
onst
ruct
ion
Sco
reP
erce
ived
E
ase
of C
onst
ruct
ion
T
ime
to C
onst
ruct
Comprehensibility
Quality of Construction
Comprehension
Independent variables
Controlled variables
Dependent variables
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Experimental Design - details
Dependent variables (3 variables in each experiment):
C – score for questions on diagrams comprehension
Q – score for quality of an interaction diagram constructed
Tc – time spent on diagram comprehension
Tq – time spent on diagram construction
Pc – subjective ranking of comprehensibility given by a subject to a certain diagram
Pq – subjective ranking of ease of construction given by a subject to a certain diagram
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Independent variables
Two type of interaction diagram used: sequence diagram collaboration diagram
Two type of application designed: RT: a Security Monitoring system, representing a real-
time reactive system involving time dimension and concurrency
MIS: a Library system, representing a management information system
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Controlled Variables: Tasks: all subjects in each experiment (separately) performed the
same task: In the comprehension experiment: answer a questionnaire
consisting of 5 multiple-choice questions which measure the comprehension of a given diagram.
In the construction experiment: construct a diagram based on a brief narrative specification.
Subjects 76 students of Information Systems Engineering Same courses of study All studied UML and the interaction diagrams
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
The Experiment Mid-term exam in the course OO Analysis & Design Theoretical material and practical examples taken from a textbook
(Maciaszek, L. 2001, Requirements Analysis & System Design)
In the 1st experiment (Comprehension) each subject received a class diagram, a use case diagram, one interaction diagram (sequence or collaboration) and a questionnaire with 5 questions.
In the 2nd experiment (Construction) each subject received a brief narrative specification of a system, a class diagram and a use case diagram of one of the systems (MIS or RT). The assignment was to create a diagram (sequence or collaboration) of the system.
Subjects assigned randomly to the 4 cells ( 2 system by 2 diagrams). Each subject performed 2 different tasks: a) comprehension of one type
of diagram of one system; b) construction of the other type of diagram of the other system.
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Factorial Design: Assignment of Groups to Treatments
GroupSizeSession 1:Comprehension
Session 2:Construction
119RT systemSequence
MIS systemCollaboration
219RT systemCollaboration
MIS systemSequence
320MIS systemSequence Diagram
RT systemCollaboration
418MIS systemCollaboration
RT systemSequence
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Grading of comprehension and quality
Comprehension:
Each correct answer to a multiple-choice question received one point (no points deducted for a wrong answer).
Quality:
a number of points deducted for each mistake - depending on the severity of the mistake and the components of each diagram. The number of points deduces for each mistake was 2, 4, 6, 8, or 10. (e.g. totally omitting an object -10 points)
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Results Analysis Strategy
two-sided t-tests Independence: each test measured one hypothesis The min. significance level for rejecting a null hypothesis
was α = 0.05
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Comprehension of Diagrams
Diagram TypeMeanStandard Deviation
tP valuePowerEffect Size
Sequence4.2500.9660.1030.9190.1560.143
Collaboration4.2220.646
No significant difference in comprehension between the two diagram types.
T-Test of Diagrams Comprehension for MIS
Diagram TypeMeanStandard Deviation
tP valuePowerEffect Size
Sequence2.3151.2492.3450.0251.0003.227
Collaboration3.3151.376
T-Test of Diagrams Comprehension for Real-Time System
Collaboration diagrams are significantly more comprehensible.
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Quality of Diagram ConstructionT-test of Diagram Quality - MIS
Diagram TypeMeanStandard Deviation
tP valuePowerEffect Size
Sequence46.4735.3372.0430.0481.0002.813
Collaboration50.8427.639
Diagram TypeMeanStandard Deviation
tP valuePowerEffect Size
Sequence43.3887.1540.9090.3690.9851.258
Collaboration41.1008.245
T-test of Diagram Quality – Real Time system
Collaboration diagrams are significantly of better quality than sequence diagrams.
No significant difference in quality between the two diagram types
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Time for Diagram Comprehension
Diagram TypeMean (min)Standard Dev.
tP valuePowerEffect Size
Sequence28.4506.3280.9620.3421.0001.313
Collaboration26.0009.235
T-Test of Comprehension Time for MIS
No significant difference between the average times spent on comprehension of each diagram type.
T-Test of Comprehension Time for Real-Time System
Diagram TypeMean (min)Standard Dev.tP valuePowerEffect Size
Sequence22.4737.8900.1990.8430.4470.274
Collaboration23.0529.907
No significant difference between the average times spent on comprehension of each diagram type.
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Time of Diagram Construction
Diagram TypeMean (min.)
Standard Deviation
tP valuePowerEffect Size
Sequence56.81219.6130.1010.2991.0001.584
Collaboration65.12826.157
Diagram TypeMean (minutes)
Standard Deviation
tP valuePowerEffect Size
Sequence42.38918.7280.9640.362 1.0001.488
Collaboration49.13618.249
T-Test of Time of Construction for MIS
T-Test of Time of Construction for Real-Time system
No significant difference between the average times spent on construction of the two diagram types.
No significant difference between the average times spent on construction of the two diagram types.
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Perceived Comprehensibility In the post-test questionnaire, each subject expressed the perceived comprehensibility of the diagram analyzed, using a 1-5 Likert scale.
(1 = very comprehensive; 5 = absolutely incomprehensible)
Diagram TypeMeanStandard Deviation
tP valuePowerEffect Size
Sequence2.8891.1310.3980.6930.7430.591
Collaboration2.7331.099
Diagram TypeMeanStandard Deviation
tP valuePowerEffect Size
Sequence2.7781.3080.5380.5940.6550.750
Collaboration3.0001.201
T-Test of Perceived Comprehensibility for MIS
T-Test of Perceived Comprehensibility for Real-Time system
Results for both MIS and RT: No significant difference between the perceived comprehensibility of diagrams.
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Perceived Ease of Construction
In the post-test questionnaire, each subject expressed the perceived ease of construction of the diagram he/she analyzed, using a 1-5 Likert scale.
(1 = very easy to construct; 5 = extremely difficult to construct )
Diagram TypeMeanStandard Deviation
tP valuePowerEffect Size
Sequence3.3680.8950.6990.4890.8990.977
Collaboration3.1660.857
Diagram TypeMeanStandard Deviation
tP valuePowerEffect Size
Sequence2.5330.8331.1210.2711.0001.674
Collaboration2.8880.963
T-Test of Perceived Ease of Construction for MIS
T-Test of Perceived Ease of Construction for Real-Time system
Results for both MIS and RT: No significant difference between the perceived ease of construction of diagrams.
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Summary of the ResultsDepended Variable MISReal-Time
Comprehension
Score=Collaboration
Time ==
Perceived comprehensibility
==
Quality of Construction
ScoreCollaboration=
Time==
Perceived ease of construction
==
= means no significant difference
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Analysis of Results Comprehension of Diagrams
Our finding contradict the finding of Otero & Dolado who found that sequence diagrams are easier to comprehend in synchronous real-time systems.
Our scores of MIS were higher than of RT system: Our participants (ISE students) lack adequate skills for RT
system Perhaps the MIS was easier than the RT system
Quality of Diagram Construction Collaboration diagrams are significantly better than sequence
diagrams in the case of MIS
No significant differences on all other variables.
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Limitations Small/simple tasks:
comprehend only one diagram;
construct only one diagram.
Only two applications (one MIS; one RT system).
Subjects were students, not experienced analysts.
Students of ISE are not familiar enough with Real-Time systems.
Comprehension based on a multiple-choice questionnaire.
Grading scheme of quality may not be valid.
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Repeat experiment with a more experienced subjects (particularly experienced analysts)
More complex tasks of comprehension and construction (many diagrams)
Various types of applications.
Compare with other (on-UML) diagram types, e.g. pseudo-code; flowcharts.
The EndThe End
Further Research
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Appendix A: Diagram Comprehension Questionnaire - Library MIS The system provides loaning services for items (journal and book titles). Specifically, it handles the following tasks: issuing and canceling a title reservation, loaning an item, returning and item and maintenance (adding/removing items from catalogue, updating borrower's status, and adding new titles). Based on the provided use case, class and sequence/collaboration diagrams,
answer the following questions :1 .In order to borrow a book or journal item, the following pre-condition must hold:
a. An order exists in the systemb. No order exists; however, the title and item are in the libraryc. The title and item are both in the library, whether an order exists or not d. None of the above answers is true; the correct answer cannot be determined from the diagrams provided
2. According to the diagrams, what are the possible reasons for an order being deleted?a. The 30-day limit has expired and deletion is automaticb. The ordered book or journal is on loanc. Both A and B are correctd. None of the above
3. What is the possible number of orders for a book or journal?a. Maximum 10b. 1-10c. Not limitedd. None of the above
4. Whenever an order is deleted, the following object is erased:a. Titleb. Itemc. Both title and itemd. None of the above
5. Whenever a client, either human or a different library, wishes to borrow a book or journal which is still on loan, the following happens?
a. The system checks for additional available items of the requested title b. The system places an order for the requested book or magazine c. An extant order for the book or journal is deletedd. None of the above answers is true; the correct answer cannot be determined from the diagrams provided
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Appendix B: Diagram Comprehension Questionnaire - Security Monitoring System The system monitors a set of detectors, providing alerts in the event of a fire or a breach.Based on the provided use case, class diagram and sequence/collaboration diagram, answer the following questions: 1. How is it possible to verify that the system is active?
a. According to the control boxb. The siren beepsc. The variable Detector_activated=Trued. None of the above answers is true as the correct answer cannot be determined from the diagrams provided
2. How is it possible to know that a fire was ignited?a. The variable Detector_activated=Trueb. The siren beeps for a period of 15 secondsc. The siren beeps for period of 30 secondsd. Both A and C are correct.
3. The sequence diagram describes a situation where?a. A burglar activates the detectorb. A fire activates the detectorc. Both A and B are correctd. None of the above answers is true; the correct answer cannot be determined from the diagrams provided
4. What will happen in the event of a concurrent fire and breach?a. The siren will beep for a period of 15 secondsb. The siren will beep for a period of 30 secondsc. The siren will beep for a period of 45 secondsd. None of the above answers is true; the correct answer cannot be determined from the diagrams provided
5. What will is the difference between the control object of the breach-detector and the control object of the fire detector?a. The relationships of the above objects are different b. The number of detectors is different c. No difference between the breach detector and fire detectord. None of the above answers is true; the correct answer cannot be determined from the diagrams provided
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Appendix C: Diagram Construction Tasks
Security Monitoring SystemThe security monitoring system controls a set of detectors capable of alerting a security guard in the case of a fire or a breach. In the event one of the detectors is activated, the siren is turned on. The monitoring system is activated and
deactivated using a key .The system's hardware comprises of the following components: control box, siren, fire and breach detectors. The security guard activates the system and a beep sound signals that the system has been successfully activated. The control box monitors the fire and breach detectors. If a fire is detected the siren is turned on for 15 minutes and in case a breach is detected the siren is turned on for 30 minutes.
Based on the following use case and class diagrams (excluding methods), you are requested to construct a collaboration/sequence diagram for the objects participating in the "activate system" scenario. You should specify a scenario where the security guard activates the system and a burglar activates the detector. The diagram should include: class names of the participating objects, relationships among objects and directed messages exchanges among the objects during the scenario.
EMMSAD-05, Porto, Portugal, June 2005Peretz Shoval: [email protected]
Appendix D: Diagram Construction Tasks
Library MISThe Library system provides loaning services for bibliographic items (journal and book titles). Specifically, it handles the following tasks: issuing and canceling a title reservation, loaning an item, returning an item and maintenance (adding/removing items from catalogue, updating borrower's status, and adding
new titles) .To issue an order, the identification data of the borrower and the journal or book item need to be provided. When loaning a title, the librarian first locates the reserved title and deletes the reservation. Next, the item's status is updates and handed to the borrower. Returning an item is performed by locating the item
in the catalogue and updating its status .Based on the following use case and class diagrams (excluding methods), you are requested to construct a collaboration/sequence diagram for the objects participating in the "borrow title" scenario. The diagram should include class names of the participating objects, relationships among objects and directed messages exchanges among the objects during the scenario