experimental comparison of sequence and collaboration diagrams in different application domains...

of 37 /37
Experimental Comparison of Sequence and Collaboration Diagrams in Different Application Domains Chanan Glezer, Mark Last, Efrat Nachmany, Peretz Shoval Dept. of Information Systems Engineering Ben-Gurion University, Beer-Sheva, Israel

Author: dominic-park

Post on 28-Dec-2015

212 views

Category:

Documents


0 download

Embed Size (px)

TRANSCRIPT

  • Experimental Comparison of Sequence and Collaboration Diagrams in Different Application Domains Chanan Glezer, Mark Last, Efrat Nachmany, Peretz Shoval

    Dept. of Information Systems EngineeringBen-Gurion University, Beer-Sheva, Israel

  • IntroductionUML is a collection of overlapping modeling techniquesDifficulty in establishing practical guidelines; especially for interaction diagrams: sequence and collaborationAttempts have been made to evaluate the comprehensibility of sequence and collaboration diagramsdid not address the quality of diagrams created by analystsThis work reports the findings from a comparative experiment on comprehensibility and quality of sequence and collaboration diagrams in two domain: Management Information System (MIS)Real-Time system (RT)

  • Sequence diagramsIntroduced into OO modeling by Objectory (Jacobsen et al. 1992)Depict an explicit sequence of stimuli messages exchanged among object instances participating in the interaction Include lifelines for the instances used to portray the temporal dimension of the modeled pattern

  • :SirenControl Box:Control Detector: BreachCorridor:Breach DetectorSecurity Guard:Activate ()Detector_operating:=Is_activated ()Control Detector: FireCorridor:Fire Detector

    Burglar:Monitoring ()Monitoring () Until [Detector_operating= True]* Activate ()Breach_occurs () Detector_operating:=Is_activated ()

    Until [Detector_operating= True]*

    Activate (Alarm_Period= Burglar){15 Minutes}Activate (Alarm_Period= Burglar)

    {30 minutes}Activate (Alarm_Period= Breach)

    Fire_occurs () Sequence Diagram Security Monitoring System

  • Collaboration diagramsHave been introduced by Wirfs-Brock et al. (1990) Directed graph where nodes represent communicating entities and edges representing communications Edges are numbered to represent the order of communications

  • Borrowing Desk :Title Borrower: Order Borrow: Item: 1. Locate Title () 3. Identify Borrower () . 1.1 Locate (name) 2.1 Locate Title (name) . 3.1 Locate (name) 3.2) ( 3.3>>
  • Security guard:Control Box:Corridor:Breach Detector

    Control Detector: FireCorridor:Fire Detector

    1. turn on ()1.1 Monitor A ()1.1 Monitor B ()2.1Breach A () 3.1 Fire ignition ()

    1.1.1 Detector_operating:= AIs_activated ()Until [Detector_operating=True]*

    1.2 Activate (time=bip) 2.2 Activate A (time=breach) {15 minutes}

    3.2 Activate B (time=fire){30 minutes}

    2. Activate ()Control Detector: BreachSiren:1.1.1 Detector_operating:= BIs_activated ()Until [Detector_operating=True]*Collaboration Diagram Security Monitoring System

  • Collaboration vs. Sequence Diagrams Both express similar information Informational equivalence But they are not Computational equivalent Sequence diagrams emphasize the time dimension; assumed to depict better the order of events. Collaboration diagrams assumed to portray better the interactions among objects. Collaboration diagrams are claimed to enable better modeling of complex branching and concurrent activities of multiple processes. But collaboration diagrams do not capture the temporal dimension; the relative order of messages exchanged between objects needs to be enumerated.

  • Related Experimental ComparisonsOtero and Dolado (2002) Subjects: 18 students of InformaticsTask: analyzed three types of diagrams: Sequence, Collaboration and State-chartsDomains: Telephone - real-time non-reactive (synchronous) systemLibrary - Management Information System (MIS) Digital Dictaphone - real time reactive (asynchronous) systemDepended variables:Comprehensibility: number of correct answers on multiple-choice questionsTime to answer the questions

  • Otero and Dolado (2004)First part - repeating the early study from 2002same basic design, diagrams types & design documentsadded independent variable Group to eliminated the effect of learning caused in the 1st experiment. Main Results: Real-Time reactive and non-reactive system: sequence diagram are easier to comprehend than collaboration diagramMIS no difference

  • Otero and Dolado (2004) cont.Second part: Comprehension of pairs of diagramsTask: Analyzed comprehension of pairs: collaboration & state-chartsequence & state-chartsequence & collaborationDomains: Parking system - Real-time non-reactive (synchronous)Insurance system - MIS Main Results: Regardless to the application domain, the most comprehensive combination is sequence diagrams + state-charts.

  • Otero and Dolado (2002 & 2004) - Summary

    comprehension+-

  • Research ObjectiveTo fill the research gaps on these interaction diagrams: compare both Comprehension and Quality, i.e. correctness of diagrams constructed by analysts. Quality was not addressed by Otero & Tolado.The research questions: Are there differences between sequence and collaboration diagrams, in terms of quality and comprehension, and with respect to RT and MIS systems?

  • Hypotheses- 2 types of diagrams x 2 types of systems x 3 variables = 12 hypotheses/tests- For each test, the Null-hypothesis is equality (no difference)

  • Experimental Design Interaction DiagramsSequenceCollaborationApplication DomainsMIS - LibraryReal Time - Security Monitoring SystemTasks

    ConstructionSubjects

    G1G2G3G4Time to Comprehend Perceived ComprehensibilityComprehensibility Score Construction ScorePerceived Ease of Construction Time to Construct Comprehensibility Quality of ConstructionComprehensionIndependent variablesControlled variablesDependent variables

  • Experimental Design - detailsDependent variables (3 variables in each experiment):C score for questions on diagrams comprehensionQ score for quality of an interaction diagram constructed Tc time spent on diagram comprehensionTq time spent on diagram constructionPc subjective ranking of comprehensibility given by a subject to a certain diagram Pq subjective ranking of ease of construction given by a subject to a certain diagram

  • Independent variables Two type of interaction diagram used: sequence diagramcollaboration diagramTwo type of application designed:RT: a Security Monitoring system, representing a real-time reactive system involving time dimension and concurrencyMIS: a Library system, representing a management information system

  • Controlled Variables:Tasks: all subjects in each experiment (separately) performed the same task: In the comprehension experiment: answer a questionnaire consisting of 5 multiple-choice questions which measure the comprehension of a given diagram.In the construction experiment: construct a diagram based on a brief narrative specification.

    Subjects 76 students of Information Systems EngineeringSame courses of studyAll studied UML and the interaction diagrams

  • The ExperimentMid-term exam in the course OO Analysis & DesignTheoretical material and practical examples taken from a textbook (Maciaszek, L. 2001, Requirements Analysis & System Design)In the 1st experiment (Comprehension) each subject received a class diagram, a use case diagram, one interaction diagram (sequence or collaboration) and a questionnaire with 5 questions. In the 2nd experiment (Construction) each subject received a brief narrative specification of a system, a class diagram and a use case diagram of one of the systems (MIS or RT). The assignment was to create a diagram (sequence or collaboration) of the system. Subjects assigned randomly to the 4 cells ( 2 system by 2 diagrams). Each subject performed 2 different tasks: a) comprehension of one type of diagram of one system; b) construction of the other type of diagram of the other system.

  • Factorial Design: Assignment of Groups to Treatments

  • Grading of comprehension and qualityComprehension: Each correct answer to a multiple-choice question received one point (no points deducted for a wrong answer).Quality: a number of points deducted for each mistake - depending on the severity of the mistake and the components of each diagram. The number of points deduces for each mistake was 2, 4, 6, 8, or 10. (e.g. totally omitting an object -10 points)

  • ResultsAnalysis Strategytwo-sided t-testsIndependence: each test measured one hypothesisThe min. significance level for rejecting a null hypothesis was = 0.05

  • Comprehension of DiagramsNo significant difference in comprehension between the two diagram types. T-Test of Diagrams Comprehension for MIST-Test of Diagrams Comprehension for Real-Time SystemCollaboration diagrams are significantly more comprehensible.

  • Quality of Diagram Construction

    T-test of Diagram Quality - MIST-test of Diagram Quality Real Time systemCollaboration diagrams are significantly of better quality than sequence diagrams. No significant difference in quality between the two diagram types

  • Time for Diagram Comprehension T-Test of Comprehension Time for MISNo significant difference between the average times spent on comprehension of each diagram type. T-Test of Comprehension Time for Real-Time SystemNo significant difference between the average times spent on comprehension of each diagram type.

  • Time of Diagram Construction

    T-Test of Time of Construction for MIST-Test of Time of Construction for Real-Time systemNo significant difference between the average times spent on construction of the two diagram types. No significant difference between the average times spent on construction of the two diagram types.

  • Perceived Comprehensibility In the post-test questionnaire, each subject expressed the perceived comprehensibility of the diagram analyzed, using a 1-5 Likert scale. (1 = very comprehensive; 5 = absolutely incomprehensible) T-Test of Perceived Comprehensibility for MIST-Test of Perceived Comprehensibility for Real-Time systemResults for both MIS and RT: No significant difference between the perceived comprehensibility of diagrams.

  • Perceived Ease of Construction In the post-test questionnaire, each subject expressed the perceived ease of construction of the diagram he/she analyzed, using a 1-5 Likert scale. (1 = very easy to construct; 5 = extremely difficult to construct )T-Test of Perceived Ease of Construction for MIST-Test of Perceived Ease of Construction for Real-Time systemResults for both MIS and RT: No significant difference between the perceived ease of construction of diagrams.

  • Summary of the Results= means no significant difference

  • Analysis of ResultsComprehension of DiagramsOur finding contradict the finding of Otero & Dolado who found that sequence diagrams are easier to comprehend in synchronous real-time systems. Our scores of MIS were higher than of RT system:Our participants (ISE students) lack adequate skills for RT systemPerhaps the MIS was easier than the RT systemQuality of Diagram ConstructionCollaboration diagrams are significantly better than sequence diagrams in the case of MISNo significant differences on all other variables.

  • Limitations Small/simple tasks: comprehend only one diagram; construct only one diagram.Only two applications (one MIS; one RT system). Subjects were students, not experienced analysts.Students of ISE are not familiar enough with Real-Time systems.Comprehension based on a multiple-choice questionnaire.Grading scheme of quality may not be valid.

  • Further Research Repeat experiment with a more experienced subjects (particularly experienced analysts) More complex tasks of comprehension and construction (many diagrams)Various types of applications. Compare with other (on-UML) diagram types, e.g. pseudo-code; flowcharts.

    The End

  • Appendix A: Diagram Comprehension Questionnaire - Library MIS The system provides loaning services for items (journal and book titles). Specifically, it handles the following tasks: issuing and canceling a title reservation, loaning an item, returning and item and maintenance (adding/removing items from catalogue, updating borrower's status, and adding new titles). Based on the provided use case, class and sequence/collaboration diagrams, answer the following questions: 1. In order to borrow a book or journal item, the following pre-condition must hold:a. An order exists in the systemb. No order exists; however, the title and item are in the libraryc. The title and item are both in the library, whether an order exists or not d. None of the above answers is true; the correct answer cannot be determined from the diagrams provided 2. According to the diagrams, what are the possible reasons for an order being deleted?a. The 30-day limit has expired and deletion is automaticb. The ordered book or journal is on loanc. Both A and B are correctd. None of the above 3. What is the possible number of orders for a book or journal?a. Maximum 10b. 1-10c. Not limitedd. None of the above4. Whenever an order is deleted, the following object is erased:a. Titleb. Itemc. Both title and itemd. None of the above5. Whenever a client, either human or a different library, wishes to borrow a book or journal which is still on loan, the following happens?a. The system checks for additional available items of the requested title b. The system places an order for the requested book or magazine c. An extant order for the book or journal is deletedd. None of the above answers is true; the correct answer cannot be determined from the diagrams provided

  • Appendix B: Diagram Comprehension Questionnaire - Security Monitoring System The system monitors a set of detectors, providing alerts in the event of a fire or a breach.Based on the provided use case, class diagram and sequence/collaboration diagram, answer the following questions: 1. How is it possible to verify that the system is active?a. According to the control boxb. The siren beepsc. The variable Detector_activated=Trued. None of the above answers is true as the correct answer cannot be determined from the diagrams provided 2. How is it possible to know that a fire was ignited?a. The variable Detector_activated=Trueb. The siren beeps for a period of 15 secondsc. The siren beeps for period of 30 secondsd. Both A and C are correct. 3. The sequence diagram describes a situation where?a. A burglar activates the detectorb. A fire activates the detectorc. Both A and B are correctd. None of the above answers is true; the correct answer cannot be determined from the diagrams provided 4. What will happen in the event of a concurrent fire and breach?a. The siren will beep for a period of 15 secondsb. The siren will beep for a period of 30 secondsc. The siren will beep for a period of 45 secondsd. None of the above answers is true; the correct answer cannot be determined from the diagrams provided5. What will is the difference between the control object of the breach-detector and the control object of the fire detector?a. The relationships of the above objects are different b. The number of detectors is different c. No difference between the breach detector and fire detectord. None of the above answers is true; the correct answer cannot be determined from the diagrams provided

  • Appendix C: Diagram Construction TasksSecurity Monitoring SystemThe security monitoring system controls a set of detectors capable of alerting a security guard in the case of a fire or a breach. In the event one of the detectors is activated, the siren is turned on. The monitoring system is activated and deactivated using a key. The system's hardware comprises of the following components: control box, siren, fire and breach detectors. The security guard activates the system and a beep sound signals that the system has been successfully activated. The control box monitors the fire and breach detectors. If a fire is detected the siren is turned on for 15 minutes and in case a breach is detected the siren is turned on for 30 minutes.Based on the following use case and class diagrams (excluding methods), you are requested to construct a collaboration/sequence diagram for the objects participating in the "activate system" scenario. You should specify a scenario where the security guard activates the system and a burglar activates the detector. The diagram should include: class names of the participating objects, relationships among objects and directed messages exchanges among the objects during the scenario.

  • Appendix D: Diagram Construction Tasks

    Library MISThe Library system provides loaning services for bibliographic items (journal and book titles). Specifically, it handles the following tasks: issuing and canceling a title reservation, loaning an item, returning an item and maintenance (adding/removing items from catalogue, updating borrower's status, and adding new titles). To issue an order, the identification data of the borrower and the journal or book item need to be provided. When loaning a title, the librarian first locates the reserved title and deletes the reservation. Next, the item's status is updates and handed to the borrower. Returning an item is performed by locating the item in the catalogue and updating its status. Based on the following use case and class diagrams (excluding methods), you are requested to construct a collaboration/sequence diagram for the objects participating in the "borrow title" scenario. The diagram should include class names of the participating objects, relationships among objects and directed messages exchanges among the objects during the scenario