lecture 17 architecture tradeoff analysis method atam topics evaluating software architectures what...
Post on 20-Dec-2015
214 Views
Preview:
TRANSCRIPT
Lecture 17Architecture Tradeoff Analysis Method
ATAM
Lecture 17Architecture Tradeoff Analysis Method
ATAM
TopicsTopics Evaluating Software Architectures What an evaluation should tell What can be examined, What Can’t ATAM
Next Time: Case Study: the Nightingale SystemNext Time: Case Study: the Nightingale System
Ref: The “Evaluating Architectures” book and Chap 11Ref: The “Evaluating Architectures” book and Chap 11
October 22, 2003
CSCE 742 Software Architectures
– 2 – CSCE 742 Fall 03
OverviewOverview
Last TimeLast Time Analysis of Architectures
Why, When, Cost/BenefitsTechniquesProperties of Successful Evaluations
New: Analysis of ArchitecturesNew: Analysis of Architectures Why, When, Cost/Benefits Techniques Properties of Successful Evaluations
Next time: Next time:
References:References:
– 3 – CSCE 742 Fall 03
Last Time Last Time Why evaluate architectures?Why evaluate architectures?
When do we Evaluate?When do we Evaluate?
Cost / Benefits of Evaluation of SACost / Benefits of Evaluation of SA
Non-Financial BenefitsNon-Financial Benefits
EvaluationTechniquesEvaluationTechniques
Active Design Review,” SAAM (SA Analysis Method), ATAM Active Design Review,” SAAM (SA Analysis Method), ATAM (Architecture Tradeoff Analysis Method), CBAM (2001/SEI) – (Architecture Tradeoff Analysis Method), CBAM (2001/SEI) – Cost Benefit Analysis MethodCost Benefit Analysis Method
Planned or Unplanned EvaluationsPlanned or Unplanned Evaluations
Properties of Successful EvaluationsProperties of Successful Evaluations
Results of an EvaluationResults of an Evaluation
Principles behind the Agile ManifestoPrinciples behind the Agile Manifesto
Architecture Tradeoff Analysis Method (ATAM)Architecture Tradeoff Analysis Method (ATAM)
Conceptual Flow ATAMConceptual Flow ATAM
– 4 – CSCE 742 Fall 03
ATAM OverviewATAM Overview
ATAM is based upon a set of attribute-specific ATAM is based upon a set of attribute-specific measures of the systemmeasures of the system
Analytic measures, based upon formal models Analytic measures, based upon formal models Performance Availability
Qualitative measures, based upon formal Qualitative measures, based upon formal inspectionsinspections Modifiability Safety security
– 5 – CSCE 742 Fall 03
ATAM BenefitsATAM Benefits
clarified quality attribute requirements clarified quality attribute requirements
improved architecture documentation improved architecture documentation
documented basis for architectural decisions documented basis for architectural decisions
identified risks early in the life-cycle identified risks early in the life-cycle
increased communication among stakeholders increased communication among stakeholders
– 6 – CSCE 742 Fall 03
Conceptual Flow ATAMConceptual Flow ATAM
Figure taken from http://www.sei.cmu.edu/ata/ata_method.html
– 7 – CSCE 742 Fall 03
For What Qualities can we Evaluate?For What Qualities can we Evaluate?
From the architecture we can’t tell if the resulting From the architecture we can’t tell if the resulting system will meet all of its quality goalssystem will meet all of its quality goals
Why? Why?
Usability Usability largely determined by user interface largely determined by user interface
User interface is typically at lower level than SAUser interface is typically at lower level than SA UMLi http://www.ksl.stanford.edu/people/pp/papers/PinheirodaSilva_ksl_02_04.pdf
ATAM concentrates on evaluating a SA based on ATAM concentrates on evaluating a SA based on certain quality attributes certain quality attributes
– 8 – CSCE 742 Fall 03
ATAM Quality AttributesATAM Quality AttributesPerformancePerformance
ReliabilityReliability
AvailabilityAvailability
SecuritySecurity
ModifiabilityModifiability
PortabilityPortability
FunctionalityFunctionality
Variability – how well the architecture can be expanded Variability – how well the architecture can be expanded to produce new SAs in preplanned ways ( important to produce new SAs in preplanned ways ( important for product lines)for product lines)
SubsetabilitySubsetability
Conceptual integrityConceptual integrity
– 9 – CSCE 742 Fall 03
Non-suitable Quality Attributes (for ATAM)Non-suitable Quality Attributes (for ATAM)
There are some quality attributes that are just to vague There are some quality attributes that are just to vague to be used as the basis for an evaluation.to be used as the basis for an evaluation.
ExamplesExamples
““The system must be robust.”The system must be robust.”
““The system shall be highly modifiable.”The system shall be highly modifiable.”
Quality attributes are evaluated in some contextQuality attributes are evaluated in some context A system is modifiable wrt a specific kind of change. A system is secure wrt a specific kind of threat.
– 10 – CSCE 742 Fall 03
Outputs of an Architecture EvaluationOutputs of an Architecture Evaluation
1.1. Prioritized statement of quality attribute Prioritized statement of quality attribute requirementsrequirements “You've got to be very careful if you don't know where
you're going, because you might not get there.” Yogi Berra Yogi also said "I didn't really say everything I said."
2.2. Mapping of approaches to quality attributesMapping of approaches to quality attributes How the architectural approaches will achieve or fail to
achieve quality attributes Provides some “rationale” for the architecture
3.3. Risks and non-risksRisks and non-risks Risks are potentially problematic architectural decisions
There are more for each specific analysis technique. There are more for each specific analysis technique. We will consider ATAM soon.We will consider ATAM soon.
– 11 – CSCE 742 Fall 03
Documenting Risks and Non-RisksDocumenting Risks and Non-Risks
Documenting risks and non-risks consists of:Documenting risks and non-risks consists of:
An architectural decision (that has not been made)An architectural decision (that has not been made)
A specific quality attribute response being A specific quality attribute response being addressed by that decisionaddressed by that decision
A rationale for the positive or negative effect that the A rationale for the positive or negative effect that the decision has on satisfying the quality attributedecision has on satisfying the quality attribute
– 12 – CSCE 742 Fall 03
Example of a Risk*Example of a Risk*
The rules for writing business logic modules in the The rules for writing business logic modules in the second tier of your three tier client-server system are second tier of your three tier client-server system are not clearly articulated. (decision to be made)not clearly articulated. (decision to be made)
This could result in replication of functionality thereby This could result in replication of functionality thereby compromising modifiability of the third tier (a quality compromising modifiability of the third tier (a quality attribute response and its consequences)attribute response and its consequences)
Unarticulated rules for writing business logic can result Unarticulated rules for writing business logic can result in unintended and undesired coupling of in unintended and undesired coupling of components (rationale for the negative effect)components (rationale for the negative effect)
*Example from: Evaluating Software Architectures: Methods and Case *Example from: Evaluating Software Architectures: Methods and Case Studies by Clements, Kazman and KleinStudies by Clements, Kazman and Klein
– 13 – CSCE 742 Fall 03
Example of a Non-Risk*Example of a Non-Risk*
Assuming message arrival rates of once per second, a Assuming message arrival rates of once per second, a processing time of less than 30 milliseconds and the processing time of less than 30 milliseconds and the existence of one higher priority process (the existence of one higher priority process (the architectural decisions)architectural decisions)
A one-second soft deadline seems appropriate (the A one-second soft deadline seems appropriate (the quality attribute response and its consequence)quality attribute response and its consequence)
Since the arrival rate is bounded and the preemptive Since the arrival rate is bounded and the preemptive effects of higher priority processes and known and effects of higher priority processes and known and can be accommodated (the rationale)can be accommodated (the rationale)
*Example from: Evaluating Software Architectures: Methods and Case *Example from: Evaluating Software Architectures: Methods and Case Studies by Clements, Kazman and KleinStudies by Clements, Kazman and Klein
– 14 – CSCE 742 Fall 03
Participants in ATAMParticipants in ATAM
The evaluation teamThe evaluation team Team leader – Evaluation leader Scenario scribe Proceedings scribe Timekeeper Process observer Process enforcer Questioner
Project Decision makers – people empowered to Project Decision makers – people empowered to speak for the development projectspeak for the development project
Architecture stakeholders – Architecture stakeholders – including developers, testers, …, users, builders of systems interacting with this one
– 15 – CSCE 742 Fall 03
Evaluation Team Roles and ResponsibilitiesEvaluation Team Roles and ResponsibilitiesTeam leaderTeam leader
Sets up evaluation (set evaluation contract) forms team interfaces with client Oversees the writing of the evaluation report
Evaluation leaderEvaluation leader Runs the evaluation Facilitates elicitation of scenarios Administers selection and prioritization of scenarios
process Facilitates evaluation of scenarios against architecture
Scenario scribeScenario scribe Records scenarios on a flip chart Captures (and insists on) agreed wording
Proceedings scribeProceedings scribe
– 16 – CSCE 742 Fall 03
Evaluation Team Roles and ResponsibilitiesEvaluation Team Roles and Responsibilities
Proceedings scribeProceedings scribe Captures proceedings in electronic form Raw scenarios with motivating issues Resolution of each scenario when applied to the architecture Generates a list of adopted scenaios
TimekeeperTimekeeper Helps evaluation leader stay on schedule Controls the time devoted to each scenario
Process observerProcess observer Keeps notes on how the evaluation process could be
improved
Process enforcer – helps the evaluation leader stay “on Process enforcer – helps the evaluation leader stay “on process”process”
Questioner raises issues of architectural interest Questioner raises issues of architectural interest stakeholders may not have thought ofstakeholders may not have thought of
– 17 – CSCE 742 Fall 03
Outputs of the ATAMOutputs of the ATAM
A Concise presentation of the architectureA Concise presentation of the architecture Frequently there is “too much” ATAM will force a “one hour” presentation of the
architecture forcing it to be concise and clear
Articulation of Business GoalsArticulation of Business Goals
Quality requirements in terms of collections of Quality requirements in terms of collections of scenariosscenarios
Mapping of architectural decisions to quality attribute Mapping of architectural decisions to quality attribute requirementsrequirements
A set sensitivity and tradeoff pointsA set sensitivity and tradeoff points Eg. A backup database positively affects reliability
(sensitivity point with respect to reliability) However it negatively affects performance (tradeoff)
– 18 – CSCE 742 Fall 03
More Outputs of the ATAMMore Outputs of the ATAM
A set of risks and non-risksA set of risks and non-risks A risk is an architectural decision that may lead to
undesirable consequences with respect to a stated quality attribute requirement
A non-risk is an architectural decision that, after analysis, is deemed to be safe
Identified risks can form the basis of a “architectural risk mitigation” plan
A set of risk themesA set of risk themes Examine the collection of risks produced looking for themes
that are the result of systematic weaknesses in the architecture
Other outputs – more documentation, rationale, sense Other outputs – more documentation, rationale, sense of community between stakeholders, architect, …of community between stakeholders, architect, …
– 19 – CSCE 742 Fall 03
Phases of the ATAMPhases of the ATAM
PhasePhase ActivityActivity ParticipantsParticipants DurationDuration
00 PreparationPreparation Team leadership/key Team leadership/key project decision makersproject decision makers
Over a few Over a few weeksweeks
11 EvaluationEvaluation Evaluation team and Evaluation team and project decision makersproject decision makers
1 day + hiatus 1 day + hiatus of 2 to 3 weeksof 2 to 3 weeks
22 Evaluation with Evaluation with StakeholdersStakeholders
Ditto + stakeholdersDitto + stakeholders 2 days2 days
33 Follow-upFollow-up Evaluation team and clientEvaluation team and client 1 week1 week
– 20 – CSCE 742 Fall 03
Steps of the Evaluation Phase(s)Steps of the Evaluation Phase(s)
1.1. Present the ATAMPresent the ATAM
2.2. Present Business driversPresent Business drivers
3.3. Present ArchitecturePresent Architecture
4.4. Identify architectural approachesIdentify architectural approaches
5.5. Generate quality attribute utility treeGenerate quality attribute utility tree
6.6. Analyze architectural approachesAnalyze architectural approaches Hiatus and start of phase 2
7.7. Brainstorm and prioritize scenariosBrainstorm and prioritize scenarios
8.8. Analyze architectural approachesAnalyze architectural approaches
9.9. Present resultsPresent results
– 21 – CSCE 742 Fall 03
Phase 0: PartnershipPhase 0: Partnership
Client – someone who can exercise control of the Client – someone who can exercise control of the project whose architecture is being evaluatedproject whose architecture is being evaluated Perhaps a manger Perhaps someone in an organization considering a
purchase
Issues that must be resolved in Phase 0:Issues that must be resolved in Phase 0:
1.1. The client must understand the evaluation method The client must understand the evaluation method and process (give them a book, make them a video)and process (give them a book, make them a video)
2.2. The client should describe the system and The client should describe the system and architecture. A “go/no-go” decision is made here by architecture. A “go/no-go” decision is made here by the evaluation team leader.the evaluation team leader.
3.3. Statement of work is negotiatedStatement of work is negotiated
4.4. Issues of proprietary information resolvedIssues of proprietary information resolved
– 22 – CSCE 742 Fall 03
Phase 0: PreparationPhase 0: Preparation
Forming the evaluation teamForming the evaluation team
Holding an evaluation kickoff meetingHolding an evaluation kickoff meeting
Assignment of rolesAssignment of roles Good idea to not get into ruts; try varying assignments Roles not necessarily one-to-one The minimum size evaluation team is 4 One person can be process observer, timekeeper and
questioner Team leader’s responsibilities are mostly “outside” the
evaluation. He can double up. (Often the evaluation leader.) Questioners should be chosen to represent the spectrum of
expertise in performance, modifiability, …
– 23 – CSCE 742 Fall 03
Phase 1: Activities in Addition to the stepsPhase 1: Activities in Addition to the steps
Organizational meeting of evaluation team and key Organizational meeting of evaluation team and key project personnelproject personnel
Form scheduleForm schedule The right people attend the meetings They are prepared (know what is expected of them) They have the right attitude
Besides carrying out the steps the team needs to Besides carrying out the steps the team needs to gather as much information as possible to determinegather as much information as possible to determine Whether the remainder of the evaluation is feasible Whether more architectural documentation is required Which stakeholders should be present for phase 2
– 24 – CSCE 742 Fall 03
Step 1: Present the ATAMStep 1: Present the ATAM
Evaluation leader presents the ATAM to all participantsEvaluation leader presents the ATAM to all participants
To explain the processTo explain the process
To answer questionsTo answer questions
To set the context and expectations for the processTo set the context and expectations for the process
Using standard presentation discuss ATAM steps in Using standard presentation discuss ATAM steps in brief and outputs of the evaluationbrief and outputs of the evaluation
– 25 – CSCE 742 Fall 03
A Typical ATAM Agenda for Phases 1 and 2A Typical ATAM Agenda for Phases 1 and 2
Figure 3.9 from “Evaluating Software Architectures: Figure 3.9 from “Evaluating Software Architectures: Methods and Case Studies,” by Clements, Kazman Methods and Case Studies,” by Clements, Kazman and Kleinand Klein
– 26 – CSCE 742 Fall 03
Step 2: Present Business DriversStep 2: Present Business Drivers
Everyone needs to understand the primary business Everyone needs to understand the primary business drivers motivating/guiding the development of the drivers motivating/guiding the development of the system system
A project decision maker presents a system overview A project decision maker presents a system overview from the business perspectivefrom the business perspective
The system’s functionalityThe system’s functionality
Relevant constraints: technical, managerial, Relevant constraints: technical, managerial, economic, or politicaleconomic, or political
Business goals Business goals
Major stakeholdersMajor stakeholders
The architectural drivers – the quality attributes that The architectural drivers – the quality attributes that shape the architectureshape the architecture
– 27 – CSCE 742 Fall 03
Step 3: Present the ArchitectureStep 3: Present the Architecture
The lead architect makes the presentation at the The lead architect makes the presentation at the appropriate level of detailappropriate level of detail
Architecture Presentation (~20 slides; 60 minutes)Architecture Presentation (~20 slides; 60 minutes)
Architectural drivers and existing Architectural drivers and existing standrards/models/approaches for meeting (2-3 slides)standrards/models/approaches for meeting (2-3 slides)
Important Architectural Information (4-8 slides)Important Architectural Information (4-8 slides) Context diagram Module or layer view Component and connector view Deployment view
Architectural approaches, patterns, or tactics employedArchitectural approaches, patterns, or tactics employed
– 28 – CSCE 742 Fall 03
Step 3: Present the Architecture (cont)Step 3: Present the Architecture (cont)
Architectural approaches, patterns, or tactics employedArchitectural approaches, patterns, or tactics employed Use of COTS (commercial off the shelf) products Trace of 1-3 most important use cases scenarios Trace of 1-3 most important change scenarios Architectural issues/risks with respect to meeting the driving
architectural requirements Glossary
Should have a high signal to noise ratio, don’t get bogged Should have a high signal to noise ratio, don’t get bogged down too deeply in detailsdown too deeply in details
Should cover technical constraints such as operating Should cover technical constraints such as operating system, hardware and middlewaresystem, hardware and middleware
– 29 – CSCE 742 Fall 03
Step 4: Identify Architectural ApproachesStep 4: Identify Architectural Approaches
The ATAM analyzes an architecture by focusing on its The ATAM analyzes an architecture by focusing on its architectural approachesarchitectural approaches
Captured by not analyzed (here) by the evaluation teamCaptured by not analyzed (here) by the evaluation team
The evaluation team should explicitly asked the The evaluation team should explicitly asked the architect to name thesearchitect to name these
– 30 – CSCE 742 Fall 03
Step 5: Generate Quality Attribute Utility TreeStep 5: Generate Quality Attribute Utility Tree
– 31 – CSCE 742 Fall 03
ScenariosScenarios
Types of Scenarios:Types of Scenarios:
Use case scenariosUse case scenarios The user wants to examine budgetary data
Growth scenariosGrowth scenarios Change the maximum number of tracks from 150 to 200 and
keep the latency of disk to screen at 200ms or less
Exploratory scenariosExploratory scenarios Switch the OS from Unix to Windows Improve availability from 98% to 99.99%
– 32 – CSCE 742 Fall 03
Step 6:Analyze Architectural ApproachesStep 6:Analyze Architectural Approaches
The evaluation team examines the highest priority The evaluation team examines the highest priority scenarios one at a time and the architect is asked scenarios one at a time and the architect is asked how the architecture supports each one.how the architecture supports each one.
– 33 – CSCE 742 Fall 03
– 34 – CSCE 742 Fall 03
Step 7: Brainstorm and Prioritize ScenariosStep 7: Brainstorm and Prioritize Scenarios
The HiatusThe Hiatus Evaluation team distills what has been learned and
informally contacts architecture for more information where needed
AfterAfter
– 35 – CSCE 742 Fall 03
ReferencesReferences
“Evaluating Software Architectures: Methods and Case Evaluating Software Architectures: Methods and Case
StudiesStudies” by Clements, Kazman and Klein, published by Clements, Kazman and Klein, published by Addison-Wesley, 2002. Part of the SEI Series in by Addison-Wesley, 2002. Part of the SEI Series in Software Engineering.Software Engineering.
SEI Software Architecture publicationsSEI Software Architecture publications http://www.sei.cmu.edu/ata/pub_by_topic.html
top related