baxter - usingahp
TRANSCRIPT
-
8/7/2019 Baxter - UsingAHP
1/32
Peter Baxter, DISTRIBUTIVE MANAGEMENT
Hampton Roads INCOSE Decision Analysis Conference - Nov 2009
USING ME A SURE ME NT A ND THE S AATY
ME THOD TO CH O O S E T H E BEST
DECISION ALTERNATIVE
-
8/7/2019 Baxter - UsingAHP
2/32
OBJECTIVES
Learn:
How to construct and weight evaluationcriteria.
How to make pair-wise comparisons ofalternatives.
How to expand the example for morecomplex, nested types of criteria.
How a measurement process can supportthe Saaty method.
D I S T R I B U T I V E
M A N A G E M E N T
2
-
8/7/2019 Baxter - UsingAHP
3/32
WHAT IS THE AHP/SAATYMETHOD?
-
8/7/2019 Baxter - UsingAHP
4/32
SIMPLE DEFINITION
Compute the matrix values for a and voila!
The Analytic Hierarchy Process (AHP)
D I S T R I B U T I V E
M A N A G E M E N T
4
-
8/7/2019 Baxter - UsingAHP
5/32
COMPARE THE SIZES
OF THESEC
IRCLES
A B C D E
A B C D E
A 1 7 9 5 3
B 1/7 1 3 1/5 1/7
C 1/9 1/3 1 1/7 1/9
D 1/5 5 7 1 1/3
E 1/3 7 9 3 1D I S T R I B U T I V E
M A N A G E M E N T
5
-
8/7/2019 Baxter - UsingAHP
6/32
SCALE OF COMPARISON
Dont need an absolute scale
(if you already know the size of all but one)
Relative scale approximates difference
D I S T R I B U T I V E
M A N A G E M E N T
6
-
8/7/2019 Baxter - UsingAHP
7/32
SIMPLE EXPERIMENT
One test is worth 1000 expert opinionsD I S T R I B U T I V E
M A N A G E M E N T
7
-
8/7/2019 Baxter - UsingAHP
8/32
WEBERS EXPERIMENT
In 1846 Weber found, for example, that people while holding in
their hand different weights, could distinguish between a
weight of 20 g and a weight of 21 g, but could not if the
second weight is only 20.5 g.On the other hand, while they
could not distinguish between 40 g and 41 g, they could
between 40 g and 42 g, and so on at higher levels. We need
to increase a stimulus s by a minimum amount s to reach a
pointwhere our senses can first discriminate between s and
s+s. s is called the just noticeable difference (jnd). The
ratio r = s/s does not depend on s.
Webers law states that change in sensation is noticed when the stimulus
is increased by a constant percentage of the stimulus itself. This law
holds in ranges where s is small when compared with s, and hence
in practice it fails to hold when s is either too small or too large.D I S T R I B U T I V E
M A N A G E M E N T
8
-
8/7/2019 Baxter - UsingAHP
9/32
WHY IT WORKS
People are inconsistent at providing an absolutescale to evaluate objects.
How much does Rock A weigh?
How much does Rock B weigh?
There are better at comparing pairs of objects.
Does Rock A weigh more than Rock B?
D I S T R I B U T I V E
M A N A G E M E N T
9
-
8/7/2019 Baxter - UsingAHP
10/32
APPLYING THE TECHNIQUE
1. Select criteria(s) to evaluate
2. Define comparison scale
3. Perform pair-wise comparison
4. Check consistency
5. Calculate values
D I S T R I B U T I V E
M A N A G E M E N T
10
-
8/7/2019 Baxter - UsingAHP
11/32
APPLYING THE METHOD
-
8/7/2019 Baxter - UsingAHP
12/32
EXAMPLE #1
SOFTWARE ESTIMATION
A controlled test where existing software sizes are known.
Ask 30 grad students to estimate the SLOC of common data
structures like stack, queue, list.
Estimate SLOC three ways:
1. Guess a number.
2. Compare to one reference structure using numeric
scale.
3. Compare to one reference structure using relative scale.
Plot results
From Establishing Software Size Using Pair-wise Comparison Method
by Eduardo Miranda
D I S T R I B U T I V E
M A N A G E M E N T
12
-
8/7/2019 Baxter - UsingAHP
13/32
EXAMPLE #1
SOFTWARE ESTIMATION
D I S T R I B U T I V E
M A N A G E M E N T
13
-
8/7/2019 Baxter - UsingAHP
14/32
EXAMPLE #1
OBSERVATIONS
Mirandas Observations
The high variability of the finger in the wind approach,
which is almost two to three times bigger than the
corresponding paired comparisons method.
The high correlation, r = .979, existing between the
relative sizes of modules independent of the estimation
method employed. This seems to corroborate the
premise that the human mind is better at establishingdifferences than at guessing absolute values.
D I S T R I B U T I V E
M A N A G E M E N T
14
-
8/7/2019 Baxter - UsingAHP
15/32
EXAMPLE #2
SYSTEM REQUIREMENTS
From an SEI study Requirements Prioritization Case Study
Using AHP by Nancy Mead.
Given nine security requirements, decide which one(s) have
greatest cost-benefit.
Benefits
By using AHP, the requirements engineer can
also confirm the consistency of the result. AHP can
prevent subjective judgment errors and increase the
likelihood that the results are reliable.
D I S T R I B U T I V E
M A N A G E M E N T
15
-
8/7/2019 Baxter - UsingAHP
16/32
EXAMPLE #2
APPROACH
Approach
1. Review requirements for completeness.
2. Apply pair-wise comparison for value.
3. Apply pair-wise comparison for implementationcost.
4. Calculate AHP matrix and diagram for value & cost.
5. Use resulting diagram for analyzing requirements.
D I S T R I B U T I V E
M A N A G E M E N T
16
-
8/7/2019 Baxter - UsingAHP
17/32
EXAMPLE #2
COMPARISON SCALE
D I S T R I B U T I V E
M A N A G E M E N T
17
-
8/7/2019 Baxter - UsingAHP
18/32
EXAMPLE #2
COMPARISON
Compare one attribute of each requirement to another
requirement using a relative scale.
#1 is same
as #1
#1 is greater
than #2
#2 is much
less than #8
D I S T R I B U T I V E
M A N A G E M E N T
18
-
8/7/2019 Baxter - UsingAHP
19/32
ABOUT CONSISTENCY
If: A > B > C
Then: C > A is wrong
AHP contains a technique to calculate the extent ofpair-wise consistency, which can then be
compared to a consistency tolerance.
AHP can also indicate which pair-wise comparison(like the one above) is inconsistent.
D I S T R I B U T I V E
M A N A G E M E N T
19
-
8/7/2019 Baxter - UsingAHP
20/32
EXAMPLE #2
COMPARISON SUMMARY
D I S T R I B U T I V E
M A N A G E M E N T
20
-
8/7/2019 Baxter - UsingAHP
21/32
EXAMPLE #2
FINDINGS
Client feedback
It may be beneficial to see the consistency matrix.
Understand weight of cost and value.
Difficult to understand the motivation of each
reviewer.
D I S T R I B U T I V E
M A N A G E M E N T
21
-
8/7/2019 Baxter - UsingAHP
22/32
OTHER EXAMPLES
From Saaty:
Evaluate the best city in China for Disney
to build a new theme park.
Determine optimum foreign relationspolicy for dealing with Iran.
Estimate market share of super retail
stores.
Selecting a school.
D I S T R I B U T I V E
M A N A G E M E N T
22
-
8/7/2019 Baxter - UsingAHP
23/32
MEASUREMENT AND AHP
-
8/7/2019 Baxter - UsingAHP
24/32
MEASUREMENT AND AHP
Goal is for Measurement to support AHP estimation
Estimate
Plan
Monitor& Control
Capitalize
Measurement Process
Use AHP
D I S T R I B U T I V E
M A N A G E M E N T
24
-
8/7/2019 Baxter - UsingAHP
25/32
TYPICAL MEASUREMENT
PROCESS
Evaluate
Establish
Capability
Technical
and Management
ProcessesINFORMATION
NEEDS
ANALYSIS RESULTS
ANALYSIS
RESULTSAND
PERFORMANCE
MEASURESIMPROVEMENT
ACTIONS
Measurement Process
USER FEEDBACK
Plan Perform
Core Measurement Process
Experience
Base
MEAS-
UREMENT
PLAN
25
-
8/7/2019 Baxter - UsingAHP
26/32
WHAT TO MEASURE?
Measurement Requirements = information needs
Based on your business needs.(not a pre-defined list)
Information needs are:
Input to the measurement process.
Provided by the management and technical process thatneed information to perform their jobs.
Become the requirements for measurement process.
Refined into measures and then resulting informationproducts are provided to the users.
D I S T R I B U T I V E
M A N A G E M E N T
26
-
8/7/2019 Baxter - UsingAHP
27/32
COMMON PROJECT
MEASURES
Cost
Staffing , Staff Hours
Functional / Requirements Size
SLOC
Defects
Estimation Project/Program Store
Factors estimate
using HP
D I S T R I B U T I V E
M A N A G E M E N T
27
Usually there is opportunity for measurement to support AHP since
there is overlap between what is estimated and what is measured
-
8/7/2019 Baxter - UsingAHP
28/32
MEASUREMENT GUIDE AHP
Measurement contains project attributes:
Lifecycle model
Principle architecture
Application domain
AHP estimation of projects with similar attributes.
Reference sizes should not differ by more than an
order of magnitude.
D I S T R I B U T I V E
M A N A G E M E N T
28
-
8/7/2019 Baxter - UsingAHP
29/32
AHPIN USE
Estimate total staff hours in person years using data from
five completed projects sharing similar attributes.
D I S T R I B U T I V E
M A N A G E M E N T
29
-
8/7/2019 Baxter - UsingAHP
30/32
RESOURCES
Requirements Prioritization Case Study Using AHP
by Nancy Mead Software Engineering Institute
Establishing Software Size Using the Paired
Comparisons Method by Eduardo Miranda
Relative Measurement and Its Generalization in
Decision Making Why Pairwise Comparisons are
Central in Mathematics for the Measurement of
Intangible Factors The Analytic
Hierarchy/Network Process by Thomas Saaty
D I S T R I B U T I V E
M A N A G E M E N T
30
-
8/7/2019 Baxter - UsingAHP
31/32
QUESTIONS
D I S T R I B U T I V E
M A N A G E M E N T
31
-
8/7/2019 Baxter - UsingAHP
32/32
Peter Baxter
Distributive Management
www.distributive.com
D I S T R I B U T I V E
M A N A G E M E N T
32