Download - Decision Making and Pairwise Comparisons
Proprietary & confidential. © Decision Lens 2010
Decision Making, Hierarchical Structures and Pairwise
ComparisonsFebruary 2010
Concepts
Decision Making and Hierarchical Thinking
Criteria
Pairwise Comparisons
Priorities
2
Analytical Hierarchy Process (AHP)
The Analytic Hierarchy Process enables decision makers to structure decisions hierarchically: the goal of the decision at the top, strategic objectives in the higher levels, evaluation criteria in the middle, and alternative choices at the bottom.
3
Criteria Definition
At the most basic, a criterion is a driving factor that is used in making a decision. In Decision Lens, we use a criteria hierarchy
4
At the highest level, objectives are stated as actions or goals to be accomplished− i.e., Increase Impact to Mission
At the lowest level, drivers or indicators used to evaluate alternatives at a detailed level− i.e., Mission Area 1
• Don’t confuse criteria with strategies, actions, or methods of execution
• The criteria should be the end state, not the path to get there.
– i.e. “Increase Growth” is a good criterion. “International Expansion” is not a criterion, it is a strategy to achieve the objectives.
– “Expand Brand” is a good criterion. “Acquire New Products” is not a criterion, it is a strategy to achieve the objectives.
Module 1: Building the Model
Criteria Modeling Example – Selecting Which New Products to Bring to Market
5Module 1: Building the Model
Best Practice Criteria Concepts
While dependencies may exist, criteria should be as mutually exclusive, collectively exhaustive (MECE) as possible
We normally see at least 2 levels of criteria where a criterion is too broad of a “bucket” but not more than 3 levels is recommended
The ideal number of criteria at the highest level is 7
6
Magic numberof
seven plus or minus two
Human Cognitive Capacity
Purpose of Comparing Criteria
Perform Pairwise Comparisons to develop relative weights on criteria hierarchy
Facilitating discussion during voting to reveal stakeholder viewpoints and positions
Defining consensus and how to use it in support of prioritization
Consistency and Alignment among decision-makers is measured as a result of their comparisons
7Module 2: Compare Criteria
Do Numbers Have an Objective Meaning?
Butter: 1, 2,…, 10 lbs.; 1,2,…, 100 tons
Sheep: 2 sheep (1 big, 1 little)
Temperature: 30 degrees Fahrenheit to New Yorker, Kenyan, Eskimo
Since we deal with Non-Unique Scales such as [lbs., kgs], [yds, meters], [Fahr., Celsius] and such scales cannot be combined, we need the idea of PRIORITY.
PRIORITY becomes an abstract unit valid across all scales.
A priority scale based on preference is the AHP way to standardize non-unique scales in order to combine multiple criteria.
8Module 2: Compare Criteria
This is Today’s Reality: Liter and KilogramThe Basis of Any Scale is Arbitrary
In 1889, the first Conference Generale des Poids et Mesures (CGPM) sanctioned the international prototype of the kilogram, made of platinum-iridium, and declared: This prototype shall henceforth be considered to be the unit of mass. The third CGPM (1901), in a declaration intended to end the ambiguity in popular usage concerning the word weight, confirmed that:
The kilogram is the unit of mass; it is equal to the mass of the international prototype of the kilogram. The International Prototype Kilogram (IPK) kept in Sèvres, near Paris, is the standard of mass. National prototype kilograms are maintained in several countries and periodically checked against the International Prototype.
9Module 2: Compare Criteria
Pairwise Comparison
Participants perform multiple sets of comparisons for each level of hierarchy
For each judgment, participants determine which criterion is more important AND by how much
Judgments are used to form ratios in a matrix; The matrix is used to calculate priorities for the judgment set (eigenvector)
10Module 2: Compare Criteria
Scale for Pairwise Comparison
1 Equal importance
3 Moderate importance of one over another
5 Strong or essential importance
7 Very strong or demonstrated importance
9 Extreme importance
2,4,6,8 Intermediate values
Use Reciprocals for Inverse Comparisons
11Module 2: Compare Criteria
Pairwise Comparison Steps
12
4
Click the Compare Criteria tab
Select the Pairwise Comparison step
Each person makes judgmentsusing keypads. First, evaluate which criteria is more important. Second, evaluate by how much using the Equal to Extreme scale
3
5
The facilitator will click Next Vote to move to the next comparison until all criteria have been compared to each other1
The facilitator will ask the group, “With respect to the goal of buying a car, what is more important, safety or performance?”
2
Use the tree view in the left navigation as an alternative way to select which criteria are being selected
Module 2: Compare Criteria
Best Practice Pairwise Criteria Concepts
Keep in mind:
The facilitator should promote group discussion to inform and provide rationale for voting discrepancies
To make a comparison one holds the smaller one in the pair in mind as the unit and estimates how many times larger the other is
13Module 2: Compare Criteria
Viewing Priority Graph
14
A) Clicking the second sub step Priority GraphB) Clicking Next Vote at the last comparison there are no more votes so it will take you to the next step)C) Or by clicking Next (see previous slides)
A CB
Once all comparisons are made you can view results in 3 ways:
Module 2: Compare Criteria
Priority Graph Options
15
Click Sort Descending to get a highest to lowest graph of the weighted priorities. Note that you will see the weights for what is selected on the criteria tree view on the left navigation panel.
1
Click the Current Properties drop down to pick which data set is shown. You can choose to see results of a specific individual’s judgments or the group.
2
You can select either the Graph or the Tree view. Graph view provides a bar chart like what you see here. Tree view is a blown up version of the tree view that you see on the left navigation pane.
The Alignment is an index shown in percentage to indicate how much consensus the re were around judgments.
4
Click either Global or Local radio button to change how the weights are shown in the tree view. Global shows overall weights of sub factors. Local shows how sub factors are distributed within that node only
5
See next slide 3
Module 2: Compare Criteria
Inconsistency and Alignment
16
Feature Definition Use Helpful Hint
Inconsistency
The consistency of the judgments is tracked using the rigorous math analytics behind the AHP to validate the decision process.
If the group’s (not an individual’s) inconsistency is more than .10 or 10%, certain pairwise comparisions should be revisited to ensure that logical integrity is kept
Individual’s inconsistency will always be higher than the groups. Groups are known to be more accurate and consistent.
Having good consistency does not mean you are closer to the “right answer” in the decision
Alignment
The final priorities in the Priority Graph, tells you the mix of values for the group, and the Alignment Index score then tells you how close that Priority Graph was for all of the individuals as compared to the group average.
The largest score in the Alignment Index is 100, telling you that the final weighted priorities for each individual came out exactly the same as the final weighted priorities for the group. There is no action regardless of the Alignment number.
The smaller the Alignment Index score drops from 100, the farther away are the individuals’ priorities from those of the group, and the less alignment there is.
Module 2: Compare Criteria
Inconsistency Diagram
A is bigger than BB is bigger than CC is bigger than A!?
This is not logical – the inconsistency tracker tracks individual and group logic when judgments are made in pairwise comparisons
Group consistency is best to be within 10%
17
A B C
Module 2: Compare Criteria
Evaluating Inconsistency
18
Click the Compare Criteria tab and the Inconsistency Analysis sub step.
Alternatively, if you are using the Next button at the right of the screen to navigate through tab 2, you will see Inconsistency at the end of each comparison set following the Priority Graph for that comparison set.
1
2
Ensure the goal level of the tree is selected
In this case, Inconsistency is fairly close to zero therefore, no action is necessary. You can click the Go To Comparison button next to those comparisons that are high. You want to decrease overall consistency to 10%. This may require changing one or more judgments.
3
Module 2: Compare Criteria
Critical Facilitation Technique – 3 Part Test
Using facilitation techniques, ensure that participants pass…..
The 3 Part Test:
1.Can I live with the decision made here today?
2.Can I refrain from behaving or speaking negatively to others about the decision?
3.Can I readily support the decision and devote the resources necessary to make the decision successful?
19Module 2: Compare Criteria
Summary
20
Prioritize decision-making criteria
Inputs Any previously developed decision criteria; Decision makers are identified; Any assumptions are understood.
Actions Complete head-to-head (pairwise) comparisons of criteria. Ensure that the group’s logic is not too inconsistent.
Helpful Hints Encourage participants to discuss the rationale behind their voting as a way to get information and insights out in the open. Ensure consistency is less than 10%.
Things to Avoid Do not let any one individual dominate the discussion. Make sure your criteria are defined very specifically and that all participants understand and agree with their meaning. When making judgments, participants are encouraged to explain their position but should not be forced to agree with everyone else.
Outputs Weighted decision criteria model.
Module 2: Compare Criteria