ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
How ESSA has successfully used Decision Analysis to
overcome challenges in multi-objective resource management
problems
General overview
January 10 2002
Developed byESSA Technologies Ltd.
David Marmorek, Calvin Peters, Ian Parnell, Clint Alexander
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Common challenges in resource management
• Getting stakeholder groups to agree on a course of action, given multiple values and objectives
• Getting scientists to agree on which uncertainties most critically affect management decisions, and what decisions are most robust to these uncertainties
• Evaluating the costs and benefits of adaptive management - is it worth it?
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
How decision analysis can help with these challenges
• It provides a toolbox for handling multiple objectives / values, and analyzing tradeoffs among these objectives
• It systematically analyzes the impacts of uncertainties on decisions
• It can be used to evaluate the ability of Adaptive Management experiments to improve decisions
• It provides a helpful way to integrate many techniques employed by managers and scientists (i.e. models, interactive workshops, sensitivity analysis) into products that better clarify management decisions
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Three examples
• Getting scientists to agree: PATH
• Getting stakeholders to agree: Cheakamus
• Evaluating adaptive management: Keenleyside
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
PATH: Decision Context
• Multiple historical changes in Columbia and Snake River ecosystems and fisheries management practices
• Endangered species listings for Snake River salmon populations
• Multiple hypotheses and uncertainties held by different groups of scientists
• Duelling models representing these hypotheses and uncertainties
• Best management policies for species recovery?
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
PATH: Washington State, US
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Decision Analysis: 8 elements
1. List of alternative management actions
2. Management objectives composed of performance measures (to rank management actions)
3. Uncertain states of nature (different hypotheses)
4. Probabilities of those states (to account for uncertainty);
5. Model to calculate outcomes of each combination of management action and hypothesised state of nature;
6. Decision tree;
7. Rank actions based on expected value of the performance measures; and,
8. Sensitivity analyses.
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Decision Analysis: Basic Elements
Module 3 -36MoF Adaptive Management Training Course
Action 1
Managementactions
Probabilities ofstates of nature
States of natureor hypotheses
Outcomes orconsequences
Action 2
P1
P2
P1
P2
Hypothesis 1
Hypothesis 2
Hypothesis 1
Hypothesis 2
C11
C12
C21
C22
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
PATH Decision Tree
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Benefits of decision analysis in PATH
• Allowed evaluation of multiple hypotheses for 14 uncertainties - scientists did not have to agree!
• Only 3 of these turned out to make a difference to the decision - created a common focus for AM, research
• Preferred actions were those which were most robust to the critical uncertainties (drawdown A3)
• Sensitivity analyses defined how much belief you would have to have in a given hypothesis to change decision
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Recent Publications on PATH
• Marmorek, David R. and Calvin Peters. 2001. Finding a PATH towards scientific collaboration: insights from the Columbia River Basin. Conservation Ecology 5(2): 8. [online] URL: <http://www.consecol.org/vol5/iss2/art8>
• Deriso, R.B., Marmorek, D.R., and Parnell, I.J. 2001. Retrospective Patterns of Differential Mortality and Common Year Effects Experienced by Spring Chinook of the Columbia River. Can. J. Fish. Aquat. Sci. 58(12) 2419-2430 http://www.nrc.ca/cgi-bin/cisti/journals/rp/rp2_tocs_e?cjfas_cjfas12-01_58
• Peters, C.N. and Marmorek, D.R. 2001. Application of decision analysis to evaluate recovery actions for threatened Snake River spring and summer chinook salmon (Oncorhynchus tshawytscha). Can. J. Fish. Aquat. Sci. 58(12):2431-2446. <same web site as above>
• Peters, C.N., Marmorek, D.R., and Deriso, R.B. 2001. Application of decision analysis to evaluate recovery actions for threatened Snake River fall chinook salmon (Oncorhynchus tshawytscha). Can. J. Fish. Aquat. Sci. 58(12):2447-2458. <same web site as above>
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Cheakamus WUP: Decision Context
• British Columbia Hydro, Water Use Planning: Stakeholder driven multi-objective consultation / decision process.
• No formal incorporation of uncertainty as for PATH
• Emphasis: values, objectives, performance measures, trade off analysis (DA steps 1, 2, 5 and 7).
• Used PrOACT approach (Smart Choices, Hammond et al 1999)
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Cheakamus WUP: ProcessPrOACT Approach
Problem
Objectives
Alternatives
Consequences
Tradeoffs
Clear choice
Many choices
WUP Steps
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Cheakamus WUP:Decision ProblemSelect operating alternatives for Daisy Lake Dam that:
1) recognize multiple water uses in the Cheakamus and Squamish Rivers, and
2) achieve a balance between competing interests and needs.
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Cheakamus WUP:Objectives and PMsFundamental
ObjectivesPerformance Measures
Average power revenue ($M/yr)
Power production (GWh)1. Maximize economicreturns from powergeneration. Greenhouse Gas emission reductions (Ktonnes/yr)
2. Protect integrity ofSFN heritage sites andcultural values.
Flood and erosion risk to ancestral burial groundsand culturally important locations
Rafting (Avg. #days/yr)
Kayaking (Avg. #days/yr)3. Maximize physicalconditions / access forrecreation (kayaking,rafting, sportfishing). Sportfishing (Avg. #days/yr)
4. Minimize adverseeffects of flood events.
Flooding (# floods >450cms at Brackendale)
Anadromous rearing Habitat Availability (m2),
Resident rearing Habitat Availability (m2)
Anadromous Effective Spawning Area (m2),
5. Maximize wild fishpopulations
Adult Migration flows (Avg. #days <10CMS)
Anadromous Riffle Benthic Biomass (kg benthos),6. Maximize area andintegrity of aquaticecosystem Resident Riffle Benthic Biomass (kg benthos)
Power
First Nations
Recreation
Flooding
Fish
Aquatic Ecosystem
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Cheakamus: WUP Alternatives
• Consultative Committee specifies operating alternatives for Hydro operations model (AMPL).
• Basic constraints: minimum flow at Brackendale gauge, minimum dam release.
• AMPL model produces 32 water years of flow data for these control points
• Flow data and other models used to calculate performance measures.
• Performance measures summarize consequences of alternatives for objectives.
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Cheakamus WUP: ConsequencesF u n d a m e n t a l
O b j e c t i v e sP e r f o r m a n c e
M e a s u r e s
1 5 M i n 3 D a m 1 5 M i n 5 D a m 1 5 - 2 0 M i n 3 -7 D a m " H y b r i d "
2 0 M i n 7 D a m 1 0 D a m
1 . M a x i m i z e e c o n o m i c r e t u r n s f r o m p o w e r
g e n e r a t i o n . A v e r a g e p o w e r
r e v e n u e ( $ M / y r )3 5 . 6 3 4 . 8 3 4 . 3 3 2 . 3 3 1 . 8
2 . P r o t e c t i n t e g r i t y o f S F N h e r i t a g e s i t e s a n d c u l t u r a l
v a l u e s .
K a y a k i n g ( A v g . # d a y s / y r )
1 2 3 . 9 1 3 7 . 7 1 9 9 . 8 2 4 2 . 0 2 0 4 . 1
S p o r t f i s h i n g ( A v g . # d a y s / y r )
5 7 . 6 7 2 . 0 8 2 . 7 1 9 2 . 8 1 2 2 . 0
5 . M a x i m i z e w i l d f i s h p o p u l a t i o n s ( x 1 0 3 m 2 )
R U A R e s i d e n t H a b i t a t R a i n b o w P a r r 3 5 . 8 3 7 . 7 4 2 . 5 4 2 . 5 4 5 . 2
E f f e c t i v e S p a w n i n g A r e a C h u m 9 . 8 9 . 2 9 . 7 7 . 3 6 . 5
6 a . M a x i m i z e a r e a a n d i n t e g r i t y o f a q u a t i c
e c o s y s t e m
R e s i d e n t R i f f l e B e n t h i c B i o m a s s ( g
x 1 0 6 )3 . 4 3 . 5 2 . 9 2 . 9 3 . 0
P a r t l y c o n s i d e r e d b y F l o o d P M s , w i l l b e a d d r e s s e d i n f u t u r e i f n e c e s s a r y .
A l t e r n a t i v e s
3 . M a x i m i z e p h y s i c a l c o n d i t i o n s / a c c e s s f o r
r e c r e a t i o n ( k a y a k i n g , r a f t i n g , s p o r t f i s h i n g ) .
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Tradeoffs (or not)Tradeoff: VOE vs. RB Parr
10Dam20Min7Dam
20Min3Dam 20Min
7Dam 15Min5Dam
15Min3Dam
5Dam
0
10000
20000
30000
40000
50000
31.00 32.00 33.00 34.00 35.00 36.00
VOE ($M/yr)
RB
Par
r H
abit
at A
vail
abil
ity
(m2)
Tradeoff: VOE vs. Chum Effective Spawning Area
15Min3Dam
5Dam
15Min5Dam
7Dam
20Min
20Min3Dam
20Min7Dam
10Dam
0
2000
4000
6000
8000
10000
12000
31.00 32.00 33.00 34.00 35.00 36.00
VOE ($M/yr)
Ch
um
Eff
. S
pw
n. A
rea
Win-Win
Win-Lose
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Cheakamus WUP: Filtering
• Use PMs to Eliminate clearly inferior alternatives.
• Drop insensitive PMs (e.g., rafting).
• Drop Objectives that don’t help the decision (e.g., flooding).
• Tradeoff analysis: Even Swaps
• Elicit values behind decisions (e.g., rating exercises)
• Develop new alternatives to address concerns (e.g., chum spawning vs. rainbow trout rearing).
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Keenleyside Problem Keenleyside Problem : Increased egg mortality from dam operation
Flow during spawningFlow during spawning
Flow during Flow during incubationincubation
stage
Proportion eggs in de-watered areaRiskRisk
Biological Biological flows too high reduce productive capacity, may drive population towards extinction
Economic Economic smaller flows may reduce de-watering mortality but reduce potential $ and operational flexibility
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Problem IIProblem II: Uncertainty True whitefish recruitment dynamics?
No reliable baseline information
Alternative Hypotheses
-
5,000
10,000
15,000
20,000
25,000
0 5 10 15 20 25
Eggs Just Prior to Hatching (millions)
Age
4 W
hite
fish
Very Sensitive
Sensitive
Neutral
Insensitive
Very Insensitive
Given typicalegg mortality,
LARGE differences in abundance
associated with these curves
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Stage 1 - Decision Analysis w current uncertainty
Columbia RiverFlows During
WhitefishSPAWNING
Kootenay RiverFlows During
WhitefishSPAWNING
Min. Columbia RiverFlows Prior to
WhitefishHATCHING
Egg-Age4RecruitmentRelationship
EggAbundance
Abundance4+ Recruits
50 kcfs Model
30 kcfs
20 kcfs
85 kcfs
80 kcfs
20 kcfs
15 kcfs
10 kcfs
55 kcfs
25 kcfs
20 kcfs
15 kcfs
85 kcfs
a3, b 3
a2, b 2
a1, b 1
a5, b 5
a4, b 4
ForegonePower
Revenues
......
ManagementActions States of Nature and their Probabilities Outcomes
Min. KootenayRiver Flows Prior
to WhitefishHATCHING
20 kcfs
15 kcfs
10 kcfs
55 kcfs
...
40 kcfs
45 kcfs
55 kcfs
60 kcfs
65 kcfs
70 kcfs
Natural variability in flow Uncertainty due to lackof understanding / data
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Stage 1 Results: Current UncertaintyExpected adult N, year 50
30,000
35,000
40,000
45,000
50,000
20 30 40 45 50 55 60 65 70 80 85
HKD Spawning Q (kcfs)
N
Minimum desired
Base Case: Current Uncertainty
Whitefish recruitment dynamics: Current state of knowledge
0
0.2
0.4
H1(sensitive)
H2 H3 H4 H5(insensitive)
P
Objective:Maintain “least cost” whitefish population nearest to or greater than 45,000 adults
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Stage 2 - Simulated learning from flow experiments and monitoring
Uses same model and uncertain components but...
Actions are now alternative experimental Actions are now alternative experimental flow regimes + monitoring programsflow regimes + monitoring programs
Assume a true relationship for population Assume a true relationship for population dynamics with process errordynamics with process error
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
What would you change if you knew the “truth”?If population insensitive, then maximize power revenues (85 kcfs)
If population sensitive, then minimize biological risk (~60 kcfs)Expected adult N, year 50
25,000
30,000
35,000
40,000
45,000
50,000
20 30 40 45 50 55 60 65 70 80 85
HKD Spawning Q (kcfs)
N
Minimum desired
Current Uncertainty
Sensitive
Insensitive
10
5
2.5
7.5$Cnd mil
Max. potential power revenues (per yr)
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Example Stage 2 Results: Good monitoring is critical for differentiating hypotheses; flow
manipulation had less effect than expected.
Flow manipulation
High Meas. Error
Low Meas. Error
High Meas. Error
Low Meas. Error
Constant 0.55 ($0.48) 0.88 ($1.55) 0.51 ($0.48) 0.74 ($1.55)Passive 0.60 ($1.23) 0.92 ($2.3) 0.57 ($1.23) 0.85 ($2.3)
Active 0.63 ($3.48) 0.92 ($4.55) 0.54 ($3.48) 0.85 ($4.55)
$ CDN MillionsBlue = things under AM practitioners controlRed = beyond AM practitioners control
Probability identify insensitive population (10-year experiments)
Low Nat Variability High Natural Variability
Natural Variability and Measurement Error
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
AM can “pay for itself”
Flow manipulation
High Meas. Error
Low Meas. Error
High Meas. Error
Low Meas. Error
Constant $0.2 (2.4) $0.6 (2.6) $0.2 (2.4) $0.2 (7.7)
Passive $0.2 (6) $0.6 (3.8) $0.2 (6.15) $0.6 (3.8)
Active $0.6 (5.8) $0.6 (7.6) $0.2 (17.4) $0.6 (7.6)
$Cnd millionsNumbers in brackets = experimental pay-back interval in yearsBlue = things under AM practitioners controlRed = beyond AM practitioners control
I ncrease in annual power revenues from operating with experimental information (insensitive population only, 10-year experiments)
Low Nat Variability High Natural Variability
Natural Variability and Measurement Error
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Is AM and monitoring worth it?Is AM and monitoring worth it?
“Yes” IfNew information leads to choice of a
different management action that better satisfies a particular objective,
or rigorously confirms that current
management action is appropriate.
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
No definitive “yes/no”No definitive “yes/no”
Management objective(fish vs. power $)
Ability to do well designed experiments
Initial level of uncertainty in alternative hypotheses
Magnitude of natural variability in the system
What “truth” really is
Inherent sensitivity of best action to uncertainty
FactorUnder AM
practitioners controlCan evaluate implications using decision analysis?
Yes
Yes
Maybe
No
No (can’t know without doing the experiment)
No
Yes
Yes
Yes
Yes
Yes
Yes
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
General ConclusionsGeneral Conclusions• Value of AM potentially large
• Whether to proceed depends on “the kind” of system you are in (i.e. previous factors)
• Decision Analysis is very helpful for evaluating these benefits
– Determine which uncertainties have strongest effect on choice of “best” management decision
– Decisions more robust to uncertainties (reduces risk - integrates broader range of possible outcomes included)
– Include new information as revised probabilities on hypotheses
ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002
Decision Analysis - SummaryDecision Analysis - SummaryElement of DecisionAnalysis
PATH –scientist
consensus
Cheakamus –stakeholderconsensus
Keenleyside –AM evaluation
Actions
Objectives
Uncertainties
Probabilities
Model
Decision Tree
Rank Actions
Sensitivity Analyses