decision making in an uncertain world · fischbach, j., managing new orleans flood risk in an...

Decision Makingin an Uncertain World

Robert LempertDirector

RAND Pardee Center on Longer-Range Global Policyand the Future Human Condition

Keeping Our Heads Above Water:Adapting to Climate Change in Southern California

May 2, 2011

2

Adaptation to Climate Change Poses BothAnalytic and Organizational Challenges

• Solution requires rethinking how we use uncertainclimate information in our planning

• Recent reports suggests:– There are limits to the applicability and usefulness of

classic decision analysis to climate-related problems

– Seeking robust strategies may prove a preferableapproach

– Any analytic approach should be embedded inappropriate process of stakeholder engagement

• Planning with statistics of future climate based on projections,rather than just replicating recent history, requires

• Usefully summarizing incomplete information from new, fast-moving, and potentially irreducibly uncertain science

• Justifying analytic choices to diverse constituencies, many ofwhom may object to implications of some particular choices

3

Believing Forecasts of the UnpredictableCan Contribute to Bad Decisions

• In the early 1970sforecasters madeprojections of U.Senergy use based ona century of data

Gross national product (trillions of 1958 dollars)2.22.01.81.61.41.21.0.8.6.4.20

180

Energy use (1015 Btu per year)0

Historicaltrend

continued1970

1920 19291940

19501960

1910

1973

19731900

1890

20 40 60 80 100 120 140 160

1975 Scenarios

4

Believing Forecasts of the UnpredictableCan Contribute to Bad Decisions

Gross national product (trillions of 1958 dollars)2.22.01.81.61.41.21.0.8.6.4.20

180

Energy use (1015 Btu per year)0

Historicaltrend

continued1970

1920 19291940

19501960

1910

1973

19731900

1890

20 40 60 80 100 120 140 160

2000 Actual

1990

19801977

1975 Scenarios2000 Actual

1990

19801977

• In the early 1970sforecasters madeprojections of U.Senergy use based ona century of data

… they all were wrong

5

Traditional Planning Methods Can ComplicateDecisions Under Deep Uncertainty

Traditional analytic methods characterize uncertainties as aprelude to assessing alternative decisions

Climate change (and many other challenges) confrontdecisionmakers with deep uncertainty, where

– They do not know, and/or key parties to the decision do not agree on, thesystem model, prior probabilities, and/or “cost” function

Decisions can go awry if decisionmakers assume risks are well-characterized when they are not

– Uncertainties are underestimated– Competing analyses can contribute to gridlock– Misplaced concreteness can blind decision-makers to surprise

Predict Act

6

Robust Decision Making (RDM) Helps Inform GoodDecisions Without Reliable Predictions

Candidateplans

Identifyvulnerabilities

Assess alternativesfor amelioratingvulnerabilities

Key idea – conduct the analysis in reverse order from predict thenact:

1. Start with a proposed plan for a specific decision maker

2. Use analytics to identify (multi-attribute) scenarios that describethe future conditions where proposed plan fails to meet its goals

3. Use these scenarios to identify potential actions to addressvulnerabilities and evaluate tradeoffs among them

4. Repeat until stakeholders agree the resulting plan is robust acrossmultiple views and unknowns regarding the future

7

Currently Applying This RDM Approach WithMany Resource Management Agencies

2004

2005

2006

2007

2008

2009

2010

2011

Long-term WaterResources Planning

Coastal Protection and Restoration

IEUA ClimateAdaptationStudies (NSF)

2005 CaliforniaWater Plan (NSF) US ACE

Risk InformedDecision Framework

Gulf CoastFisheriesStudy

CORiverStudy

SierraNevadaClimateAdaptationStudy(PIER)

2009CaliforniaWater Plan

MWD 2009IntegratedResourcePlan

DenverWaterPilotProject Louisiana OCPR

Annual Plans & 2012Master Plan Update

New OrleansRiskMitigationStudy(NOAA)

Port of L.A.& sea levelrise (NSF)

World BankCase Studies:Mexico City,Vietnam,Kosovo

2013California

Water Plan

Water Resources FoundationCO Springs Utilities & NYC

8

RDM Consists of Analytic Methods Embedded in aProcess of Stakeholder Engagement

Recent U.S. National Academy of Science report recommends forclimate-related decision support a process of deliberation withanalysis, which is an iterative process that:

• Begins with the many participants to a decision working togetherto define its objectives and other parameters,

• Working with experts to generate and interpret decision-relevantinformation, and

• Then revisiting the objectives and choices based on thatinformation.

9

Importantly, RDM Consists of Analytic MethodsEmbedded in a Process of Stakeholder Engagement

Recent U.S. National Academy of Science report recommends forclimate-related decision support a process of deliberation withanalysis, which is an iterative process that:

• Begins with the many participants to a decision working togetherto define its objectives and other parameters,

• Working with experts to generate and interpret decision-relevantinformation, and

• Then revisiting the objectives and choices based on thatinformation.

To address specific cognitive and organizational barriers togood decisions, RDM implements such a process by:

• Beginning with one or more policies under consideration, and anunderstanding of what would constitute success

• Identifying vulnerabilities of those policies

• Identifying and evaluating potentially robust responses

10

– Current water sources include:

• Groundwater 56%

• Imports 32%

• Recycled 1%

• Surface 8%

• Desalter 2%

Helped Inland Empire Utilities Agency (IEUA) IncludeClimate Change in Their Long-Range Plans

– IEUA currently serves 800,000people

• May add 300,000 by 2025

– Water presents a significantchallenge

RAND TR-505-NSF

11

– Current water sources include:

• Groundwater 56%

• Imports 32%

• Recycled 1%

• Surface 8%

• Desalter 2%Focus of IEUA’s 25 year plan

Helped Inland Empire Utilities Agency (IEUA) IncludeClimate Change in Their Long-Range Plans

– IEUA currently serves 800,000people

• May add 300,000 by 2025

– Water presents a significantchallenge

RAND TR-505-NSF

12

Model Performance of plans

IEUA Plans

System data & climate forecasts

Simulation Model Assesses Performance ofIEUA Plans in Many Different Scenarios

Temp: +1.6Temp: +1.6ooC Precip: -10%C Precip: -10%

Scenario BPlan suffers shortages inadverse future climate

005050

100100150150200200250250300300350350400400

20052005 20102010 20152015 20202020 20252025 20302030YearYear

Ann

ual s

uppl

y (ta

f)A

nnua

l sup

ply

(taf)

Recycled

GroundwaterGroundwater

Local SuppliesLocal Supplies

ImportsImports

Dry-year yieldDry-year yieldSurplusSurplus

ShortageShortage

Temp: +0.7Temp: +0.7ooC Precip: +3%C Precip: +3%

Scenario APlan generates surpluses inbenign future climate

005050

100100150150200200250250300300350350400400

20052005 20102010 20152015 20202020 20252025 20302030YearYear

Ann

ual s

uppl

y (ta

f)A

nnua

l sup

ply

(taf)

RecycledRecycled

GroundwaterGroundwater

ImportsImportsSurplusSurplus

Local SuppliesLocal Supplies

Groves et. al. (2007)

13

Many Uncertain Factors Could Impactthe Performance of Current IEUA Plan

NaturalProcesses

• Future temperatures

• Future precipitation

• Changes in groundwater processes

Performance ofManagementStrategies

• Development of aggressive waste-waterrecycling program

• Implementation of groundwaterreplenishment

Costs of FutureSupplies andManagementActivities

• Imported supplies

• Water use efficiency

14

0 1.0 2.0 3.0 4.0PV shortage cost ($ billions)

2.5

3.0

3.5

4.0

PV s

uppl

y co

st ($

bill

ions

)

Scenario A

Scenario B

• Adverse climate• $3.4 billion in supply

cost• $1.9 billion in

shortage cost

Current IEUA plan forever

• Benign climate• $3.3 billion in supply

cost• $0 in shortage cost

“Scenario Maps” Help Decision MakersVisualize a Plans’ Vulnerabilities

Source: David G. Groves, Robert J. Lempert, Debra Knopman, Sandra H. Berry, 2008: Preparing for an UncertainFuture Climate in the Inland Empire: Identifying Robust Water-Management Strategies, DB-550-NSF, RAND

Choosecandidatestrategy


Identify &assessoptions

15


2.5

3.0

3.5

4.0

PV s

uppl

y co

st ($

bill

ions

)


(200 Scenarios)


RAND DB-550-NSF

Consideroptions


Assessalternativeresponses

16


2.5

3.0

3.5

4.0

PV s

uppl

y co

st ($

bill

ions

)

Current IEUA Plan Forever

$3.75 billioncost threshold

Current plangenerates high costsin 120 of 200Scenarios


Consideroptions


Assessalternativeresponses

17


2.5

3.0

3.5

4.0

PV s

uppl

y co

st ($

bill

ions

)


Statistical “Scenario Discovery” AnalysisIdentifies Scenario Where Existing Plan Fails

These three factorsexplain 70% ofvulnerabilities ofIEUA’s current plans

RAND DB-550-NSF

Consideroptions


Assess alternativeresponses

18

IEUA Considered Response Options ThatEvolve Over Time with New Information

Act now toAct now toaugmentaugment

2005 Plan?2005 Plan?

NO

Monitor, andMonitor, andtake additionaltake additional

action ifaction ifsupplies dropsupplies drop

too lowtoo low

In 2015, 2020, 2025, ….

YES

ImplementImplementadditionaladditionalefficiency,efficiency,

recycling, andrecycling, andreplenishmentreplenishment

In 2015, 2020, 2025, ….

Monitor, andMonitor, andtake additionaltake additional

action ifaction ifsupplies dropsupplies drop

too lowtoo low

Consideroptions


Assessalternatives

19

Compare Alternative Plans With DifferentMixes of “Act Now” vs. “Act Later”

0 20 403010

Static options

Update options

Number of Scenarios(PV Costs > $3.75 billion)

UWMP with updates

UWMP + replenishment with updates

UWMP + efficiency

UWMP + efficiency with updates

UWMP + DYY and recycling with updates

UWMP + all enhancements

Economic CostsDecrease, ButUnquantifiedOpportunity

Costs Increase

RAND DB-550-NSF

Consider options

Identify vulnerabilities

Assess alternatives

Current static UWMPvulnerable in 120 scenarios

20

Compare Alternative Plans With DifferentMixes of “Act Now” vs. “Act Later”

0 20 403010

Static options

Update options

Number of Scenarios(PV Costs > $3.75 billion)

UWMP with updates

UWMP + replenishment with updates

UWMP + efficiency

UWMP + efficiency with updates

UWMP + DYY and recycling with updates

UWMP + all enhancements

IEUA chose to accelerate their dry-year yield and recyclingprograms, and adapt as needed down the road

RAND DB-550-NSF

Economic CostsDecrease, ButUnquantifiedOpportunity

Costs Increase

21

Such Robust Adaptive Plans AlsoProving Useful at the Regional Level

– Blue Ribbon Committee tolook out 50 years and“identify and recommendnew business models andstrategies that will help theregion meet its long-rangewater needs.”

– Planning staff to identifyearly warning indicatorsneeded to implement theiradaptive managementIntegrated Resources Plan

RAND is using these approaches to help theMetropolitan Water District’s:

22

Some Key Ideas for Adaptation Planning

• Approach described here uses sophisticated analytic toolswithin a specific process of stakeholder engagement

• Key idea is even more broadly applicable:• Use analysis to identify vulnerabilities of specific plans and

compare robust responses• See for instance, suite of approaches described by World

Resources Institute: http://www.worldresourcesreport.org/decision-making-in-depth/managing-uncertainty

Encourage policy makers to change the question from

“What will the future bring?”

to

“What steps can we take today to most assuredly shape thefuture to our liking?”

23

More Information

Bryant, B. P., and Lempert, R. J., "Thinking inside the box: A participatory, computer-assisted approach to scenario discovery." Technological Forecasting and SocialChange, 77(1), 34-49, 2010.

Fischbach, J., Managing New Orleans Flood Risk in an Uncertain Future Using Non-Structural Risk Mitigation. Santa Monica, RAND. 2010.

David G. Groves, Robert J. Lempert, Debra Knopman, Sandra H. Berry: Preparing foran Uncertain Climate Future: Identifying Robust Water Management Strategies,RAND DB-550-NSF, 2008.

David G. Groves, Debra Knopman, Robert J. Lempert, Sandra H. Berry, and LynneWainfan, Presenting Uncertainty About Climate Change to Water ResourceManagers, RAND TR-505-NSF, 2007.

Lempert, R., and Collins, M. (2007). "Managing the Risk of Uncertain ThresholdResponses: Comparison of Robust, Optimum, and Precautionary Approaches." RiskAnalysis, 27(4), 2007.

Steven W. Popper, Robert J. Lempert, and Steven C. Bankes: "Shaping the Future,"Scientific American, vol 292, no. 4 pp. 66-71, April 2005

www.rand.org/ise/projects/improvingdecisions/

24

Thank you!

decision making in an uncertain world · fischbach, j., managing new orleans flood risk in an...

Documents