NOAA CLIMAS

NASA EOS

NSF SAHRANOAA GAPP

Considering Equity in Climate Prediction Applications: Implications for Development and

Evaluation of Decision Support Tools

1Department of Hydrology and Water Resources, University of Arizona2Department of Civil and Environmental Engineering, University of California-Irvine

Holly Hartmann1 and Soroosh Sorooshian2

NASA HyDIS

Raytheon Synergy

Calls for Societally Relevant Research and ProductsCalls for Societally Relevant Research and Products

• Information is appropriate to the knowledge and concerns of the recipient.

• Ensure that.. modeling improvements and data products are useful to the water resources management community.

• Develop a strategy for… how these could be made more useful for [user] purposes.

• Need studies of the benefits and costs of [hydroclimatic] information services.

• Increase the value of weather and related … information to society.

• Bring scientific outputs and users’ needs together.

• Make climate forecasts more socially useful.

• Stronger sense of responsibility for delivering timely and relevant tools.

• Accelerate activities to integrate science with the needs of decision makers.

• Integrate user needs… and ensure that research results are provided in a form useful for users.

Sources: Various USGCRP and NRC reports, 1997-2001

Issue: So Many Stakeholders!Issue: So Many Stakeholders!

Continental Scale: Focus of modelers

Watershed/Local Scale: Where impacts happen Where stakeholders exist

Different Scales (time & space)Different IssuesDifferent Stakeholders

Poor interactions with users affects:• opportunities for future work• credibility of agencies, institutions and products

“What are your motives?” (agenda)

“How long is your project really going to last?” (failed promises of past projects)

“What did you do with the last survey?” (checking your responsiveness)

Lessons from StakeholdersLessons from Stakeholders

Building Expectations and Trust

• Building trust requires repetition & responsiveness

• Concerns: agendas, science will be used to hurt them

• Effective stakeholder integration generate support for science funding & programs

• Changed decisions & decision processes

• Enabling system-wide change (transferability, scalability)

• Public support for climate research

Concerns for Climate Science Enterprise

Project Objectives Affect…

• Metrics

• Structure of stakeholder interactions

• Research products

• Perceptions of climate science enterprise

• Research funding

Evaluating Societally Relevant Research and ProductsEvaluating Societally Relevant Research and Products

Objective: Economic EfficiencyObjective: Economic Efficiency

Metrics: Cost/benefits. Return on Investment.

Stakeholder Interaction Structure: Consultant-client relationship with high-value clients (e.g., hydropower).

Research Products: Customized Decision Support Systems. System optimization rules.

Perceptions: Science serving special interests. Increasing competitive imbalances.

Research Funding: By clients through private sector.

Objective: Agency ImpactObjective: Agency Impact

Metrics: Policy and regulatory impact.

Stakeholder Interaction Structure: Work with agencies. Important role for policy analysts/scientists.

Research Products: Traditional products. Refereed methodology and results. Hold up in court.

Perceptions: Science serving special interests, agendas. Increasing regulatory burden.

Research Funding: By managed sector, perhaps public.

Objective: Societal EquityObjective: Societal Equity

Metrics: Breadth/diversity of applicability, accessibility, usability. Sectoral ‘market’ penetration.

Stakeholder Interaction Structure: Engagement with diversity of stakeholders. Important role for social scientists. Potentially huge demand on researchers’ time.

Research Products: Diverse. Non-traditional, but not “dumbed down”.

Note: data << information << knowledge << wisdom

Perceptions: Science providing useable information and practical tools. Increasing capacity to adapt to climate variability.

Research Funding: Public.

Evaluating Success of Products and ProcessEvaluating Success of Products and Process

???

PRODUCTS: Forecasts

- traditional publications

- MS/PhD degrees

- newsletter outreach

- database of forecasts

- forecast evaluation tool

- “Climate in a Nutshell” (450+)

- presentations to stakeholder groups (25+)

- workshops (research/forecast/stakeholder) (8+)

Southwest Climate in a Nutshell

Tom Pagano Department of Hydrology & Water Resources

J.W. Harshbarger Bldg Room 324K

University of Arizona, Tucson AZ 85721

pagano@hwr.arizona.edu (520) 621-3973

Pho

to C

redi

t:Bri

an M

ayeu

x

Your Needs

What’s Potentially Predictable

Current Climate Forecasting Ability

Frequent interaction, from the outset

Interaction… not outreach! Getting and givingStarting where the stakeholders are

Moving dialogue & action forward

Common across all groups

Uninformed, mistaken about forecast interpretation

Understand implications of “normal” vs. “unknown” forecasts

Use of forecasts limited by lack of demonstrated forecast skill

Stakeholder Use of Hydroclimatic ForecastsStakeholder Use of Hydroclimatic Forecasts

Common across many, but not all, stakeholders

Have difficulty distinguishing between “good” & “bad” products

Have difficulty placing forecasts in historical context

Common across all groups

Uninformed, mistaken about forecast interpretation

Understand implications of “normal” vs. “unknown” forecasts

Use of forecasts limited by lack of demonstrated forecast skill

Stakeholder Use of Hydroclimatic ForecastsStakeholder Use of Hydroclimatic Forecasts

Unique among stakeholders

Relevant forecast variables, regions (location & scale), seasons, lead times, performance characteristics

Role of of forecasts in decision making

Technical sophistication: base probabilities, distributions, math

Common across many, but not all, stakeholders

Have difficulty distinguishing between “good” & “bad” products

Have difficulty placing forecasts in historical context

Efficiency

Work with hydropower agencies & other high-value clients

Develop customized evaluation tools

Transfer to agencies

Equity

Also work with stakeholders affected by changing supplies & policies

Develop tools for knowledge development and diverse decision processes

Requires on-going support of research products and tools

Impact

Work with regulatory & policy agencies

Inform water supply policy via peer-reviewed science & policy analysis

Forecast Assessment: CLIMAS AlternativesForecast Assessment: CLIMAS Alternatives

http://hydis6.hwr.arizona.edu/ForecastEvaluationTool/

Initially for NWS CPC climate forecasts

Six elements in our webtool:• Exploring Forecast Progression•Forecast Interpretation - Tutorials• Forecast Performance• Historical Context• Use in Decision Making • Details: Forecast Techniques, Research

1. Forecast Progression

Unknown

ECSometimes forecasters don’t know what the chances are…

EC - EQUAL CHANCES THE PROBABILITY OF THE MOST LIKELY CATEGORY CANNOT BE DETERMINED

= Unknown Chances!!

63%33%

3%

“+30% Chanceof Warm” Each colored

contour indicates a shift in the normal chances.

33%33%

33%

Climatology

Climatology is only a reference (1971-2000), not a substitute forecast

2. Forecast Tutorial

3. Forecast Performance Evaluation

Sub-setting: Seasons, Leadtimes, Regions

Criteria: Simple/Intuitive to Complex/Informative

Transparency: Data behind analysis

3. Forecast Performance Evaluation

Sub-setting: Seasons, Leadtimes, Regions

Criteria: Simple/Intuitive to Complex/Informative

Transparency: Data behind analysis

4. Historical Context for Forecasts

Recent History | Possible Futures

Requested by Fire managers…Applicable to any climate variable

20032002

Neutral Non-ENSO sequences

2004

La Nina

Wet Near- Dry

Normal

10 years had more than 3.7 inches10 years had less than 1.9 inches10 years were in the middle

Willcox Jan-March Total Precipitation 1930-2001

Year

Pre

cip

itati

on

(In

ch

es)

1971-2000

Willcox Jan-March Total Precip. 1971-2000

Dry

Norm

Wet

0”

1.9”

3.7”

8+”

Exceedance Probability

4. Historical Context for Forecasts

La NinaEl Nino

El Nino La Nina50% Wet 0%30% Norm 25%20% Dry 75%

Willcox, AZ: Precipitation, JFM

4. Historical Context for Forecasts

Custom real-time data access, analysis, and information

Value-added interpretation

Multiple entry points along continuum of sophistication

Opportunities and tools for increasing sophistication

Knowledge development emphasis vs. decision support

Data << Information << Knowledge << Wisdom

Are these concepts & tools transferable and scalable?

• Test with new products, inter-RISA opportunities

Lessons of FET for Climate Services

Other issues: Accessibility, ease of use (information management, updating)

Ease of Use Profile and Projects: save a history of your work on each "project", so you can return to your work any time, easily repeat past analyses using updated data.

Facilitating Information Intermediaries

Accessibility Report Generation

• create PDF reports of your analyses for non-Internet users

• automatically includes legends, data sourcing, contact information, caveats, explanations

• sections for user-customized comments

Future: Automated Updating & Additional Products: water supply forecasts, experimental climate forecasts, drought monitoring

Lessons Learned: Knowledge Development Tools

StakeholdersInformation

needs, understanding,

access Social ScienceEffective

communication

Natural Science

Forecast skill,interpretation

• Transferable, scalable tools are possible!

• Focus on knowledge development, not just data & information.

Lessons Learned: Knowledge Development Tools

StakeholdersInformation

needs, understanding,

access Social ScienceEffective

communication

Natural Science

Forecast skill,interpretation Computer

ScienceWeb

programming

• Transferable, scalable tools are possible!

• Focus on knowledge development, not just data & information.

Interactive webtools require major commitment and resources.

Prototypes insufficient!Stakeholders need reliable tools, which require solid software foundation, organized development, sustainability for maintenance and expansion.

Climate Research Impacts Society… Unexpectedly?

Have you heard comments about role of science?

Which objectives are supported by your research and products?

What are your success metrics?

Are there synergies in webtool development?