using indicators for program management and program assessment draft framework with examples donna...

Using indicators for program management and program

assessmentDraft framework with examples

Donna Podger

[email protected]

(916) 445-5269

Revised framework for indicators

• Evolution of what has come before• Consistent with current efforts • Framework to organize and present

existing monitoring data, provide mechanism to bring science and monitoring together and identify gaps.

Proposed ApproachPhase 1: Refine frameworkPhase 2: Try it out – develop examples

Independent science reviewPhase 3: Develop core set of indicators (web) Independent science reviewPhase 4: Refine core indicators and develop publication Independent science reviewPhase 5: Develop more complete set of indicators Independent science reviewPhase 6: Annual refinement – update of website, develop

publication Independent science review

Why use indicators?

• To help understand relationships between cause and effect

(indicators)

• To assess progress towards program goals

(performance measures)

Indicators for different audiences

Technical audience • Develop and refine

common understanding of the system

• In-depth, detailed, indicators at different scales

• Product: detailed web-based, GIS based, “drill down” for details

Non-technical audience• Describe progress

towards goals

• Fewer simple indicators and summarization of technical data

• Product: annual publication

3 levels of indicators

1. Administrative indicators ($, projects, programs)

2. Driver indicators (factors influencing outcome, including

management actions)

3. Outcome indicators (environmental state, DWQ, water supply

reliability)

Conceptual Models

Conceptual models• Be explicit as possible• Based on current scientific knowledge• Describe linkages between drivers and

outcomes, including a discussion of importance, understanding and predictability

• Basis of discussion for expected outcomes of management actions

• Can be quantitative models

Benefits of conceptual models

• Documents rationale for decision making• Allows multi-disciplinary review and

discussion• Reduces chances of faulty reasoning or

unintended consequences• Provides a basis for incorporating new

information and continually improving knowledge of system

Ideas for web-based communication product

CALFED Bay Delta Program

Using indicators to inform science, management and adaptive management

The CALFED Bay Delta Program is using indicators to help understand the relationships between management actions and outcomes related to program goals.

For a summary of the program as a whole, click on the mission statement below or on the elements of the graphic on this page.

The mission of the CALFED Bay Delta Program is to develop a long-term comprehensive plan that will restore ecological health and improve water management for beneficial uses of the Bay Delta system.

To find out more about progress towards each of the program’s goals: click on each of the goals listed below:

•Improve water supply reliability

•Restore the Bay-Delta ecosystem

•Improve water quality

•Enhance levee stability in the Delta


Back to indicators home page

2. Introduction slide

3. Framework overview

Framework

Process

Glossary

4. Program indicators and PM

5. Overall program Water supply reliability Water quality Ecosystem

RestorationLevee Integrity

6. Entire program

Policy WSR Policy DWQ Policy ERP Policy Levees Policy

7. Entire program

Implementation

8. Entire program

Cause & effect

WSR Implementation

DWQ Implementation

ERP Implementation

Levees Implementation

WSR Cause and effect

DWQ Cause and effect

ERP Cause and effect Levees Cause

and effect

Other WQ Cause and

effect

Other WQ Implementation

Other WQ Policy




The diagram at the right shows the relationship between the 3 levels of indicators and policy, management and science. A series of web pages explains the policy, implementation and outcomes related to the goals of the CALFED Bay Delta program. Click on the arrows to the right or the list below to find out more about the overall program:

•What are the goals defined by policy makers and the resources allocated to reach them?•How are the resources being implemented into actions?•How are actions and other factors causing progress towards goals?

To find out more about the specific progress and goals of the program, click on the links below:

•Improve water supply reliability•Restore the Bay-Delta ecosystem•Improve water quality•Enhance levee stability in the Delta


CALFED Bay Delta Program :

Policy and Administrative Indicators

This slide will describe or be a portal for explanations about who / how policy decisions are made including:

Defining the goals and objectives for the program

Giving short-term directions / plans

Allocating resources to achieve the goals

There may also be links to related organizations that make policy decisions and allocate resources to areas of mutual interest.

Policy decision makers

Defining goals and objectives

Setting direction

Allocating resources to achieve

goalsAdministrative indicators


CALFED Bay Delta Program : Implementation and Driver Indicators

This page will give a summary / portal to discuss how resources are being implemented to achieve actions. This will include a description of investigative efforts, monitoring, planning, and implementation actions.

Implementation agencies

Resources allocated

InvestigationsMonitoring

Actions

Planning

Administrative indicators

Driver indicators

Back to CBDA indicators page

CALFED Bay Delta Program : Understanding cause and effect

Drivers and Outcome indicators

System-wide Conceptual Model

Geographic Information Access

Wat

er s

uppl

y re

liabi

lity

Ecos

yste

m

Res

tora

tion

Wat

er Q

ualit

y

Leve

e sy

stem

in

tegr

ityOutcomes

Drivers

ActionsUncontrollable

factorsThis page will give a summary / portal to discuss an overall conceptual model that integrates the four program objectives, and presents information on key outcome indicators for each of the program goals.

Back to CBDA indicators page

Levee Integrity

CALFED Objective: Enhance Levee Stability in the Delta


The diagram at the right shows the relationship between the 3 levels of indicators and policy, management and science. A series of web pages explains the policy, implementation and outcomes related to the goals of the CALFED Bay Delta program. Click on the arrows to the right or the list below to find out more about the overall program:

•What are the goals defined by policy makers and the resources allocated to reach them?

•How are the resources being implemented into actions?

•How are actions and other factors causing progress towards goals?
