effectiveness monitoring: what’s working to restore puget sound? leska fore, puget sound...
TRANSCRIPT
Effectiveness Monitoring: What’s Working to Restore Puget Sound?
Leska Fore, Puget Sound PartnershipConstance Sullivan, Friday Harbor LabsKen Dzinbal, Puget Sound PartnershipScott Collyard, Dept. of Ecology
& Many Contributors
West Sound Watersheds Council & West Central LIO
January 21, 2015
• Measures of biological or physical parameters to determine whether actions were effective in creating a specific outcome.
-- Redman et al., 2013
What do we mean by “Effectiveness Monitoring”?
• Measures of biological or physical parameters to determine whether actions were effective in creating a specific outcome.
-- Redman et al., 2013
What do we mean by “Effectiveness Monitoring”?
Other fields call this:
Program Evaluation
1 Context for Effectiveness Monitoring
2 Statistical Meta-analysis
3 Communication
TOPICS
Action
1 Context for Effectiveness Monitoring
Ecosystem
Pressure
Animals
Habitat
Action
1 Context for Effectiveness Monitoring
Ecosystem
Pressure
Animals
Habitat
• Did the trees live?
Action
1 Context for Effectiveness Monitoring
Ecosystem
Pressure
Animals
Habitat
• Did the trees live?
• Did temperature go down?
Action
1 Context for Effectiveness Monitoring
Ecosystem
Pressure
Animals
Habitat
• Did the trees live?
• Did temperature go down?
• Are there more salmon?
Action
1 Context for Effectiveness Monitoring
Ecosystem
Pressure
Animals
Habitat
• Did the trees live?
• Did temperature go down?
• Are there more salmon?
• Did salmon population increase?
Action
1 Context for Effectiveness Monitoring
Ecosystem
Pressure
Animals
Habitat
• Did the trees live?
• Did temperature go down?
• Are there more salmon?
• Did salmon population increase?
Easy to measure
Harder to measure
1 Context for Effectiveness Monitoring
2 Statistical Meta-analysis3 Communication
TOPICS
“System-wide learning”
“Shared measures”
“Collaborative learning”
2 Statistical Meta-Analysis
An analysis approach that supports:
2 Statistical Meta-Analysis
Many restoration projects across Puget Sound…
…how do we compare without using the same methods?
• Widely used in education and medical research
• Change statistics are standardized across projects
• Measures effectiveness of actions
Before and After Statistics
TempAfter – TempBefore
(Pooled Variance)0.5Change = = Cohen’s d
Change statistic (Cohen’s d) is difference of the means divided by the standard deviation
2 Statistical Meta-Analysis
Before Action After
1986 2001 2010
Musselwatch begins
BAN on TBT
2 Statistical Meta-Analysis
Effectiveness of Tributyltin BAN
16* Effect size +/- 90% CI
ACTION: Ban Tributyltin (anti-fouling paint)EFFECTIVE: Reduced toxics in Mussels
Tributyltin (Mussel)
17* Effect size +/- 90% CI
Tributyltin (Mussel)
Interpretation
Large effect size >0.8Medium 0.2 – 0.8Small effect < 0.2
ACTION: Ban Tributyltin (anti-fouling paint)EFFECTIVE: Reduced toxics in Mussels
18
Not effective/ Effect Size Effective/Worsening Improving
Fecal bacteria reduced
ACTIONS: Samish Basin - Inspections, farm plansEFFECTIVE: Reduced fecals, opened shellfish beds
19
Variety of Actions evaluated for Effectiveness in Puget Sound
Vetting the Results2 Statistical Meta-Analysis
PSEMP Steering Committee PSEMP Toxics Work GroupDale NortonRob DuffGeorge OnwumereScott CollyardLincoln LoehrJim WestSandie O’NeillScott RedmanTracy CollierMark MeyersLyndal Johnson
Kate O’LaughlinJosh LatterellJason ToftMaggie Bell-McKinnonScott CollyardDebby SargeantMindy FohnStuart WhitfordRick HaleyMichael SeePascale Warren
Effectiveness Monitoring & the Action Agenda• Nearshore habitat restoration increased benthic taxa richness
B2. Protect and restore nearshore and estuary ecosystems
• Pesticide partial ban not effective; Eagle Harbor cap improved fish health; TBT ban was effective
C1. Prevent, reduce, and control the sources of toxic contaminants entering Puget Sound
• Lake treatments reduced PhosphorusA2. Protect and restore upland, freshwater, and riparian ecosystems
• PIC programs in Liberty Bay reduced fecals in bays and streamsC7. Ensure abundant, healthy shellfish for ecosystem health and for
commercial, subsistence, and recreational harvest consistent with ecosystem protection
1 Context for Effectiveness Monitoring
2 Statistical Meta-analysis
3 Communication
TOPICS
3 Communication
Science
Decisions
3 Communication
Science
Decisions
Who is your audience?
3 Communication
Intended audience for results of Effectiveness Monitoring
• Project Sponsors• Project Implementers• Funders
What do they need to know?
• Actions• Outcomes• Costs• Challenges
Actions Outcomes Costs
Fact Sheet + Narrative• Samish Bay: Fecals & Shellfish• King Co: Reforestation• Nisqually: Estuary restoration
www.psp.wa.gov/effectiveaction.php
3 Communication
28
29
1 Surprising amount of data to measure effectiveness and compare costs.
2 Info buried in tech reports.
3 Don’t wait for standardization – use change statistics now.
4 Lots of interest in what’s working to restore Puget Sound.
CONCLUSIONS
Coming up next…
Effectiveness of Estuary Restoration• Tidal wetlands• Floodgates• Channel formation• Species use