evaluating the performance of collaborative environmental … · 2007-09-05 · 1. evaluating the ....
TRANSCRIPT
![Page 1: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/1.jpg)
1
Evaluating the Performance of Collaborative Environmental
Governance
Tomas M. KoontzSchool of Environment & Natural Resources
Ohio State University
Craig W. ThomasDaniel J. Evans School of Public Affairs
University of Washington
![Page 2: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/2.jpg)
2
The Problem We FaceCollaboration has increasingly replaced
other forms of environmental governance • Such as centralized planning and
regulation• This may be a good thing…
But we have very little knowledge about the environmental outcomes of collaboration
• Collaboration appears to be good for social outcomes (like social capital)
• But it may be bad for the environment
![Page 3: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/3.jpg)
3
What is Collaborative Governance?
Multiple governmental and nongovernmental stakeholders working together to develop and advance a shared vision
![Page 4: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/4.jpg)
4
Research Questions
Is collaborative governance good for the environment?
Does collaborative governance produce better environmental outcomes than other forms of governance (such as centralized planning or regulation)?
![Page 5: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/5.jpg)
5
How Collaboration Might Affect the Environment
Negatively: Lower common-denominator solutions during policy adoption
Positively: More durable solutions during implementation
![Page 6: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/6.jpg)
6
Inputs(e.g., human,
financial,and technicalresources)
Endoutcomes
(social andenvironmental)
Processes(e.g., collabor-
ation)
Intermediateoutputs
(e.g., plans)Intermediate
outcomes(e.g. humanbehavior)
All othernon-programmatic
variables
Endoutputs
(e.g., projects)
Basic Logic Model
![Page 7: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/7.jpg)
7
State of the Literature on Collaborative Environmental
GovernanceEmpirical research has focused on:• Inputs (e.g., human, technical, financial)• Processes (e.g., consensus, ADR)• Outputs (e.g., plans, permits, projects) • Social outcomes (e.g., trust, social capital)
Very little research on environmental outcomes
![Page 8: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/8.jpg)
8
Recent Empirical Claims about Environmental Outcomes
Leach and Sabatier (2005) measure participant perceptions of environmental change
Others use outputs as proxies for outcomes (either intentionally or carelessly)- Salmon recovery plans; farmland preservation; Superfund projects; “choices”- Beware: outputs may not be good proxies for outcomes, due to intervening variables
![Page 9: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/9.jpg)
9
Possible Research Designs
Experimental
Quasi-experimental
Statistical
Case Studies
![Page 10: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/10.jpg)
10
Experimental Methods What
–Randomized trials–With focus on collaboration as a
management intervention
Feasibility–Very low–Primarily due to long time horizons for
environmental change
![Page 11: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/11.jpg)
11
Quasi-Experimental Methods What
– Also focus on collaboration as a management intervention
– But w/o randomized controls or other features of classic experimental design
Feasibility– Easy to find natural experiments– Harder to find matched-case controls (using
most-similar comparative methods)– Harder to find valid longitudinal data
![Page 12: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/12.jpg)
12
Statistical Methods
What– Analyzes correlations among variables– to test probability of causal effects (not causal
mechanisms)
Feasibility– High for existing data sets
e.g., PART (with caveat)– Harder to find valid longitudinal data
![Page 13: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/13.jpg)
13
Inputs(e.g., human,
financial,and technicalresources)
Endoutcomes
(social andenvironmental)
Processes(e.g., collabor-
ation)
Intermediateoutputs
(e.g., plans)Intermediate
outcomes(e.g. humanbehavior)
All other non-programmatic(exogenous) variables
Endoutputs
(e.g., projects)
Experimental, Quasi-Experimental, and Statistical Methods
![Page 14: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/14.jpg)
14
Case Study MethodsWhat
– Emphasizes process tracing in causal sequences
– Searches for causal mechanisms, not just causal effects
Feasibility– High– But requires careful use of counterfactuals,
congruence with theory, and controlled comparisons (holding other variables constant)
![Page 15: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/15.jpg)
15
Inputs(e.g., human,
financial,and technicalresources)
Endoutcomes
(social andenvironmental)
Processes(e.g., collabor-
ation)
Intermediateoutputs
(e.g., plans)Intermediate
outcomes(e.g. humanbehavior)
All other non-programmatic(exogenous) variables
Endoutputs
(e.g., projects)
Case Studies
![Page 16: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/16.jpg)
16
Accountability for Performance
Collaboration should be held to the same performance standards as other forms of governance
Collaboration may be quite complementary to developing performance mgt systems– Incentives for partners to monitor, gather
information, develop metrics for success
But how can we do it from a methodological perspective?
![Page 17: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/17.jpg)
17
Our Research AgendaIn process• Case studies of public sector collaborative efforts
• To show links from processes through to outcomes
• Coding PART reports• Matched case comparisons for statistical analysis:
collaborative v. non-collaborative cases within programs• But many outputs/outcomes are miscoded
• Link to theory for generalizability
Future Research• Quasi-experimental method
• Using remote sensing data, water quality reports, etc.• Ideas for comparative cases, anyone?
• How do perf. mgt. systems get started?
![Page 18: Evaluating the Performance of Collaborative Environmental … · 2007-09-05 · 1. Evaluating the . Performance. of Collaborative Environmental Governance. Tomas M. Koontz. School](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2aae5a9748fa4a2442bd10/html5/thumbnails/18.jpg)
18
Conclusion• Is collaborative governance good for the
environment?
• Does collaborative governance produce better environmental outcomes than other forms of governance (such as centralized planning or regulation)?
These are very important (and too often unasked) questions
But they are difficult to answer
And they will keep us busy for a long time