![Page 1: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/1.jpg)
WELCOME to the webinar
“How to evaluate interventions in
complex dynamic environments?” 28 February 2012
This Live Webinar will start at 11:30 AM, New York time.
All microphones & webcams are disabled and we will only enable microphones during
the Q&A portion.
Therefore, you will not hear any sound/noise till the beginning of the webinar.
![Page 2: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/2.jpg)
Series of 17 live webinars on
“Equity-focused Evaluations” Interact live with 28 world-level evaluators
This series of webinars addresses the challenges and
opportunities in evaluating the effects of policies, programmes
and projects to enhance equitable development results, with a
special focus on the effects to the most excluded, marginalized
and deprived groups.
![Page 3: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/3.jpg)
“Evaluation for Equitable Development Results”
will be available early 2012
Available in MyM&E
![Page 4: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/4.jpg)
The book will be available at MyM&E Virtual Library
www.mymande.org
![Page 5: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/5.jpg)
Colin KIRK
Penny HAWKINS
Evaluation to accelerate progress towards
equitable development
6 September 2011
9:30 AM NY time
Belen SANZ
Flaminia MINELLI
Human rights and Gender equality in evaluations
21 September 2011
9:30 AM NY time
Marco SEGONE
Michael BAMBERGER
How to design, implement and use equity-
oriented evaluations
4 October 2011
11:30 AM NY time
Saville KUSHNER
Case study evaluation as an intervention for
promoting equity
11 October 2011
9:30 AM NY time
Bob WILLIAMS
Martin REYNOLDS
Systems approach (CSH) to address ethical
issues
14 November 2011
3:00 PM NY time
Patricia ROGERS
Richard HUMMELBRUNNER
Program theories and LogFrames to evaluate
pro-poor and equity programs
22 November 2011
4:00 PM NY time
Michael Quinn PATTON
Developmental Evaluation
6 December 2011
11:30 AM NY time
Webinars on Equity-focused Evaluation 2011
![Page 6: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/6.jpg)
Webinars on Equity-focused Evaluation 2012
Donna MERTENS
Methodological guidance in evaluation for Social
Justice
24 January 2012
9:30 AM NY time
Jennifer GREENE
Values-Engaged Evaluation
15 February 2012
1:00 PM NY time
Michael Quinn PATTON
How to evaluate interventions in complex
dynamic environments?
28 February 2012
11:30 AM NY time
Rodney HOPSON
Katrina BLEDSOE
Cultural Responsiveness in Applied Research
and Evaluation Settings
15 March 2012
2:00 PM NY time
Francisco GUZMAN
Evaluation of the ILO’s strategy to eliminate
discrimination in employment and occupation
April 2012
Juha UITTO
Oscar GARCIA
Evaluating equity-focused public policies. The
case of Brazil and Mexico
April 2012
Katherine HAY
Sanjeev SRIDHARAN
A healthy discomfort? Development, equity and
evaluation
Ten Questions that Evaluations of Health Equity
Initiatives should answer
May 2012
Julian BARR
Ken CHOMITZ
Evaluation of climate change interventions for
excluded populations
June 2012
![Page 9: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/9.jpg)
Interact with Questions and Answers
Type here
![Page 10: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/10.jpg)
Keynote Speaker
Michael Quinn PATTON, Founder and Director,
Utilization-Focused Evaluation, and former President of
the American Evaluation Association
![Page 11: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/11.jpg)
Agenda 11:30 – 11:35 Welcome and introduction
Marco Segone, Systemic Management, UNICEF
Evaluation Office
11:35 – 11:55 Michael Quinn PATTON, Founder and Director, Utilization-
Focused Evaluation, and former President of the American
Evaluation Association
11:55 – 12:25 Questions and Answers
Moderator: Stewart Donaldson, Dean & Chair of
Psychology School of Behavioral & Organizational Sciences,
Claremont Graduate University
12:25 – 12:30 Wrap-up: Penny Hawkins, Evaluation Office,
The Rockefeller Foundation
![Page 12: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/12.jpg)
How to evaluate interventions in complex dynamic environments:
Developmental Evaluation
Michael Quinn Patton
![Page 13: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/13.jpg)
Challenge:
Matching the evaluation process
and design to the nature of the
situation
13
![Page 14: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/14.jpg)
Seeing Through A
Complexity Lens
14
![Page 15: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/15.jpg)
Complex Nonlinear Dynamics
• Nonlinear: Small actions can have large reactions. “The
Butterfly Wings Metaphor”
• Emergent: Self-organizing, Attractors
• Dynamic: Interactions within, between, and among
subsystems and parts within systems can volatile,
changing
• Getting to Maybe: Uncertainty, unpredictable,
uncontrollable
15
![Page 16: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/16.jpg)
Getting to Maybe:
How the World Is
Changed?
Frances Westley,
Brenda Zimmerman,
Michael Q. Patton
Random House
Canada,2006
16
![Page 17: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/17.jpg)
17
Chapter 4
Situation Recognition and
Responsiveness:
Distinguishing Simple,
Complicated and Complex
![Page 18: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/18.jpg)
Conceptual Options
• Simple
• Complicated
• Complex
18
![Page 19: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/19.jpg)
Situation Analysis
Matrix:
Mapping the Territory
Degree of Certainty
Close to Far from
Clo
se t
o
19
![Page 20: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/20.jpg)
Simple Space
Certainty Close to Far from
Clo
se t
o
Simple Plan, control
20
![Page 21: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/21.jpg)
Following a Recipe A Rocket to the Moon Raising a Child
Complicated Complex
zThe recipe is essential
zRecipes are tested to assure replicability of later efforts
zNo particular expertise; knowing how to cook increases success
zRecipes produce standard products
zCertainty of same results every time
Simple
21
![Page 22: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/22.jpg)
Technically Complicated
Certainty Close to Far from
Clo
se t
o
Simple Plan, control
Technically Complicated Experiment, coordinate expertise
22
![Page 23: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/23.jpg)
Following a Recipe A Rocket to the Moon Raising a
Child • Formulae are critical and necessary
• Sending one rocket increases assurance that next will be ok
• High level of expertise in many specialized fields + coordination
• Rockets similar in critical ways
• High degree of certainty of outcome
Complicated Complex
zThe recipe is essential
zRecipes are tested to assure replicability of later efforts
zNo particular expertise; knowing how to cook increases success
zRecipes produce standard products
zCertainty of same results every time
Simple
23
![Page 24: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/24.jpg)
Socially Complicated
Certainty Close to Far from
Clo
se t
o
Simple Plan, control
Technically Complicated Experiment, coordinate expertise
Socially Complicated Build relationships, create common ground
24
![Page 25: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/25.jpg)
Socially complicated
Implementing human rights agreements, like gender equity or outlawing child labor
Environmental Initiatives Many different and competing stakeholders
Diverse vested interests
High stakes
25
![Page 26: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/26.jpg)
Socially complicated situations
pose the challenge
of coordinating and
integrating
many players
26
![Page 27: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/27.jpg)
Know When Your Challenges Are In the
Zone of Complexity
Certainty Close to Far from
Clo
se t
o
Simple Plan, control
Zone of Complexity
Technically Complicated Experiment, coordinate expertise
Socially Complicated Build relationships, create common ground
27
![Page 28: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/28.jpg)
Following a Recipe A Rocket to the Moon Raising a
Child
Sending one rocket increases assurance that next will be ok
High level of expertise in many specialized fields + coordination
Rockets similar in critical ways
High degree of certainty of outcome
• Formulae have only a limited application
• Raising one child gives no assurance of success with the next
• Expertise can help but is not sufficient; relationships are key
• Every child is unique
• Uncertainty of outcome remains
Complicated Complex
zThe recipe is essential
zRecipes are tested to assure replicability of later efforts
zNo particular expertise; knowing how to cook increases success
zRecipes produce standard products
zCertainty of same results every time
Simple
28
![Page 29: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/29.jpg)
Following a Recipe A Rocket to the Moon Raising a Child
• Formulae are critical and necessary
• Sending one rocket increases assurance that next will be ok
• High level of expertise in many specialized fields + coordination
• Separate into parts and then coordinate
• Rockets similar in critical ways
• High degree of certainty of outcome
• Formulae have only a limited application
• Raising one child gives no assurance of success with the next
• Expertise can help but is not sufficient; relationships are key
• Can’t separate parts from the whole
• Every child is unique
• Uncertainty of outcome remains
Complicated Complex
zThe recipe is essential
zRecipes are tested to assure replicability of later efforts
zNo particular expertise; knowing how to cook increases success
zRecipe notes the quantity and nature of “parts” needed
zRecipes produce standard products
zCertainty of same results every time
Simple
29
![Page 30: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/30.jpg)
“A Leader's Framework for Decision
Making” by David J. Snowden and
Mary E. Boone, Harvard Business
Review,
November, 2007:
Wise executives tailor their approach to fit
the complexity of the circumstances
they face.
30
![Page 31: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/31.jpg)
Wise evaluators tailor their
approach to fit the complexity of
the circumstances they face.
31
![Page 32: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/32.jpg)
Prospective Example
Leadership Program Example
Simple elements
Complicated elements
Complex elements
32
![Page 33: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/33.jpg)
Simple outcomes
• Increase knowledge and skills of
participants
Evaluation: Pre-post data and
documentation of learning
33
![Page 34: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/34.jpg)
Complicated Impacts
• Change participants’ organizations
Evaluation:
Case studies
of
organizational change
34
![Page 35: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/35.jpg)
Complex Vision
• Infuse energy into the moribund
not-for-profit (voluntary) sector
• Make the sector more dynamic
• Create network of leaders who
actively engage in change
35
![Page 36: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/36.jpg)
Evaluating the Complex
• Real time follow-up of network
connections and actions
• Follow-up is an intervention
• Rapid feedback of findings permits
infusion of resources in support of
emergent outcomes
36
![Page 37: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/37.jpg)
Alice in Wonderland
• Evaluation lesson from her encounter
with the Cheshire Cat
37
![Page 38: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/38.jpg)
Complex
Interdependencies
38
![Page 39: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/39.jpg)
Insert action into the
system
39
![Page 40: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/40.jpg)
EMERGENCE
40
![Page 41: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/41.jpg)
Dealing with the Unexpected
and Unpredicted
41
![Page 42: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/42.jpg)
Taking Emergence Seriously
• Beyond “unanticipated
consequences” to
genuine openness
42
![Page 43: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/43.jpg)
Paradigms and Lenses
• The importance of interpretive
frameworks
• Complexity as an interpretive
framework
43
![Page 44: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/44.jpg)
44
2011 Nobel Prize in Economics
The Nobel in economic science was awarded Monday (Oct
10, 2011) to Thomas J. Sargent at New York University
and Christopher A. Sims at Princeton University for their
research on the cause and effect of government policies
on the broader economy, a major concern of countries
still struggling to address the aftermath of the recent
financial crisis.
Sims said; “The methods that I’ve used and that Tom has
developed are central for finding our way out of this
mess.” But asked for specific policy conclusions of his
research, he responded, “If I had a simple answer, I
would have been spreading it around the world.”
Dr. Sims, 68, is president-elect of the American Economic
Association. http://www.nytimes.com/2011/10/11/business/american-economists-share-nobel-
prize.html?hp
![Page 45: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/45.jpg)
Conditions that challenge traditional model-testing
evaluation
• High innovation
• Development
• High uncertainty
• Dynamic
• Emergent
• Systems Change
Adaptive
Management
and
Developmental
Evaluation
45
![Page 46: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/46.jpg)
46
Attribution analysis
vs.
Contribution Analysis
![Page 47: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/47.jpg)
47
Developmental Evaluation Defined
Purpose: Developmental evaluation (DE) informs and supports
innovative and adaptive development in
complex dynamic environments.
DE brings to innovation and adaptation the processes of
asking evaluative questions, applying evaluation logic, and
gathering and reporting evaluative data to support project,
program, product, and/or organizational development with
timely feedback.
![Page 48: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/48.jpg)
Key DE Characteristics
• Focus on development (versus improvement,
accountability or summative judgment)
• Takes place in complex dynamic environments
• Feedback is rapid (as real time as possible).
• The evaluator works collaboratively with social
innovators to conceptualize, design and test
new approaches in a long-term, on-going
process of adaptation, intentional change,
and development.
48
![Page 49: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/49.jpg)
Mintzberg on Strategy
Unrealized Strategy
Intended
Strategy
Deliberate
Strategy
Realized
Strategy
Emergent Strategy
20
![Page 50: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/50.jpg)
50
Primary developmental evaluation purpose
Complex system challenges Implications
1. Ongoing development Being implemented in a complex and dynamic environment
No intention to become a fixed/standardised model Identifies effective principles
2. Adapting effective principles to a new context
Innovative initiative develops ‘own’ version based on adaption of effective principles and knowledge
Top-down—general principles knowledge disseminated Bottom-up—sensitivity to context, experience, capabilities and priorities Adaptation vs Adoption
3. Developing a rapid response in turbulent disaster situations
Planned interventions must adapt and respond as conditions change suddenly
Planning, execution and evaluation occur simultaneously
Five purposes of developmental evaluation
![Page 51: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/51.jpg)
51
Primary developmental evaluation purpose
Complex system challenges Implications
4. Pre-formative development of potentially scalable innovative
Changing and dynamic situations require innovative solutions to worsening conditions Model needs to be developed/does not exist
Models may move into formative and summative evaluation, others remain in developmental mode Inform different potential scaling options
5. Major systems change and cross scale developmental evaluation
Disrupt existing system Taking an innovation to scale Major systems change and changing scale will add levels of complexity, new uncertainties and disagreements
System is the unit of change (and unit of analysis) Adaptive cross-scale systems change innovations assume complex, nonlinear dynamics requiring agility and responsiveness
Five purposes of developmental evaluation
![Page 52: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/52.jpg)
Some Particularly Appropriate
Applications of DE:
Examples of Innovative Arenas
• Social Movements and networks
• Human rights and advocacy Evaluation
• Large-scale, cross-sector, collaborative
initiatives
• R & D in public health, technological
innovation, science
• Public policy implementation
52
![Page 53: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/53.jpg)
53
![Page 54: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/54.jpg)
References
Developmental Evaluation: Applying Complexity
Concepts to Enhance Innovation and Use.
Guilford Press, 2011.
Utilization-Focused Evaluation, 4th ed.,
Michael Quinn Patton, Sage, 2008.
Essentials of Utilization-Focused Evaluation,
Michael Quinn Patton, Sage, 2012
54
![Page 55: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/55.jpg)
55
Website sample chapter:
http://www.guilford.com/excer
pts/patton.pdf
website for the book:
http://www.guilford.com/cgi-
bin/cartscript.cgi?page=pr/patt
on.htm&dir=research/res_eval
&cart_id=824067.29797
![Page 56: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/56.jpg)
THANK YOU!
![Page 57: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/57.jpg)
Questions and Answers
Stewart Donaldson, Dean & Chair of Psychology
School of Behavioral & Organizational Sciences,
Claremont Graduate University
MODERATOR
![Page 58: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/58.jpg)
Audience Questions
![Page 59: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/59.jpg)
Wrap-up
Penny Hawkins, Rockefeller Foundation Evaluation
Office, is the former Head of Evaluation for the New
Zealand Aid Program, Ministry of Foreign Affairs and
Trade and Vice-Chair of the OECD-DAC Network on
Development Evaluation. She is a past President of the
Australasian Evaluation Society, a founding board
member of the International Organization for Cooperation
in Evaluation (IOCE) and an IPDET (International
Program for Development Evaluation Training) faculty
member.
![Page 61: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/61.jpg)
Next webinar, 15 March 2012,
2:00PM New York time
Rodney HOPSON, Duquesne University
“CULTURAL RESPONSIVENESS in
EQUITY-FOCUSED EVALUATIONS”
Katrina BLEDSOE, Education Development Center,
Washington
![Page 62: WELCOME to the webinar - EvalPartners · 2016-07-07 · WELCOME to the webinar “How to evaluate interventions in complex dynamic environments?” 28 February 2012 This Live Webinar](https://reader036.vdocument.in/reader036/viewer/2022070909/5f98f783d517c10a746877e5/html5/thumbnails/62.jpg)
Evaluation of Webinars
Survey
Your opinion/feedback is important to us, therefore
we ask that you complete this short evaluation on
today’s webinar.
http://2.28.2012.questionpro.com