0

Funders & Evaluators Meeting

October 21-

22,2014

Baltimore,

MD

Framing Questions

1

• What leadership strategies are contributing to measurable changes in the condition of well-being of people, their communities and the planet?

• What leadership assessment frameworks hold promise for measuring the

contributions of high impact leadership strategies to community, population and systems level results?

• How can we aggregate data and findings across programs and initiatives to

accelerate our learning about the efficacy of different leadership strategies?

Research Preparation Methodology

2

Interviews • 18 funders investing in leadership

• 13 evaluation thought leaders

Review of Current Research • Analysis of successful evaluation of initiatives

• Review of current scholarly research

• Scan of innovative leadership approaches

WHY ARE FUNDERS INVESTING IN LEADERSHIP DEVELOPMENT?

3

4

“ No one organization is going to be able to advance the level and depth of change that needs to happen on any issues that we deal with.” (Funder)

“People need to learn how to work more collaboratively.” (Funder)

Assumptions informing leadership investments

5

Require collective leadership

Image Source: http://theteemingbrain.wordpress.com/2011/08/25/bank-of-america-four-alarm-emergency-possible-failure-imminent-and-boa-is-just-the-canary-in-the-coal-mine/

Complex problems…

The leadership environment is changing

6

We need a new kind of leadership

Networks, movements and systems level change needs connected and collaborative leadership

7

Individual Leadership

• Leader of followers

• Setting vision and directing

• Control and planning

• Exercising power

• Leadership hierarchy

• Centralized decision making

• Personal claim or blame

• Individual responsibility

Collective Leadership

• Self as leader

• Aligning purpose and actions

• Adaptive action learning

• Transparent power sharing

• Relational shared leadership

• Collective input and process

• Group reflection/learning

• Group accountability

• Group creativity and wisdom

Collective leadership requires new behaviors

8

“ Funders ultimately fund people,

not projects, and support work

because leadership is compelling,

visionary and smart.” (Funder)

“It’s about people. You can’t support

organizations and think you can make

change without supporting the people

doing the work.” (Funder)

People are at the Heart of all Change

WHAT CHANGES ARE FUNDERS HOPING TO ACHIEVE?

9

Developing a Common Evaluation Language

10

One of the obstacles to learning across leadership development evaluations is the lack of a common language and frame. For this report we suggest the following usage of terms and welcome modifications and ideas about useful terminology at the meeting. Outcome: Often short term or medium term effects of an intervention’s outputs that produce changes in knowledge, awareness, behavior, skills, relationships, decision making, and social action. Results: Changes in a state or conditions often associated with a cause and effect relationship, e.g. a population condition of well-being. The external effects of an intervention. Impact: The social, economic and environmental contributions of the programs that are often long term and associated with program goals. Indicators: A specific piece of information that measures and quantifies all or part of the condition, behavior or experience that is the desired state of well-being or result. It may be necessary to use several indicators to get a sense of progress toward the desired goal.

Domains of Desired Impact Cited By Funders

11

•Fields & Systems

Examples: Desired Outcomes, Results & Impact

12

Examples Impact Domain Outcome Results

Many Leadership Programs

Individual & Organizational

Better manager & organizational leader

Improved community service

Barr Fellowship Network

Neighborhood & Community

Mission driven collaboration

Innovative projects transforming social sector

Building Healthy Communities

Neighborhood & Community

Collaboration Grassroots advocacy Youth organizing

Progress on community identified health results

New Leaders Network Neighborhood & Community

A cross sector collaborative leadership network

Positive development and innovation in Fresno

Examples: Desired Outcomes, Results & Impact

13

Examples Impact Domain

Outcome Results

Results Based Leadership (Leadership in Action)

Population Level Self agency Group alignment and commitment

Kids entering school ready to learn

Health Leadership Fellows Program

Population level Network of collaborative leaders

Better health outcomes for frail seniors & vulnerable children

ReAMP Energy Network

Systems and Field Leaders develop shared systems analysis and alignment on levels

Reducing carbon emissions (coal plant closing, policy)

MAR-L Systems and Field Collaborative and innovative leaders network

Policies that improve the health of the reef, “no-take zones” etc.

Leadership Outcomes

14

Stronger organizations

Increased collaborations

Networks, movement

building

Policy, advocacy

Social and racial equity

Based on funder interviews

WHAT IS BEING ACHIEVED THROUGH LEADERSHIP INVESTMENTS?

15

Measuring Population Level Results: Leadership in Action: Annie E. Casey Foundation

16

Results-based Leadership

17

LAP Program Theory

Theory of Aligned Contributions

Measuring Collaboration Outcome: New Leadership Network (James Irvine Foundation)

18

There are currently 86 NLN collaborations, 38 of which are resulting in real outputs in the prototype or testing stages

Source: The James Irvine Foundation New Leadership Network, Heather McLeod Grant, David Sawyer, David Ehrlichman

19

New Leadership Network Design

Source: The James Irvine Foundation New Leadership Network, Heather McLeod Grant, David Sawyer, David Ehrlichman

Measuring Indicators of Progress on Goal: ReAmp Energy Network (Garfield Foundation)

20

State

New

Coal

Stopped

Renewable

Standards

Energy

Efficiency

Transport

Fuels

Reduced

Miles

Driven

IL

IA

MI

MN

OH

WI

Success Policy in play

21

• 17 people from 10 nonprofits and 7

foundations from the midwest were selected

• The group was supported with systems

mapping and convened over a one year period building relationships and studying the system

• The group aligned around 4 levers of

change and then recruited others to the network

ReAMP Network Development Approach

ReAMP is not a traditional leadership program but shares similar cohort components

Measuring Systems Level Results: MAR-Leadership (Summit Fund and Partners)

22

Policy and Infrastructure: • Three communities in Guatemala established no-take zones • The creation of a Tri-national Alliance for the Gulf of Honduras coordinating

NGOs in Belize, Guatemala and Honduras focused on MPA’s

• The passage of a people’s referendum in Belize to stop offshore oil drilling

• Cordelia Bank declared a Site of Wildlife Importance, 5/12

23

Program Components • Cohort recruitment of young cross-sectoral

leaders from 4 countries in the MAR region to increase regional, cross-sector collaboration

• Responsible for planning and implementing

a conservation project • Personal visionary leadership • Policy training and data • Communications • Conservation specific issues

MAR-Leadership Program Design

Measuring Collaboration Outcomes: Health Leadership Program (Health Foundation for Western

and Central New York)

24

Cohorts 1-3 concluded their programs and 4 has just begun.

Cohort 1

Cohort 2

Cohort 3

Cohort 4

Advisors

Measuring Collaboration as a Leadership Outcome

25

Health Leadership Program

Measure Network Outcomes:

Barr Fellowship Network (Barr Foundation)

26

From Only Connect: How an Investment in Relationships Among Social Change Leaders Is Changing Them, Their Organizations, and Their City

Measuring Network Influence in Issue Clusters

27

Barr Fellowship Network

Measuring Advocacy Outcomes: Building Health Communities (The California Endowment)

28

29

“It’s not like we have one leadership program we are putting people through, we partner with other foundations and support multiple ways of becoming leaders. There are varying entry

points to start pulling together organizations to think about incorporating leadership development, youth organizing work.” (Funder)

Over the past 4 years of the Building Health Communities Initiative, TCE has invested in a number of leadership development strategies responding the specific needs or requests of the 14 sites throughout California. Here is a small sample of the diverse investments made: ● LeaderSpring: Runs a program attended by the Hub leaders from each of the sites focusing on how

to build strong collaborative relationships. ● Race Forward: Provides a framework on governing for racial equity attended by 15 people across

the sites. ● Urban Habitat Boards and Commissions Leadership Program: Focused on preparing community to

serve on boards and commissions and mobilize their constituents and networks. ● Rockwood: Supporting TCE staff to model collaboration and work more effectively across silos. ● Movement Strategy Center: Collective youth leadership development

Building Health Communities

WHAT ARE WE LEARNING ABOUT STRATEGIES THAT ARE ACHIEVING SIGNIFICANT RESULTS?

30

How Leadership Investments are Shifting

31

• Place-based leadership programming: W.K. Kellogg Foundation,

James Irvine Foundation, Barr Foundation, Annie E. Casey Foundation

• Results based leadership: Annie E. Casey Foundation, Promise

Neighborhood Institute, Health Foundation for Western and Central New York

• Networks and relationship building: Barr Foundation, Compton

Foundation, W.K. Kellogg Foundation, James Irvine Foundation, David and Lucile Packard Foundation

• Investing at scale: Kansas Health Foundation

• Shifting power: James Irvine Foundation, The California Endowment,

Surdna Foundation

• Integrating leadership into large scale initiatives: Open Society

Institute, The California Endowment, Hewlett Packard

Common Elements of Programs Focused on

Large Scale Results

32

Systems

Relationships

Self

• Understanding System

• Using Data

• Learning By Doing

• Focus on Real Problems

• Collaboration

• Boundary Crossing

• Building Networks

• Personal Agency

• Reflective & Adaptive

Learning from Successful Collaborations

33

We can learn about critical leadership competencies from these evaluations of collaborative work and initiatives

• Building bridges from the margins: The work of leadership in social change organizations. Sonia Ospina, Erica Foldy, Leadership Quarterly

• Collective Insights on Collective Impact: Stanford Social Innovation Review

• Needle Moving Community Collaboratives, The Bridgespan Group

• There is Something Happening Here: A Look at The California Endowment’s Building Healthy Communities, USC Program for Environmental and Regional Equity

• Leading for Results: Developing Talent to Drive Change, The Annie E. Casey Foundation

• Building Commitment to a Postsecondary Systems Change Agenda: OMG Center for Collaborative Learning

Common ingredients of successful initiatives

that leadership investments could bolster

34

Vision & Framing

Commitment, Alignment,

Accountability

Diverse Stakeholder Engagement

Equitable Governance

Structures & Processes

Data, Learning & Adaptation

WHAT ARE PROMISING STRATEGIES FOR EVALUATING AND LEARNING ABOUT THE CONTRIBUTIONS OF LEADERSHIP INVESTMENTS TO LARGE SCALE CHANGE?

35

“People want to see short-term change so you have to track that and

make that visible. At the same time we cannot allow ourselves to fall into

a linear short-term view. The main problem of foundation initiatives is

that the timeframe is far too short.” - Foundation V.P. of Learning

“We are awash in data. It’s more important to know which data we need

when, and who needs to process it, to what end. We need smarter

systems.” – Evaluation firm executive

“What are we doing to look across different models and approaches over

time? We need better data and not just tiny experiments. We need a

meta analysis and some larger scale experiments.” – Foundation Director

36

Funders & Evaluators on Evaluation

A Framework For Assessing Leadership

Networks

37

Pri

ma

ry N

etw

ork

Str

ate

gy

Bruce Hoppe and Claire Reinelt, Social Network Analysis and the Evaluation of Leadership Networks http://leadershiplearning.org/system/files/SNA%20and%20Leadership%20Networks%20-%20LQ.pdf

Key Outcomes for Measuring Four Types of

Leadership Networks

38

PEER LEADERSHIP

Trust

Peer advice and support

Learning from each other’s

experiences

COLLECTIVE LEADERSHIP

Self-organizing

Clustering

Communities of practice

ORGANIZATIONAL LEADERSHIP

Bridging silos

Aligned contribution

Learning across boundaries

FIELD-POLICY LEADERSHIP

Framing

Partnering

Mobilizing critical mass

Influencing direction and decision-

making

Assessment Tool for Aligned Contribution

39

Jolie Pillsbury and Raj Chawla, High Action and High Alignment, http://www.rbl-apps.com/PDF/HighActionHighAlignment.pdf?View+APP=View+The+APP

Integrating Leadership into Larger TOC

40

The California Endowment The Bill and Melinda Gates Foundation

Drivers of change for building healthy communities:

• Building resident power

• Developing youth leadership

• Enhancing collaboration

• Changing the narrative

• Leveraging partnerships

Drivers of change for improving post-secondary results:

• Building stakeholder

commitment

• Using data

• Building sustainable

partnerships

• Aligning policies and practices

Promising Learning & Evaluation Approaches

41

• Developing learning systems that integrate internal and external stakeholders in a learning process (e.g., Annie E. Casey Foundation, The California Endowment)

• Using network analysis to identify collaborations and harvest collaboration narratives (e.g., The Barr Foundation, The James Irvine, Hunt Alternatives, Health Leadership Fellows Program of Central and Western NY)

• Designing cluster evaluations to analyze multiple strategies for achieving a desired outcome (e.g., Community Leadership Project Evaluation - Irvine, Hewlett, and Packard, The California Endowment’s Boundary-Crossing Leadership Program Evaluation)

• Using scorecards and dashboards to focus attention on critical progress indicators (e.g., Re-AMP, Results Based Leadership, Promise Neighborhoods)

Critical Gaps and Opportunities

42

Challenge Opportunities Too many leadership measures without a shared language or framework for assessing leadership outcomes that matter

Conduct a meta-analysis to identify cross-cutting leadership measures, and evidence for what works in different contexts

Narrowly focused learning strategies and structures that limit understanding and inhibit learning about leadership development

Design cluster evaluations to learn across programs and initiatives

Inadequate learning systems that make access to data, knowledge-sharing, and field learning problematic

Experiment with new technologies, tagging systems and learning communities to accelerate shared learning

Evaluation timeframes that are too short to measure and capture the full contribution of leadership development to large-scale change

Invest in longitudinal (and/or retrospective) leadership studies that measure and narrate why and how leadership development makes a difference

Edge Questions for Leadership Investments

43

• What will it take for leadership programming to be infused with an understanding of leadership as a relational process and to experiment with new delivery strategies?

• How can we increase accountability for measuring the desired results of leadership investments and increase the validity of indicators of success?

• What would it take as a field to accelerate our learning about the impact of different leadership development strategies?

• How can we develop evaluation strategies that can keep pace with innovations in leadership development approaches that are seeking large scale change?