refreshing evaluation in support of the social movements revival
TRANSCRIPT
Refreshing Evaluationin Support of theSocial Movements Revival2017 American Evaluation Association Annual Conference
November 10, 2017
Existing approaches to evaluating policy change advocacy focus on
campaigns, educating the public or constituents, cultivating decisionmaker
champions, and advancing policy.
While these bodies of knowledge are useful starting places, they do not
adequately capture the development of movements into powerful social
forces, the organic and exponential engagement that can occur at later
stages of movements, the broader societal changes that movements can
catalyze, or the complicated dynamics within movements of movement
building organizations and other actors.
Traditional funding practices are incongruous with what movement building
requires. Movement-building requires long-term investments in organizing,
infrastructure, networks, and collaboration. But the lack of understanding
about movement structure, development, success factors, and needs
hinders the development of new funding practices for movements.
Guiding Questions
1. How do social movements relate to other kinds of grantee activities, such as organizing, field-building, advocacy and policy change, or leadership development?
2. How do you know that a movement is “healthy”? How do you know that it is developing and becoming stronger over time? How do you know it has sufficient capacity and infrastructure?
3. How do you know that a movement is successful, short of accomplishing its overall goal(s)? How can progress be measured? Which measures matter to funders and movement builders?
Approach and Methods
• Four phases of work
◦ Phase 1: Research
–Conversations with experts
–In depth literature review
◦ Phase 2: Design – Develop evaluation guidance
◦ Phase 3: Test – Pilot test guidance and tools
◦ Phase 4: Share – Share with evaluators, movement builders, and philanthropy
What we’ve learned about social movements…
Movements are distinct• “Social movement” label is
applied liberally
• Long-term and sustained collective action
• Commitment to large-scale social change
• Clearly defined, powerful opponent
• Extrainstitutional tactics
Movements are amorphous
• Ambiguous beginnings and endings
• Non-linear, fluid, and iterative development
• Evolving structure
Movement Moments
Movement Taxonomy
SM IndustrySM Industry
Participants ParticipantsParticipants Participants Participants Participants
Organizations OrganizationsOrganizations Organizations Organizations
Networks NetworksNetworks NetworksLeadersLeaders
LeadersLeaders
Leaders
Networks are foundational to movements• Networks are essential to movement building
• There is no single ideal network structure
• Digital technologies are changing network dynamics
• Resource constraints and competition hinder networks
Movements are hard to measure
• They are amorphous and continually changing
• They are larger than organizations
• They encompass diverse actors and actions
• They lack accepted metrics of success to show progress
Existing Evaluation Approaches
• A shared common language◦ Transactions Transformations Translations: Metrics that Matter for Building, Scaling,
and Funding Social Movements by Manual Pastor, Jennifer Ito, and Rachel Rosner (2011)
• Movement building benchmarks tied to the stages of a movement◦ Social Movements and Philanthropy: How Foundations Can Support Movement
Building by Barbara Masters and Torie Osborn (2010)
• Movement capacity assessment◦ Movement Capacity Assessment Tool by the Global Fund for Women (2016)
• Discussion/reflection guide◦ Movement Building Indicators by Maria Nakae, Moira Cowman, and Eveline Shen and
the Asian Communities for Reproductive Justice (2009)
• Evaluation tools/guidance that can be drawn from related sub-fields like organizing, policy change, leadership development, etc.
14
1: Movement Stages and Evaluation
Is it practical to use the social movements stages to guide the evaluation of social movements? Why or why not? How is it useful? How is it not useful?
2: Evaluand: Networks and/or Organizations
Contemporary scholarship on the structure of social movements has focused on the primacy of networks over organizations. And yet, many practical constraints--such as philanthropic funding and evaluation--focus on the organization as the unit of analysis. How should evaluators approach this dilemma? What can be learned by focusing the evaluation on the organization? Or on the network? Or both?
3: State of Evaluation Practice for Social Movements
Given your background or experience in evaluation, what other tools or approaches could be used for evaluating movements? What would be some limitations in evaluating social movements?
