module 8 - monitoring and evaluation
TRANSCRIPT
Monitoring & Evaluation (M&E)Module 8
Value of M & E
Learn from experience, and share this learning (especially important if you are part of an alliance/coalition)
Adapt plans to respond to events
Improve the effectiveness of future advocacy work
Ensure that resources are used in the most effective way possible
Be accountable to managers, colleagues and funders
What’s the Difference?!
Monitoring - is a process that tracks the implementation of activities
[It checks that we are implementing activities according to our action plan.]
Evaluation - assesses the results of our project at one point in time
[In short-scale projects, this would be on completion – but in longer-scale projects, it could be done periodically.]
Monitoring is about keeping to the plan:
Not about questioning the plan
It takes place during
And may involve diagnosis and remedial action to get back on track
Evaluation questions the plan:
Is/was it the best way?
Will objectives be/have they been achieved?
Were we on time (expected progress made?)
What have we learned?
What can we change to increase success?
Is it worth doing (again)?
Designing Your M&E System
Before designing your M&E system, think carefully about the changes you are trying to achieve, and how these can be brought about
Do at action planning stage, and build results into plan
Theory of Change – see next slide
Advocacy Change
Advocacy changes are difficult to measure
They also occur at multiple levels – See the next slide!
Changing the world for animals!
Animal Welfare
Policy/Policy-makers
Advocacy Coalition/Alliance
Partners
Organization
Impact – scale and significance of change in animal welfare
Outcome – change in policy and/or behaviour of policy-makers
Outcome –capacity development: changes in capacity, policy or practices of the coalition/alliance (if any)
Outcome – capacity development: changes in
capacity, policy or practices of your partner (if any)
Outcome – change in policy,
capacity and/or practices of
your organization
Ripples of Advocacy
Change
Key Questions
What is the purpose of carrying
out M&E
Who needs, uses M&E
information
Who carries out M&E?
How is M&E carried out?
When should M&E be carried
out?
Frequency?
What Sort of Data?
See the Module Notes for info on the type of data which you could collect on your advocacy plan
But decide on the data that is most relevant for measuring your objectives
Do what suits your advocacy plan!
The data needed, and the reasons for its collection, should be known and understood by all
Methods of Collecting Data
Reports (written or verbal)
Inspection/Field visits
Surveys/Questionnaires/Interviews
Participatory Meetings
Participatory Techniques (e.g. SWOT
Analysis, Matrix Ranking—see our
Advocacy Toolkit) – build teamwork and
understanding
Measuring
We need indicators to measure results
Indicators
Help to measure change by showing progress towards meeting objectives
Observable, measurable, and agreed upon as valid markers of progress towards objectives
Indicators differ from objectives in that they address specific criteria that will be used to judge the success of the project or program
Features of Good Indicators
Valid: Measures the effect it is supposed to
measure
Reliable: Gives same result if measured in the
same way
Precise: Is operationally defined so people are
clear about what they are measuring
Timely: Can be measured at an interval that is
appropriate to the level of change expected
Comparable: Can be compared across
different project approaches
Proxy Indicators
Proxy indicators can be helpful in advocacy work
because of the difficulty of attributing results (and
often intangible outcomes)
They are plausible indicators of outcomes or progress
towards an outcome
In particular, they can be a legitimate way to measure
an individual organisation’s roles in coalition work.
There is more about Proxy Indicators in the module notes
Means: End:
Inputs
Processes
Outputs
• Inputs and Process are only a means to an end
• We should monitor inputs, processes and outputs
• Outputs have greatest importance
• Monitoring should also be used to help us understand:
relationships between inputs and outputs: e.g.
When an input changes, what effect does this
have on outputs?Which is the most important input? Etc.
After Evaluation
Celebrate successes
Replicate successful formulas
Don’t apportion blame
Communicate and learn through failures
Continuous Improvement
With the pace of change ever quickening, the need to develop mechanisms for continuous learning and innovation is greater than ever
M&E is an important part of your organizational learning and continuous improvement
Organizational learning will create more knowledgeable staff and a flexible organization where people will accept and adapt to new ideas and changes through a shared vision
The Learning Cycle
Carrying out an
advocacy
campaign
Evaluating or
reviewing the
advocacy campaign
Making conclusions
about changes or new
actions needed
Planning the next
step
Towards Perfection!
“To improve is to change: to be perfect is to change often”
- Winston Churchill