monitor and evaluation-steve giddings

8/9/2019 Monitor and Evaluation-Steve Giddings

1/21


2/21

Overview

1. What is monitoring and

evaluation (M&E)?

2. Some common M&E methods

3. Considerations for incubator

M&E


3/21

What is M&E?Monitoring: a continuing function that uses systematiccollection of data on specified indicators to provide

management with indications of the extent of progress and

achievement of objectives and progress in the use of

allocated funds

Evaluation: the process of determining the worth or

significance of a program to determine the relevance of

objectives, the efficacy of design and implementation, the

efficiency or resource use, and the sustainability of results.

M&E are synergistic monitoring is a necessary, but

not sufficient, input into evaluation


4/21

M&E

Tools, methods and approaches for M&E

(from Monitoring and evaluation: some tools, methodsand approaches. The World Bank, 2004)


5/21

Performance indicators

Measures of inputs, processes, outputs, outcomes,

and impacts

When supported with sound data, indicators canassist track progress, demonstrate results, and take

corrective action to improve service delivery

Important to include key stakeholders in defining

indicators - they are more likely to understand andthen use indicators for management decision-making


6/21

Performance indicators

USED FOR:

Establishing performance targets and then evaluatingprogress

Indicating whether an in-depth evaluation or review is needed

ADVANTAGES: Effective means to measure progress toward objectives

DISADVANTAGES:

Poorly defined indicators are not good measures of success

Tendency to set too many indicators, or those without

accessible data sources - costly, impractical and thenunderutilized

Often a trade-off between selecting the best indicators andaccepting those which can be measured using existing data


7/21

The Logical Framework

Helps to clarify objectives of any project,program, or policy

Aids in the identification of the expectedcausal inputs, processes, outputs outcomes,

and impact Leads to the identification of performanceindicators as well as risks which might impedethe attainment of the objectives

Useful for engaging partners in clarifying

objectives and designing activities Useful tool to review progress and takecorrective action


8/21

The Logical Framework

USED FOR:

Improving quality of programme designs by requiring thespecification of clear objectives, performance indicators & riskassessment

The preparation of detailed operational plans

Objective basis for activity review, monitoring, and evaluation

ADVANTAGES:

Sets out key questions and analyses of assumptions and risks.

Engages stakeholders in the planning and monitoring process

A management tool to guide implementation, monitoring and

evaluation

DISADVANTAGES:

Can restrict creativity

Tendency to be static

Training and follow-up often needed


9/21

Theory-based evaluation

Similar to Log Frame but more detailed in

understanding programme logic

Seeks to identify causal or determining factors

seen as important for success and then what

should be monitored

Ultimately leads to the determination of critical

success factors (CSFs)

Evaluation of CSFs used to inform likelihoodof programme success


10/21

Theory-based evaluationUSED FOR: Mapping the design of complex activities

Improving planning and management

ADVANTAGES:

Early feedback about what is or is not working, and why Allows early correction of problems as soon as they emerge

Assists identification of unintended side-effects of the program

Helps in prioritizing which issues to investigate in greater depth

Provides basis to assess the likely impacts of programs

DISADVANTAGES: Can easily become overly complex if the scale of activities islarge or if an exhaustive list of factors and assumptions isassembled

Stakeholders might disagree about which determining factorsthey judge important, which can be time-consuming to address


11/21

Formal surveys

Formal surveys can be used to collectstandardized information from a carefully selectedsample of people or households

Surveys often collect comparable information

for a relatively large number of people in particulartarget groups


12/21

Formal surveys

USED FOR:

Providing baseline data against which the performance of the

strategy, program, or project can be compared

Comparing actual conditions with the targets

Providing key inputs into a formal evaluation of the impact of a

programme or project

ADVANTAGES:

Findings from the sample can be applied to wider target group

Quantitative estimates for the size and distribution of impacts

DISADVANTAGES:

Processing and analysis can be a bottleneck for larger surveys

Many kinds of information are difficult to obtain through formal

interviews


13/21

Rapid appraisals

Rapid appraisal methods are quick, low-

cost ways to gather the views and feedback

of beneficiaries and other stakeholders, in

order to respond to decision-makers needsfor information

Can include: key informative interview;

focus group discussion; direct observation;

mini survey etc.


14/21

Rapid appraisals

USED FOR:

Providing rapid information for management decision-making,

especially at the project or program level

Providing qualitative understanding of highly interactive social

situations, or peoples values, motivations, and reactions

Providing context and interpretation for quantitative data

collected by more formal methods

ADVANTAGES:

Can be conducted quickly and at low cost

Provides flexibility to explore new ideas

DISADVANTAGES:

Less valid, reliable, and credible than formal surveys

Difficult to generalize from findings


15/21

Participatory methods

Participatory methods provide active involvement in

decision-making for those with a stake in a project,

program, or strategy and generate a sense of

ownership in the M&E results and recommendations

Can include: stakeholder analysis; beneficiary

assessment etc


16/21

Participatory methods

USED FOR:

Identifying problems and trouble-shooting problems during

implementation

Providing knowledge and skills to empower people

ADVANTAGES:

Examines relevant issues by involving key players in design

Establishes partnerships and local ownership of projects

Enhances local learning, management capacity, and skills

Provides timely, reliable information for decision-making

DISADVANTAGES:

Sometimes regarded as less objective (domination and misuse)

Time-consuming if key stakeholders are involved in a meaningful

way


17/21

Cost benefit evaluation

Cost-benefit and cost-effectiveness analyses are

tools for assessing whether or not the costs of an

activity can be justified by the outcomes and impacts

Cost-benefit analysis measures both inputs and

outputs in monetary terms

Cost-effectiveness analysis estimates inputs in

monetary terms and outcomes in non-monetary

quantitative terms


18/21


19/21

Impact assessment

The systematic identification of the effects

positive or negative, intended or not caused

by a program or project

Impact evaluations can range from large scale

sample surveys to small-scale rapid assessment


20/21

Impact assessment

USED FOR: Measuring outcomes and impacts of an activity and distinguishing

these from the influence of other, external factors

Helping to clarify whether costs for an activity are justified

Drawing lessons for improving the design and management of

future activities

ADVANTAGES:

Provides estimates of the magnitude of outcomes and impacts

Provides answers to: what extent are we making a difference?

What are the results on the ground?How can we do better?

Rigor gives managers added confidence in decision-making

DISADVANTAGES:

Can be expensive and time-consuming which reduces utility whendecision-makers need information quickly


21/21

Incubator M&E

Many ways of doing M&Eis there any one

correct method ?

Central question for M&E is what do we want

to achieve and how do we best measure it?

Need to ensure sound M&E in place to

measure progress AND convince stakeholders

of value (especially E)

Approach must match pragmatism with need

Experience shows hybridization needed to

overcome:

Situational issues

Resources (skills, time, money)

Ability to find what can really be measured

monitor and evaluation-steve giddings

Documents