mr oliver seale m&e learning network tuesday, 15 may 2007

Building Capacity for effective government wide Monitoring and Evaluation

Mr Oliver SealeM&E Learning Network Tuesday, 15 May 2007

Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development

2

Capacity building for Monitoring & Evaluation BackgroundAims and objectives

The aim of the system is to contribute to improved governance and enhanced effectiveness of public sector institutions.

The system aims to collect, collate, analyse and disseminate information on the progress and impact of programmes.

Result Areas Accurate information on progress in the implementation of public

sector programmes is updated on an ongoing basis; Information on the outcomes and impact achieved by

government is periodically collected and presented; The quality of monitoring and evaluation (M&E) practices in

government and public bodies is continuously improved.


3

Capacity building for Monitoring & Evaluation Provincial needs analysis: example of feedback (A)

1. Little coherent or articulated strategy in provinces, though expenditure on expensive systems to collate M&E data.i. What would a coherent strategy need to contain? ii. What is an articulated strategy? What type of links

are we looking for? iii. What systems are there? What do we mean by a

system?iv. What data are there? How were they obtained? What

is the quality of this data? v. What collation is taking place? How? Can a system

collate data?

What are the

implications for training?


4

Capacity building for Monitoring & Evaluation Provincial needs analysis: example of feedback (B)

2. Monitoring programmes are just about collecting data, very little analysis and feedback given.

i. What data, why and how are they being collected?

3. Alignment of plans doesn’t existi. What planning does take place and how? ii. How is M&E incorporated into planning?iii. What do we mean by alignment why do we

need it? iv. Is alignment always possible and necessary?

What are the



5

Capacity building for Monitoring & Evaluation Provincial needs analysis: example of feedback (C)

4. Planning without indicatorsi. What type of indicators do we mean and how

should these be developed?ii. What indicators do exist and how are they

measured?iii. How are they decided on?

5. Lack of or poor baseline datai. What baseline data do exist and do we evaluate

it?ii. What type of baseline data are required?iii. How are these presently obtained?

What are the



6

Capacity building for Monitoring & Evaluation Conceptual framework for training

1.1. DescriptionDescription2.2. Existing data basesExisting data bases3.3. Data collection methodsData collection methods4.4. Baseline data Baseline data

1.1. What will be done (strategy)What will be done (strategy)2.2. Why will it be done (policy)Why will it be done (policy)3.3. How will it be done (operations)How will it be done (operations)4.4. Indicators and criteria (how to measure)Indicators and criteria (how to measure)5.5. When (timeframes)When (timeframes)

MonitoringMonitoring

Existing situationExisting situation New project or New project or programmeprogramme

PlanningPlanning

1.1. System to be used (MIS)System to be used (MIS)2.2. Indicators Indicators 3.3. MethodsMethods4.4. Baseline dataBaseline data5.5. Inputs Inputs 6.6. TrackingTracking

i.i. ProcessesProcessesii.ii. ActivitiesActivities

7.7. Interventions and modificationsInterventions and modifications8.8. OutputsOutputs9.9. OutcomesOutcomes

EvaluationEvaluation

1.1. System to be used (EIS)System to be used (EIS)2.2. IndicatorsIndicators3.3. MethodsMethods4.4. Baseline dataBaseline data5.5. CriteriaCriteria6.6. AssessmentAssessment7.7. ProcessProcess8.8. ImpactImpact9.9. Lessons learnedLessons learned10.10. FeedbackFeedback


7

Capacity building for Monitoring & Evaluation Training principles♦ Integral part of planning, not an add-on

Process requires careful design in relation to priorities Process requires participation of all involved, including beneficiaries

(e.g. user-satisfaction surveys, participatory research methods)♦ Emphasis on quantification

Measurement of goals and objectives (efficacy) Analysis of methods used (efficiency)

♦ Emphasis on understanding and using existing data-bases and collecting data where necessary

Indicators should be identified from the outset♦ Documentation essential

Teach reporting and writing skills♦ M&E training incorporate planning, goal setting, problem

identification, and problem solving


8

Capacity building for Monitoring & Evaluation Training principles for various levels

Principles

1. Understanding of basic principles of M&E

2. Applying principles to a specific project

3. Applying principles to a programme

4. Applying principles to overall management in departments

5. Applying principles across departments/provinces

6. Actually performing evaluations

Levels

Basic for general users of information Basic for project managers Intermediate for programme managers Advanced for executive managers Advanced for CFOs and DDGs Specialist technical training for M&E staff


9

Capacity building for Monitoring & Evaluation Target audiences1. Users

i. Political heads and parliamentarians (incorporated into report-backs to portfolio committees)

ii. Accounting officers (DGs)iii. Executive managers and managers in govt departmentsiv. Users of the service or the information outside government

2. Producersi. Programme managersii. Project managersiii. Operations staffiv. Participants

3. M&E staff in national and provincial departments


10

Capacity building for Monitoring & Evaluation

Examples of current provisionInstitution Programme Duration Level Content

University of Stellenbosch

Diploma in Monitoring and Evaluation Methods

1 year Post Graduate

♦ General principles & paradigms ♦ Clarificatory evaluation♦ Process evaluation & programme monitoring♦ Data collection methods♦ Statistical and qualitative methods♦ Impact assessment designs

University of Cape Town

Workshop on Monitoring and Evaluation

1 week All ♦ General principles♦ Measuring Public projects

Regenesys Short course in Monitoring and Evaluation

3 days NQF 4 ♦ Monitoring and evaluation concepts♦ Results-based management♦ Concepts of outcomes♦ Project objectives and indicators♦ Monitoring and evaluation system♦ Team performance improvements♦ Performance standards ♦ Risks and the impact thereof♦ Success and failure factors♦ Project evaluation reports


11

Capacity building for Monitoring & Evaluation Strategy and Plan of Action♦ Progress Report

Terms of reference developed for Task Team. 15 workshops on M&E for programme and project management

(340 officials). Initial needs analysis on provincial M&E capacity. Consultation with key internal and external stakeholders. Focus group sessions and sector-wide survey

♦ Plan of Action Determine current providers of M&E - HEIs, privates, NGOs etc.

(April ‘07). Undertake training needs analysis for M&E (May-June ‘07). Develop M&E training programme (Sept ‘07) Roll-out M&E training programme (Nov. ‘07).


12

Project Team

Oliver Seale: Tel. 0824257386 email. [email protected]

Ros Hirschowitz: Tel. 0824546209 email. [email protected]

Marisa Labuschagne: Tel. 0832611544 email. [email protected]

Siyabonga, Thank you, Rolivhuwa, Dankie, Nakhensa, Re a leboga

mailto:[email protected]



mr oliver seale m&e learning network tuesday, 15 may 2007

Documents

evaluation capacity

building capacity

monitoring programmes

type of baseline data

trainingcapacity building

existing databases

skillsme training

example of feedback