1 government-wide monitoring and evaluation system (gwm&e) presentation to the public service...

1

Government-wide monitoring and evaluation system (GWM&E)

Presentation to the Public Service and Administration Portfolio Committee

18 October 06

2

Presentation outline

Aim of GWM&E GWM&E Results Implementation phases Management structures Achievements

3

Government-wide monitoring and evaluation system (GWM&E)

The aim of the GWM&E system is to contribute to improved governance, promote learning and enhance the effectiveness of public sector organisations and institutions and accountability reporting

The system objectives are the collection and collation, analysis and dissemination and the application of information on the progress and impact of programmes in order to:

Ensure transparency and accountability Promote service delivery improvement Ensure compliance with statutory and other requirements and Promote the emergence of a learning culture in the public sector

4

Results to be achieved

Result one: Accurate and reliable information on progress in the implementation of government and other public sector programmes are collected and updated on an ongoing basis

Result two: Information on the outputs, outcomes and impact achieved by government and other public bodies are periodically collected and presented

Result three: The quality of monitoring and evaluation practices in government and public bodies is continuously improved

5

GWM&ES Proposal and Schedule & Implementation Plan

Phase One: Phase One: Setting the basis for implementing the reporting practice Setting the basis for implementing the reporting practice 1. Development of Principles & Practices2. Compendium of Development Indicators for SA3. OPSC assessment of departmental M&E Systems4. Capacity Building and AccreditationPhase Two: Phase Two: System Functionality, Alignment and Standardisation.System Functionality, Alignment and Standardisation. 1. Drafting, consultation & finalization of data gathering templates and reporting

formats2. Develop ICT platform & architecture to receive, collate, and report on data collected.3. Ensuring the interoperability and integration between this ICT platform (GWM&ES)

and existing departmental ICT systems.

6


Phase Three: Phase Three: System Functionality, Alignment and System Functionality, Alignment and Standardisation.Standardisation.

Initiate a pilot with a few departments to test system functionality.

Develop a detailed project plan to serve as a template for later roll-out.

Independent evaluation of the pilot system roll-out using the evaluator's findings and recommendations to make system adjustments.

Gradual extension of the system to all government bodies, starting with national departments, then provincial departments, local authorities, and all SOEs.

7


Phase One: Phase One: Setting the basis for implementing the reporting practice Setting the basis for implementing the reporting practice 1. Development of Principles & Practices

What are principles and practices?

Government is adopting an implementation approach based on shared principles and common practicesThis is a different strategy to one based on rules and prescriptions (old-style public administration)Basing our emergent system on principles provides a clear framework while it encourages adaptive thinking, consultation and communication

8

Purpose of M&E Principles Purpose of M&E Principles & Practices& Practices

The document sets out the national M&E principles and practices, including minimum norms and standards for monitoring and evaluating government policies, projects and programmes.

It also focuses on procedures to be adopted by national, provincial, local and public entities in the monitoring and evaluation of implementation of government policies and programmes/projects

9

Core principles

Principles are the unchanging, non-negotiable values that underlie the system

Although proposed, the GWM&ES’s core principles have not yet been determined

The system will have different kinds of principles at its heart:

10

Types of principles

The GWM&ES must be built on: Ethical principles: Rights-based; transparent and accountable Managerial principles: A balance of efficiency, economy,

effectiveness; service delivery oriented Technical principles: Indicator based, participatory, people-

centered, developmental, inclusive etc. Proprietary Principles: are intended to ensure that an evaluation will be

conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results

Feasibility Principles: The feasibility principles are intended to ensure that an evaluation will be realistic, prudent, diplomatic and frugal; The evaluation procedures should be practical, to keep disruption to a minimum while needed information is obtained; Be cost effective

11


Phase One: Phase One: Setting the basis for implementing the reporting practice Setting the basis for implementing the reporting practice 1. Development of Principles & Standards

Milestones AchievedMilestones Achieved

Initial Draft completed: May 2006Initial Draft completed: May 2006

Workstream established : June 2006Workstream established : June 2006

TORs for the work stream approved by M&E Task Team: July 2006TORs for the work stream approved by M&E Task Team: July 2006

Conference to deliberate on the matter held: 19-23 June 2005Conference to deliberate on the matter held: 19-23 June 2005

Comments/Input from conference incorporated: July 2006Comments/Input from conference incorporated: July 2006

Consultations with Relevant Stakeholders completed : August 2006Consultations with Relevant Stakeholders completed : August 2006

Ongoing progress on the activity reported to Cluster & Cabinet via Ongoing progress on the activity reported to Cluster & Cabinet via POA: Bi-monthlyPOA: Bi-monthly

12


Phase One: Phase One: Setting the basis for implementing the reporting practice Setting the basis for implementing the reporting practice 1. Development of Principles & Practices (including Norms & Standards)


Draft Guidelines for M&E being discussed at Work Stream level

Key Glossary of M&E Terms completed August 2006

13


Phase One: Phase One: Setting the basis for implementing the reporting practice Setting the basis for implementing the reporting practice 2. Compendium of development Indicators for SA

Milestones AchievedMilestones AchievedBased on the Ten Year Review, appropriate set of the indicators for five clusters have been defined, which comprise of 163 indicators and a total of 335 subset indicators that would be used to measure the impact of government performance in the long term. The matrix constitutes eight significant variables namely: Index (Category); Index Description; Indicator; Subset Indicators; frequency of data collection; data source; data Sets and Level of disaggregationDeliberated upon at a workshop in November 2005 & Conference in June 2006Indicator refinement & definitions by PCAS/NSSD: Ongoing workDiscussed at Cluster Level: January/March 2006Draft baseline document completed & circulated for comments: May & Sept 2006Baseline data collection: completed Sept- 2006 (for quantitative indicators)

14


Phase One: Phase One: Setting the basis for implementing the reporting practice Setting the basis for implementing the reporting practice 2. Compendium of development Indicators for SA (Cont)

Milestones AchievedMilestones AchievedData has been collected and has been captured in several computer packages (MS Access, MS Excel, MS Word and SPSS) Integrated all the available databases into one usable database - DevInfo in order to achieve a:

a common understanding of indicators programmeAnalysis, assessment and evaluation of the validity and reliability of SA information Ability to collate, disseminate and present both government, millennium development goals and other international commitmentsThe software is available to SADC countries at no cost

15


Phase One: Setting the basis for implementing the reporting practice Phase One: Setting the basis for implementing the reporting practice 3. OPSC assessment of departmental M&E Systems

DescriptionDescription

The OPSC was requested to undertake an audit of departmental M&E systems and reporting requirements. The purpose of this report is to provide an assessment of what exists in terms of M&E at a departmental level and an understanding of how departments undertake their reporting requirements.

The report is currently being finalised and will be presented to various stakeholders soon.

In the context of the above, it should be noted that the PSC is implementing its governance monitoring system across departments. To-date over 35 such reports have been completed. .

16


Phase One: Phase One: Setting the basis for implementing the reporting practice Setting the basis for implementing the reporting practice 4. Capacity Building and Accreditation

ApproachApproach

One core training manual will be developed (based on the Oxfam model) in the GWM&ES, the training will be based on the system in particular, not just on M&E in general.

Adopting this approach will ensure that many self proclaimed M&E experts will not be allowed to market their training as “GWM&ES training”, this will be done for government, by government agencies.

Different facilitation guides will ensure that the level and complexity target beneficiaries will be addressed, in other words, if an M&E manager is being trained, the complexity and depth of the training will be commensurate with the M&E responsibilities that she has to fulfill. The same applies to frontline staff who have very basic M&E responsibilities, for instance, recording information on client contact or services rendered.

17

The Proposed Capacity Building Strategy

Assessment of M&E capacity needs at provincial level Does the province have an M&E Strategy that is clearly articulated?

Has the province conducted departmental M&E Audits?

Does the province have an M&E Strategy that is clearly articulated? Does the province have an M&E Learning Network?

What provincial M&E capacity building initiatives have taken place thus far?

Is the provincial M&E Strategy clearly aligned with principles and standards of the GWM&ES?

Does the provincial M&E Strategy accommodate cluster reporting

Does the provincial M&E Strategy provide useful information on the Service Delivery Objectives of the Office of the Premiere?

Has the province launched its IDP nerve centre?

18

Conceptual learning framework - capacity building consist of three dimensions

Training and development Competency-based training - Knowledge, skills and values Additional seminars/ workshops more detail eg Advanced Statistics Accredited training to be recognised at tertiary institutions further studies Morning sessions for Chief directors and above – advocacy for GWM&E

Organisation development Consultation and support with development of systems, data and reporting Change management

Community of practice Principles and practices, learning networks to support. SAMEA to support the M&E Forum for Government officials and recognition

for officials declared competent on training

19

Development of training material

Focus group engagement to determine learning outcomes / topical issues and methodology as design framework for training material

Consultation workshop with Premiers’ offices and key stakeholders on the Curriculum framework

Approval from M&E task team and SAMDI Curriculum Review Committee

Submit funding proposal for development of training material (and implementation in key departments)

One training manual – differentiate through facilitators guidelines and target groups identified (Application of learning; exercises; action plans)

Appointment of persons (internal or external) to develop training material and obtain approval from Curriculum Committee.

Quality Assurance and Research on capacity development to enhance training and development adding value to relevance and implementation.

20


Phase Two: Phase Two: System Functionality, Alignment and Standardisation.System Functionality, Alignment and Standardisation. 1. Drafting, consultation & finalization of reporting formats


The joint team of NT and DPSA have commenced with the development of the The joint team of NT and DPSA have commenced with the development of the requirements of a system that would ensure the monitoring of the departmental strategic requirements of a system that would ensure the monitoring of the departmental strategic plans – The requirements will be finalised by the end of October 2006plans – The requirements will be finalised by the end of October 2006

The work stream has commenced with the development of the framework of an end of The work stream has commenced with the development of the framework of an end of year report that would complement the current report with information currently within the year report that would complement the current report with information currently within the transversal systems – The framework will be finalised at the end of November 2006transversal systems – The framework will be finalised at the end of November 2006

21

Annual State of the Nation Report(End of year report)

This report will inform/support the January Lekgotla and the SONA

Content: Cluster annual progress reports based on POA Combination of transversal systems information Aggregation of departmental strategic plans, annual reports and

departmental individual performance plans

The reporting format, indicators, collection & reporting system are still being developed

22

GWM&E Database

Adm

in &

M

anag

Reporting Formats 2009

Annual Reports& Quarterly Rep.

Strategic Plan

DPLG

National Treasury

OPSC

Public Man Watch

HR Utilisation

Rep

ortin

g

Req

uire

men

ts

POA

Ou

tpu

ts

Fin

an

cia

l

Hu

man

R

esou

rce

Bath

o

Pele

Cluster Performance

Sector Performance

Dept/Prov Performance

Outco

me/

Impact

23

Current Transversal M&E Systems

System Lead Department Comment Public Service M & E System

Office of the Public Service Commission

Fully operational

Cluster matrix of indicators Presidency Based on outcome indicators developed by the different clusters

Programme of Action Presidency Fully operational based on report cards

Non Financial Reporting National Treasury Implemented but data quality still lacking

In Year Monitoring National Treasury Fully operational Public Management Watch Dept of Public Service and

Administration Implemented w.e.f Dec 06

Annual HR Report Dept of Public Service and Administration

To be published in the second quarter of 2006/07

Sectorial specific systems Health, Education, Social Development

Various degrees of implementation

Provincial based systems Office of Premiers Various degrees of implementation

24

Public Service M&E System – (OPSC)

SA Constitution lists 9 basic values and principles that govern the public service

The PS M&E System looks at the extent to which public service departments comply with these principles

The system aims to Gather info that can be used to identify areas needing attention,

ultimately contributing to improved public service performance

This information is published on an annual basis in the State of the Public Service Report

25

Cluster progress – Presidency

This system is based on the matrix of indicators which is a set of indicators for each of the clusters

The aim of the system is to track the progress of the different clusters in reaching their outcomes

The system indicates Those areas where progress is slow and intervention is needed Those areas of good progress as examples of best practices

This system is supported by the NSS and reports are updated on a quarterly basis

26

Programme of Action (Presidency)

The system is based on the Government Programme of Action that flows from the President’s State of the Nation Address that provides the strategic direction for the Public Service

The aim of the system is to provide an indication for progress in implementing the priorities of government.


The system is electronic database Excel templates and are updated every quarter with a two-month lag per quarter as due date for reporting

Feedback on progress is given on a quarterly basis by means of consolidated reports on the intranet of Treasury once the quality of the data is credible it will be available on the internet

27

In Year Monitoring (National Treasury)

Monthly monitoring reports: Focus on performance against budget Alert managers where remedial action is required

Reports consolidated and published: National Departments – monthly publication in the national

Government Gazette Provinces – Monthly reporting and monitoring BUT quarterly

publication in the national Government Gazette Section 32 Reports

28

Non Financial Reporting (National Treasury)

This system is based on provincial departments’ reporting against targets set in their Annual Performance / Strategic Plans

The provincial indicators are set per sector to enable cross- provincial comparison


This system together with the In Year Monitoring System would facilitate cost benefit analysis

The system is updated on a quarterly basis The system has been rolled out but there is still a problem with the

quality of the data provided by departments

29

Public Management Watch (DPSA)

Early warning type system with regard to management within departments Consists of the following processes:

Statistical rating based on payroll and financial information Communication to and self assessment by identified departments On site assessment of selected department Intervention into the management of departments

The system is functional and will be applied every quarter The system -

Indicates those departments/areas in need of intervention Indicates those departments that have proven to be examples of best practices Facilitate the knowledge exchange between departments

This system aims to direct the interventions of the centre of government to those departments that require assistance

30

Annual HR Report (DPSA)

Annual analysis of the status of HR in the Public Service The following would serve as information sources for this report

Departmental annual reports Public Management Watch findings Payroll and Financial Systems Reports – OPSC, AG, DPSA, etc.

The system - Indicates trends within the HR environment Identifies those bottlenecks effecting the efficient and effective utilisation

of HR Identifies those remedial steps taken to address the bottlenecks

between departments This system aims to guide the centre of government and HR

managers on the areas in need of attention

31


Phase Two: Phase Two: System Functionality, Alignment and Standardisation.System Functionality, Alignment and Standardisation. 2. Develop ICT platform & architecture to receive, collate, and report on data collected


The development of the IT system is dependent upon the finalisation of the user The development of the IT system is dependent upon the finalisation of the user requirements (data and reporting requirements)requirements (data and reporting requirements)

32

GWM&E – Proposed reporting outputs

Bi-monthly reports against the government Programme of Action (POA)

Mid-year progress report Annual State of the Nation Report (End of year report) Term of office PS Annual report 2014 – Review Special sector M&E reports – eg Education and Health

33

Management structure of the Institutional Framework for Implementing the GWM&ES

Project Steering Committee

Governance & Administration

Cluster

The Presidency

M&E Task Team

Principles and Standards

Work Stream guided by a TOR

Reporting and Databases Work

Stream guided by a TOR

Capacity Building Work Stream guided

by a TOR

Intellectual Leadership & Strategic Direction

Project Management, Systemic Gearing and Capacity Building

Project Implementation Tools, Standards, Templates, Platforms, Training Interventions

34

Institutions Responsible for Implementing Work Streams

Principles and Practices

Reporting and Databases

Capacity Building

Lead Agency DPSAPresidency SAMDI

Partner Agencies OPSC

Stats SA

National Treasury

Presidency

SITA

Stats SA

DPLG

National Treasury

DPSA

Presidency

Stats SA

DPLGDPSA

35

Overall Achievements to date

A public commitment to creating the system is an important first step and is shown in Cabinet’s adoption of the Strategy and Implementation Plan

Political support (key to public service reform at this level) has been consistently provided

Consistent, sustained membership and involvement of core task team members

Recent agreements on appointment of task teams and identification of members to facilitate reporting on the programme of action 2006

POA system is fully functional and promotes transparency and accountability

Indicator development Transversal systems have been developed and are operational

1 government-wide monitoring and evaluation system (gwm&e) presentation to the public service...

Documents

gwme system

system functionality

implementation of government

emergent system

system objectives

system adjustments

public sector slide

public bodies