by:

Albert Byamugisha, PhD. Commissioner, Monitoring and & Evaluation

Office of the Prime Minister

Contents of Presentation

1. Introduction

2. Context

3. Policy Framework for Monitoring & Evaluation

4. How we plan & evaluate in Uganda’s public sector

5. Governments role in coordinating and /or using evaluations

6. Institutional Framework in promoting use of credible Evaluations in Government

7. Institutional Mechanism Arrangement for M&E

8. Current Initiatives to promote M&E in Uganda

9. Planned evaluations (2012-2014)

10. Challenges

11. Conclusion and Way forward

2

Paper will:

Highlight how Uganda is institutionalizing evaluation by linking supply and demand: Focusing on roles and responsibilities in government (especially executive)- Mention will also be made on how the executive associates with the parliament and civil society,

Highlight existing policy framework for evaluation and to what extent it clarifies roles and responsibilities on evaluation for different institutions,

Highlight role played by government in initiating, coordinating and/or using evaluations

Highlight current initiatives in promoting the use of credible evaluations

1.0 Introduction

3

In Uganda, over the past two decades, considerable efforts have been made to establish a strong and robust basis for assessing public spending, and its effects on development

The Office of the Prime Minister (OPM) has a constitutional mandate to oversee reforms and service delivery in all Government Ministries, Departments & Agencies; and therefore has established an M&E function to support this role.

One of its key functions is to co-ordinate, monitor and evaluate the implementation of Government Policies and Programmes.

A National Strategy for Monitoring and Evaluation of Government programmes (NIMES) also exists within OPM, which inter alia, enhances M&E capacity in Uganda as well as ensuring that sound evidence based Data & Information are available to inform decision making in national policy frameworks.

2.0 Context

4

Government introduced a series of reforms to enable Ministries, Departments and Agencies (MDAs) and Local Governments (LGs) to plan and budget annually according to clear budget lines, and against the provision of products and services. MDAs report quarterly on spending and progress towards stated output targets

The political interest and pressure to monitor spending and results has increased since the re-introduction of multi-party politics in Uganda in 2006 and growing attention of domestic media, CSOs and international community

Executive and parliament have taken cognizance of these issues, and have placed increasing demands on the public service to improve its stewardship of resources and ensure effective development

3.0 Context……………………………………………………..contn’

5

The National M&E Policy was passed by Cabinet in March 2013.

• Primary purpose:

To ‘improve the performance of the public sector’ through the strengthening of the operational, coordinated and cost-effective production and use of objective information on implementation & results of national strategies, policies, programmes and projects.

• Specific objectives:

Embed M&E in the management practices of MDAs and LGs;

Expand the coverage of public policy and programmes that are subjected to rigorous evaluation;

Clarify M&E roles and responsibilities;

Strengthen coordination in the supply and demand of M&E;

Strengthen capacities of MDAs & LGs to implement the policy.

3.0 Policy Framework for Monitoring & Evaluation

6

The current planning framework (BFP, MPS) does not include impact measures, but we do evaluate to assess the impact of our interventions.

Impacts

Performance Accountability Framework

Development results

Sector Outcomes Each sector has up to three outcomes in its BFP, each with key indicators that measure the overall changes in the use, behaviour or satisfaction of the public services provided as measured in the general population.

Vote level (intermediate)

outcomes

Vote Function (VF) Outputs are the summary statements of the goods and services provided under a particular vote function (often a department).

VF Outputs

Each vote is required to define up to two intermediate outcomes in their BPF and MPS, with key indicators.

Inputs (particularly

spending)

Financial inputs are linked to VF outputs under the budget. These include the wage, non-wage recurrent and development costs. The costs associated with delivering particular outputs needs to be refined, to assess value-for-money.

4.0 How we plan & evaluate in Uganda’s public sector

7

To respond to its mandate, OPM established bi-annual Government Performance Assessments.

Reports arising from the assessment are discussed at Cabinet Retreats to track performance and spending and take corrective measures.

Process generates demand pressures on Ministries to improve performance and monitoring & evaluation practices.

Government performance reports use scorecard with “Traffic Light” system to assess all MDAs on performance indicators against set annual targets and spending against budgets

Green – target has been

Yellow – the level of progress against target/action is slightly below borderline

Red – the target has not been met

Grey – insufficient or no data and therefore assessment is not possible

Proceedings from the Retreats include:

The specific actions agreed upon to address issues raised

Progress made against these actions is tracked and reported on at the subsequent retreat. UNDP signed an MoU with OPM to strengthen this initiative

5.0 Governments Role in Coordinating and /or using evaluations

8

Baraza initiative: Seeks to strengthen citizen’s engagement with the state

Enable them to oversee Government spending at Local Government level

It is a critical participation evaluation activity and Presidential directive in Uganda.

It is a “town-hall” style meeting: Government representatives present on their activities during the previous year

Present spending and then the public respond with questions, queries and analysis of their own.

Barazas help the civil and political leaders: To explain to the community what Local Government does

The amount of funds received from central government how they are spent.

Through Barazas Government leaders and implementers get to know issues that affect their citizens and make an input-

6.0 Governments Role in Coordinating and /or using evaluations..cont’d

9

Government Evaluation Facility (GEF)

Established by OPM to address the poor coverage and quality of public policies and investments by evaluation in Uganda.

Provides a systematic basis for expanding the supply of rigorous assessments to address public policy & investment questions surrounding effectiveness of Gov’t interventions.

Helps in tackling underlying constraints to improved service delivery.

Elements of GEF: Has a two year rolling agenda of evaluation topics, approved by Cabinet [2012-2014].

Has a Virtual fund to finance evaluations [Initially with support from the development partners and government of Uganda].

Has Standards, process guidelines and database for guiding & communicating findings Seeks to address previously identified problems of ownership capacity and utility by locating the facility in Government (OPM)

Its being used to build analytical and evaluative capacity amongst its members.

Overseen by sub-committee made up of experts from public sector institutions [OPM, Ministry of Finance, National Planning Authority, Uganda Bureau of Statistics], academia, Non-Government Organization and Donor Community

6.0 Governments Role in Coordinating and /or using evaluations….cont’d

10

Rigorous Public Sector Evaluations have been identified as a constraint to improving the culture of debating empirical evidence in public policy, hence OPM began an initiative to strengthen the framework and production of rigorous evaluations across the public service.

In 2008, OPM led on the design, implementation and dissemination of evaluations of the successes and failures of the PEAP over the decade of implementation ( 1997-2007).

PEAP evaluation process was managed by a steering committee composed of OPM, MoFPED, NPA and DPs (this was a good example of inter-ministry coordination. Because it was supply driven the evaluation found an audience amongst policy makers with findings and recommendations discussed by cabinet.

Results of the evaluation influenced the shape of the PEAP successor, the five-year National Development Plan and development of a national policy on M&E.

6.0 Institutional Framework in promoting use of credible Evaluations in Government

11

Sector Working Groups and their benefits:

i. National Monitoring & Evaluation (NM&E) • Improved identification of cross-cutting M&E performance constraints for

the attention of MDAs. • Enhanced technical ability of the working group to act as a clearing house

for M&E policies , programs and project. • Ministries, Departments and Agencies increasingly share experiences in the

use of the sector-wide approach as monitoring and evaluation tool.

ii. National Monitoring and Oversight (NM&O) • Minimized duplication of roles /activities brought about by conflicting

mandates e.g. OPM utilizes data from UBOS, BMAU-MoFPED & MLG in preparation of GAPR, thus supporting individual institutions to develop full capacity so as to improve quality of their products.

iii. Evaluation Sub-Committee(ESC)

• Increased use of reference groups ( international and local) to enhance quality of evaluations;

• Enhanced focus on comparative strength and need for independence from the evaluand in the selection of the evaluation lead institution.

7.0 Institutional Mechanism Arrangement for M&E

12

1. Strengthening Monitoring Capacities in the Ugandan Public Sector

This is a 2 – year project supported by UNDP aiming at contributing to increased effectiveness, efficiency and accountability in Uganda’s public sector by strengthening results-oriented monitoring capacities across Government.

Training of top civil servants in Monitoring Government program-practical approach

Operationalisation of National M&E Policy

2. Evaluation Capacity Development (ECD) Project

Three year Project funded by the Republic of Germany Government through GIZ. The project focuses on the following four components:

Information events, (populalising evaluation) Gender-sensitive and development evaluation standards, Continuous training courses in short and medium on evaluation, and, Establishment of a blended learning Master’s course in evaluation at

Makerere University

8.0 Current Initiatives to promote M&E in Uganda

13

3. Supported Uganda Evaluation Association in developing of Evaluation standards which have enhanced the selection and practice of evaluation to ensure that evaluations serve the public interest.

4. Short term courses in Evaluation which has created a critical mass of trainers on evaluations in Uganda. A Training of Trainers (17 in total) was carried out and these have supported over 10 trainings for staff of MDAs, CSOs & Parliament.

5. Prime Minister’s Integrated Management Information System (PMIS)

The project aims at enabling interfacing between Management Information Systems across government institutions.

UNDP, with technical assistance from National Information Technology Authority (NITA-U), is supporting the establishment of the Prime Minister Information Management System; which will link all the data/information systems from all sectors of government.

6. Government Evaluation Database Acts as a single storage facility for retrieval for all government evaluations that

have been undertaken for cross-referencing. It also enables trends to be established by comparing various evaluation studies

done in the country

9.0 Current Initiatives to promote M&E in Uganda …Cont’d

14

7. Strengthening of Partnerships with Parliament

Capacity Building in Monitoring and evaluation

Formation of Parliamentary Forum on Development Evaluation

8. Prime Minister’s Integrated Management Information System (PMIS)

This project aims at enabling interfacing with all relevant and expected government information systems to facilitate the Office of the Prime Minister in the fulfillment of its mandate.

9. Holding National Evaluation week every year (Next year May )

9.0 Current Initiatives to promote M&E in Uganda …Cont’d

15

9.0 Planned evaluation (2012-2014)

16

No Evaluation Topic Specific questions 1 Effectiveness of

Government’s response to absenteeism in the public service [completed]

How appropriate have the policy and programmatic responses to absenteeism across the public service been?

How coherent have the policy and programmatic responses to absenteeism across the public service been?

How effective, and where possible, impactful, has been the Government’s response to absenteeism in the public service?

How sustainable are the main initiatives being undertaken by Government to address absenteeism in the public service?

What lessons can be taken from the Government’s response to absenteeism, and what can we learn from other countries?

2 Evaluation of Government’s Employment Strategy [on-going]

How the policies/strategies have effectively addressed the unemployment problem? How effectively have individual Government interventions addressed to problem of unemployment?

What lessons can be drawn to help decision makers improve future interventions?

3 Effectiveness of the Land Act and Registration of the Titles Act in curbing illegal land evictions. [procurement process]

How appropriate and effective have the land Act provisions on Landlord tenant relationship been? Whether or not the causes of illegal evictions are due to the overlapping or multiple rights on the same piece of land, and The extent to which the Land Act has adequately or inadequately addressed illegal evictions.

Is there coherence: Within the different land amendments; With other laws (for example the Succession Act); and Across the various responses to address illegal evictions and land mismanagement

What outcomes and impact have resulted from the implementation of the land amendment Acts?

How effective have Government interventions to prevent illegal evictions been including units set up in Police to handle eviction related matters?

10.0 Planned evaluations (2012-2014)…………………….Cont’d

17

No Evaluation Topic Specific questions 4 Evaluation of the public

procurement and disposal authority’s role in ensuring effectiveness and efficiency of public procurement

[completed]

To what extent has PPDA ensured the application of fair, competitive, transparent, non-discriminatory and value for money procurement and disposal standards set by PPDA and practised by the identified PDEs? To what extent has PPDA monitored compliance of procuring and disposing entities, including administrative reviews? Were all functions in PPDA Act are implemented? If not, why? By and large what are PPDA’s notable achievements and accomplishments to date, challenges encountered, and degree of success? What are the main factors that contributed to the observed success or lack of success e.g. political economy factors, institutional factors, etc.? How do the achievements/results registered so far by PPDA compare to the 2003 baseline results? Can the observed changes over time (if any) be attributed to PPDA’s role? What other factors might have contributed to the observed changes? To what extent has PPDA succeeded in harmonizing the procurement and disposal policies, systems and practices? To what extent has PPDA set standards for the public procurement and disposal systems in Uganda? Was PPDA the best intervention for preventing leakages in public resource use? What other alternative interventions/mechanisms might have been more appropriate & why? How likely are the achievements of PPDA going to be sustained over time, especially [sustainability of] the procurement and disposal capacity built by PPDA? What factors are most likely to favour the sustainability of the achievements, and what factors are most likely to limit their sustainability including capacity building?

9.0 Planned evaluations (2012-2014)……………..Cont’d

18

No Evaluation Topic Specific questions 5 Summative Evaluation of

the effectiveness of the Avian and Human Influenza Project (AHIP) [completed]

To what extent has the project achieved its objectives? Was the coordination structure efficient (and effective)?

Did the project provide value for money?

6 i) PRDP impact evaluation (NUSAF II, Community Infrastructure Rehabilitation (CIR) component)

[on-going]

Was the change observed a result of the outputs delivered under the Community Infrastructure Component or did other factors affect the outcome; and How will future increases in the improvement in community welfare be funded after completion of the programme.

7 i) Impact Evaluation of the Barazas programme

[on-going]

What are the results (outcomes & impacts) that can be attributed to the program) Did a change in service deliver take place?

i. Capacity Gap: Improvements in M&E require strong skills and experience in quantitative and

qualitative methods

iii. Use of Monitoring and Evaluation Reports: Inadequate use of evidence to support policy-making and decision-making

iii. Inadequate use of evidence to support policy-making and decision-making

iv. Inadequate coordination & coherence among Gov’t institutions & programmes

v. Inadequate information system dedicated to improve the quality statistical production

vi. Capacity & funding to support the M&E processes

vii. Need to find other ways to communicate key messages to stakeholders like media & publication of articles in referred academic journals (Media can play a big role in this area)

10.0 Challenges

19

Government of Uganda is committed to improve its M&E systems by using evidence for decision making and policy

Resources have been committed and ring fenced for M&E activities from Government Budget.

Evaluation findings are to be disseminated at the bi-annual gov’t performance retreats of Cabinet Ministers, Permanent Secretaries & Local government representatives.

Evaluations are beginning to be viewed positively, not just as a punitive exercise but with positive comments & ownership by departments & stakeholders

There is need to ensure that results from evaluation are translated into use.

As Government starts to produce findings from evaluations, these will be rolled into the bi-annual Cabinet Retreats.

11.0 Conclusion and Way forward

20

End of presentation

Thank you!