METHODOLOGY FOR THE REVIEW/EVALUATION OF POLICY

DOCUMENTSBy Kwami DADJI, Health Officer HIV/AIDS, TB, Malaria & OID

African Union Commission

Presentation Outline

1. Introduction2. Objectives of the Evaluation3. Evaluation questions4. Methodology

1. Introduction• The main health policy documents of the African Union are

expiring on 2015.

• The final evaluation of the Abuja Call and for the Maputo Plan of Action will be conducted to assess their level of implementation by the UA Member States.

• The outcome will be presented to the Specialized Technical Committee on Health in April 2015 and to the AU Assembly of Heads of State and Government in June / July 2015.

2. Objectives

General objectiveTo document and assess the outcomes according to the set objectives of the Abuja Call and the Maputo Plan of Action.

Specific objectives • Assess the relevance (the global context) of the Abuja Call and the

Maputo Plan of Action; • Assess the coherence of priority areas under the Policy documents

(Abuja Call and the Maputo Plan of Actions) and indicators to measure them (nature of the indicators, ease of measurement, tools, methods and means of collection, analysis, etc.);

• Assess the effectiveness of actions implemented in the various countries in the strategic areas identified by the Policy documents ;

• Determine the level of achievement of key indicators in the context of monitoring and evaluation;

2. ObjectivesSpecific objectives • Assess the adequacy of the main indicators used in the monitoring and

evaluation framework with policy documents and national strategies; • Analyze the partnership engaged at the country, regional and

continental level (type, functionality, advantages and disadvantage); • Identify the strengths and weaknesses of the implementation of the

Policy documents ; • Analyze the implementation of the recommendations of the mid-term

reviews; • Propose a decision regarding the continuation, reorientation or

termination of implementing the Policy documents , taking in account the results and recommendations of the review as well as the African Common position on the Agenda beyond 2015;

• In case of continuation or redirection, provide consistent recommendations for better implementation of the Abuja Call and the Maputo Plan of Actions

3. Evaluation questions• General QuestionAbuja Call• The key question that this final evaluation seeks to answer is:

what changes the implementation of the Abuja Call has induced in the fight against HIV and AIDS, Tuberculosis, Malaria and other infectious diseases in the AU Member States?

Maputo Plan of Action • The key question that this final evaluation seeks to answer is:

what changes the implementation of the Maputo Plan of Action has induced in the improvement of the Sexual Reproductive Health and the MNCH in the AU Member States?

3. Evaluation questions• Specific questionsSpecific questions that will guide the overall work of the evaluation are as followsAbuja Call- What is the relevance of the Abuja Call in the fight against the priority diseases? - What is the efficiency in the implementation of the Abuja Call in the 5 African

regions? - The expected outcomes were they produced? There were any unexpected

outcomes? - What impact(s) in the medium term can be expected from the implementation of

the Abuja Call at country, regional and continental levels? -What is the added value of the Abuja Call in the fight against these priority

diseases, for example on health financing in Africa and the strengthening of health systems?

- Should we continue with the priority areas proposed by the Abuja Call? termination or reorientation?

3. Evaluation questions• Specific questionsSpecific questions that will guide the overall work of the evaluation are as

followsMaputo Plan of Action- What is the relevance of the Maputo Plan of Action in the improvement of the

Sexual Reproductive Health and the MNCH?- What is the efficiency in the implementation of Maputo Plan of Action in the 5

African regions? - The expected outcomes were they produced? There were any unexpected

outcomes? - What impact(s) in the medium term can be expected from the implementation

of the Maputo Plan of Action at country, regional and continental levels? -What is the added value of the Maputo Plan of Action in the improvement of

the Sexual Reproductive Health and the MNCH and the strengthening of health systems?

- Should we continue with the priority areas proposed by the Maputo Plan of Action? termination or reorientation?

4. Methodology

• Conducted by the Department of Social Affairs of the African Union Commission (AUC)

• The approach to be adopted is based primarily on the analysis of policy documents and commitment of the Member States, the monitoring and evaluation Plan, activity reports, the mid-term reports and any other documents relating to the implementation

4. Methodology

The evaluation approach will thus structured around the following activities: – Technical meeting at the AUC involving key stakeholders,

to validate the methodology and tools for data and information collection;

– Send data collection tools to members States and to partners for filling and collection of data on indicators.

– Invitation of selected Member States to discuss the relevance of selected indicators, measurement and difficulties of providing information in real time;

– Analysis of data and information; – Draft the evaluation report;– Validation workshop to finalize report;– Translation and submission of the report to the relevant

stakeholders

4. Methodology

Sampling• After analyzing the mid-term evaluation reports, it will be

selected in the five African regions, 10 countries including five which have the highest levels of key indicators and 5 which have the worst levels of the indicators. The countries representatives of the selected countries will be invited to AUC to discuss the main issues related to the implementation of the policy documents in their respective countries, strengths and weaknesses.

• Moreover, the discussions will also address the challenges of availability and sharing of information on key health indicators to enable monitoring to achieve the Abuja Call.

4. Methodology

Data collection tools

Technique Tools

Mixed: Quanti - Qualitative Questionnaire for country indicators

Qualitative

Guide for group discussion with partners

Guide for discussion with representatives of countries invited to the meeting to exchange with the AUC or by telephone interview, Skype etc ...

4. Methodology

Data analysisData analysis will be done at three levels. • The first level will be to analyze the relevance of policy

documents from an approach based on the qualitative analysis of the situational analysis of health in Africa and the framework of these documents.

• Regarding effectiveness in the implementation of the policy documents, the analysis will be based on the measurement of the level reached by the different Member States for the main indicators used in their monitoring and evaluation plan.

• In a third step, an contained analysis will be made on qualitative information gathered from key stakeholders (partners and representatives of the countries)

I thank you