Targeted Data Quality Review -M&E System Assessment Component

16 March 2016Geneva, SwitzerlandPresented to LFAs

Targeted Data Quality Review: M&E Systems Assessment Component• Part of Data Quality Assessment

• The M&E Systems Assessment is one of 3 components of the Data Quality Review. The M&E Systems Assessment component evaluates the extent to which critical elements of the reporting system adhere to a set of minimum acceptable standards; it is conducted at the facility and/or district level.

• Tools:

M&E Systems Assessment component of the WHO Data Quality Review Toolkit

Targeted Data Quality Review Reporting Template

1

Objectives

a. The objective of the M&E Systems Assessment component of the DQR is to assess the capacity of the data system to produce good-quality data.

b. The results of M&E Systems Assessment can also serve to inform the reasons for the results of the Data Verification component of the DQR.

2

Description

a. The M&E systems assessment is one of 3 components of the Data Quality Review.

b. The M&E Systems Assessment component evaluates the extent to which critical elements of the reporting system adhere to a set of minimum acceptable standards.

c. The M&E Systems Assessment is conducted at the health facility level, and can also be conducted at the district level. There is both a facility level and a district level M&E Systems Assessment paper tool in the WHO Data Quality Review Toolkit.

d. The M&E Systems Assessment component generally should be conducted alongside the Data Verification component of the DQR (i.e. at the same facilities/districts at the same time).

3

Changes/difference from OSDV Model

a. Tools: The M&E Systems Assessment within the WHO DQR Toolkit is harmonized from Global Fund and GAVI tools, so is similar to the M&E Systems Assessment within the OSDV.

b. Analysis/Reporting will be the red/yellow/green “scorecard” for all facilities/districts.

c. Much less focus on comments. No comments per individual indicator reported.

d. More focus on reporting a small number of key issues/recommendations based on the indicator data.

4

Tools (1)

Guidance: The WHO Data Quality Review Toolkit should be used for conducting the M&E Systems Assessment component of the DQR.

Planning: The Targeted Data Quality Review Planning Template and adapted Targeted Data Quality Review TOR

Data Collection: The WHO DQR Toolkit Data Collection Tools file contains the questionnaire for the facility level M&E Systems Assessment, as well as the questionnaire for the district level M&E Systems Assessment

Data Analysis: General guidance for analysing and presenting the M&E Systems Assessment results are given in the WHO DQR Toolkit.

Reporting: The Targeted Data Quality Review Reporting Template.

5

6

M&E System Assessment – Health Facility LevelTools (2)

7

M&E System Assessment – District HMIS/Program UnitTools (3)

Planning (1)

a. Adapting the TOR: Customize the M&E Systems Assessment relevant sections of the sample Targeted Data Quality Review for the country specific activity in agreement with the GF Country Team.

i. Geographic area

ii. Sampling

iii. Which level(s) the M&E Systems Assessment will be conducted at

iv. Expected LOEv. Planning Template: After the TOR is agreed on, the Targeted Data Quality Review

Planning Template should be submitted to the country team for approval, with the TOR as an annex .

8

Planning (2)

c. Communications needed with CT, with PR, between CT and PR, with other stakeholders

d. Other planning considerations:

i. Logistics of facility visits

ii. Permissions/letters of introductions from relevant government sources to visit the facilities

iii. Adapt, reproduce survey tools

9

Sampling

Unless there are special circumstances, the same facilities sampled for the Data Verification component of the DQR should also be used for the M&E Systems Assessment.

This is most efficient, but also allows for the M&E Systems Assessment results to inform the Data Verification findings.

10

Data Collection

The elements of the reporting system that are evaluated are: Monitoring and evaluation (M&E) structure and function; Indicator definitions and reporting guidelines; Data collection tools and reporting forms; Data quality monitoring and supervision; Data maintenance and confidentiality; Demographic information (district level only); Use of data for decision-making (district level only).

Walkthrough the Facility M&E Systems Assessment questionnaireWalkthrough the District M&E Systems Assessment questionnaire

11

Data Entry

a. WHO is creating a CSPro data entry application for the Data Verification and the M&E Systems Assessment components of the DQR, but this will be geared towards nationally representative implementations of the DQR.

b. For the Targeted Data Quality Review assessment activities, the Targeted DQR Reporting Template can be used for data entry and reporting of the M&E Systems Assessment component.

12

Data Analysis (1)

a. The WHO Data Quality Review Toolkit discusses data analysis for the M&E Systems Assessment component

b. For the Targeted Data Quality Review assessment activities, the analysis will focus on the color-coded scorecard as described on page 42 of the WHO DQR Toolkit.

13

14

Data Analysis (2)

Reporting

a. For the Targeted Data Quality Review assessment activities, the Targeted DQR Reporting Template can be used for data entry and reporting of the M&E Systems Assessment component.

b. Walkthrough M&E Systems Assessment section of the Targeted DQR Reporting Template

15

Follow-up and Data Use

a. Debriefing: Whether or not to debrief at the site level and/or the PR, when, and what items should be included in the debrief must be agreed on with the Global Fund Country Team in the Targeted DQR TOR prior to implementation of the activity

b. Action planning: Specific action planning expectations or requirements with the PR and the Global Fund should be agreed upon with the Country Team in the Targeted DQR TOR.

16

Questions?

17