advocacy, communication and social mobilization at country level

2nd Meeting of the Subgroup on Advocacy, Communications and Social

Mobilizationat Country Level

Advocacy, Communication and Social Mobilization at Country Level

Presentation to the ACSM Subgroup at Country Level



Outline

1. What are the monitoring and evaluation challenges for ACSM?

2. What does the 10-year ACSM Framework prescribe?

3. What 5-7 indicators could be used in a country’s ACSM performance profile?



2. What has the ACSM Sub Group at country level achieved?



…but M&E not yet standardized

19

9

0

2

4

6

8

10

12

14

16

18

20

2002 2005 2007

HBC's withNational ACSMPlans

2

12

18

0

2

4

6

8

10

12

14

16

18

20

2002 2004 2006*

Countries

2002 data as gathered from ACSM Needs Assessment Survey of HBCs 2006-2007 data gathered from HBC DOTS Expansion survey (self reporting)*Approved grants from 5th round GFATM

HBCs with GFATM funding via rounds 1-5 for ACSM activities

Rapid Growth in Plans for ACSM utilization

Momentum Building…

$63 million over 5 years



3. What are the monitoring and evaluation challenges?



Currently, we have:Minimal dataOld dataUnsystematic data No centralized collection point



Colombia



The Philippines' story

0

20

40

60

80

100

120

140

160

no. of symptomatics no. of TF referrals no. of new smearpositive

no. of new smearpositive from TF

referrals

2001 2002 2003 2004 2005

Barangay Pangobilian, Brooke’s Point, PalawanCase Finding Activities



ACSM impact can be shown but the measurement can be disputed without standard measurement

Consultant report on Kerala November 2005

There were increases in all but one of the 14 districts in the State with half the districts (7 of the 14) registering increases of 13% - 41%.

The comparison between 4th Qrt, 2003 and 1st Qrt, 2004 showed an increase of 19%.

The comparison between 2nd Qrt, 2003 and 2nd Qrt, 2004 showed an increase of 7%.

RNTCP collected data



ACSM impact can be measured in carefully designed evaluation studies

However we have challenges: Validity, reliability, and repeatability Contribution analysis is not straightforward Contextualization Lack of uniformity of indicators across

countries Evaluation of ACSM studies not part of routine

TB M&E Need for participatory monitoring and

evaluation

Lessons



These challenges can be overcome,

BUT only with some retooling and investment in M&E



4. What does the 10-year ACSM Framework prescribe?



Five streams of evidence (at least)

Country-level ACSM develop their own M&E mechanisms (using Guideline as a starting point)

Regular ACSM evaluations commissioned by the Stop TB Partnership (akin to ACSM country case studies)

5-7 ACSM standard indicators for country-by-country reporting

Regular reports from ACSM agencies partnered to NTPs through Technical Service Contracts

Regular NTP reviews to incorporate ACSM measures



Work in ProgressMonitoring and Evaluating ACSM in National TB Programmes: A Guideline on Indicators ACSM capacity ACSM activity Sputum testing Treatment adherence Stigma and discrimination Monitoring and evaluating

advocacy Measuring significant

changes Measuring social change

communication TB social change

indicators



5. What 5-7 indicators could be used in a country’s annual ACSM performance profile?



Preface

Suggested indicators are illustrative Any standard set of indictors will need

country and partner consultation Based on more than 50 years of health

communication experience and study



Case detection

Inputs Outputs

InputsCase cure Outputs

Outcomes

Outcomes

Impact

Impact

Research,

plans,

resources,

supplies,

staff,

etc.

Research,

plans,

resources,

supplies,

staff,

etc.

Social mobilization and communication activities,

knowledge,

policies,

laws,

incentives

Social mobilization and communication activities,

knowledge,

policies,

laws,

incentives

Sputum-testing,

reduced stigma,

reduced discrimination,

other significant social changes

Treatment adherence,

reduced stigma,

reduced discrimination,

other significant social changes

Increase case detection rates

Increase cure rates,

decrease TB incidence,

decrease TB mortality,

reduce risk of MDR-TB,

improve quality of life

ACSM Indicator Flow



ACSM 10-year Framework Action Areas (Illustrative) Increasing capacity and technical assistance

for ACSM Improving case detection and treatment

adherence Combating stigma and discrimination Empowering people and communities

affected by TB Mobilizing political commitment and

resources for TB control



ACSM Capacity Country has an evidence-based,

sufficiently resourced ACSM Strategic Plan

Composite measures: % districts/provinces with TB ACSM plans based on

recent formative research % national health budget devoted to TB ACSM (5-

15%) % districts/provinces with designated ACSM staff with

appropriate qualifications

Parks W. with T Pennas and J Deane. 2006. Monitoring and evaluating advocacy, communication and social mobilization in national TB control programs.: a guideline on indicators. Geneva, World Health Organization.



Case detection and case cure % of population who know that going to a

DOTS clinic is the best way to diagnose TB % of districts that have active pro-poor

incentive schemes for TB patients on DOTS

ACSM contribution to CDR and CCR could be produced through evaluation of “sentinel sites” (e.g. districts – pre/post, time series, intervention/control)

Parks W. with T Pennas and J Deane. 2006. Monitoring and evaluating advocacy, communication and social mobilization in national TB control programs.: a guideline on indicators. Geneva, World Health Organization.



Combating stigma and discrimination % of population expressing accepting

attitudes towards TB suspects, patients and survivors

Most Significant Change Stories (qualitative)

Davies, R & J Dart. 2005. The most significant change 'MSC' technique: a guide to its use. Version 1.00. http://www.mande.co.uk/docs/MSCGuide.pdf

International Centre for Research on Women. 2002. Understanding HIV-related stigma and resulting discrimination in sub-Saharan Africa.



Patient empowerment # of TB patients or survivors trained to

become educators, monitors or advocates Most Significant Change Stories

(qualitative)

The Global Fund to Fight AIDS, TB and Malaria. Monitoring and Evaluation Toolkit: HIV/AIDS, Tuberculosis and Malaria, Second Edition. 2006. www.theglobalfund.org.

Davies, R & J Dart. 2005. The most significant change 'MSC' technique: a guide to its use. Version 1.00. http://www.mande.co.uk/docs/MSCGuide.pdf



Political commitment

Multi-sectorial involvement exists at the national level for TB control

% national budget dedicated to TB control

Singhal A and E Rogers. 2003. Combating AIDS: communication strategies in action. New Delhi, Sage Publications: 351-353.



5. What is the Subgroup's counsel?



1. To what extent is it desirable to develop a standard set of ACSM indicators that are applicable across all programs and contexts to inform on the quality and progress of ACSM interventions?

2. How can M&E of ACSM be better integrated into current reporting structures?

3. How should WHO/ACSM Sub-Group advance this M&E work?

4. What is the best way forward?



Recommendations Should there be standard indicators? Need to communicate in better ways Standardized surveys Link to other surveys (DHS and others) Diagnostic delay needs to be included Need simplicity Everyone is not starting at the same place. Population based may not be the best way to go. Country based M&E needs to be enhanced Options:

Throw away the guide Circulate and comment Revise the guide and circulate.

Countries NEED some guidelines to base M&E plan. Reporting at global level and local level.

Develop a hierarchy for results or reporting Can we look similar indicators across countries and measure those? Look at the workplans to revise the guide to find common indicators To look at the beneficiary of the information. Develop circulation

advocacy, communication and social mobilization at country level

Documents