Centre for University Continuing Education under mandate from the Federal Office of Public Health

Universität Bern

Evaluation in Public Health –Lessons Learned and Future Directions

Monitoring and evaluating HIV

Michel Caraël, Université Libre de Bruxelles

17 June, Zentrum für universitäreWeiterbildung, Bern,

World time

2 milestones in HIV evaluation at global level

• 1993-94. At the peak of global vulnerability to HIV: Sense of failure to mobilize and respond to a global threat. Need to increase program effectiveness.

• 2001-2002. Redefinition of AIDS as a human security and development issue at global level. Massive investment in HIV response generated strong accountability demands.

Global program on AIDS M&E pioneers searching for a case study

Pablo Picasso

30

20

10

0

35

25

15

5

Million

1980 1985 1990 1995 2000 2005

1.1

The first momentum

o 1993,124 countries with national AIDS program & medium term plan & 80 external national program reviews conductedo 1994, Evaluation unit created in WHO GPAo 1994, Methods package : Evaluation of national AIDS program; prevention of HIV infection

People living with HIV

2010

Lessons learned in GPA

1. Recognition of the need to use standardized approaches in the evaluation of AIDS program

2. Bringing in behaviors and integrate HIV surveillance with behavioral surveillance

3. Tension between M&E for reporting and for program improvement

4. Fragmentation of M&E efforts across many agencies: need for harmonization and coordination

5. Implementation of M&E by National programs requiring strong capacity building

30

20

10

0

35

25

15

5

Million

1980 1985 1990 1995 2000 2005

1.1

2d momentum: HIV as global priority

• 2000, UN Security Council resolution on AIDS2000, UN Security Council resolution on AIDS• 2001, UNGASS declaration on HIV2001, UNGASS declaration on HIV• 2001, The Global Fund2001, The Global Fund• 2003, The three ones principles: one M&E framework2003, The three ones principles: one M&E framework

People living with HIV

2010

Estimated global resources for HIV% of countries with M&E systems

N° of global M&E guides & indicators

0

8 billion US $

1996 1998 2000 2003 2005

Declaration of Commitment on

HIV/AIDS

1,000

75% with M&E Systems

The M&E army

UNAIDS, WHO, UNICEF, and the US Government have placed over 80 M&E experts in priority countries.

Role is to assist national governments in the implementation of unified M&E systems.

Focus is on capacity-building of national counterparts in evaluation.

Interconnected world

156 million hits for « HIV »46 millions hits for « HIV

evaluation »

Challenges and directions for the future in HIV

1. M&E system for program improvement and/or accountability?

2. M&E: a need to strengthen E ?

3. Evaluation of complex systems: are gold standards still gold?

1. Reporting, accountability and/or management

• Accountability demands tend to overthrow program improvement.

• Most M&E frameworks are not designed as management tool.

• Tension between bottom-up and top-down design

2. Still great evaluation needs

• Some saturation of HIV information

• Continuous need for strategic data for policy level

• Knowledge is lacking on which combinations of prevention and treatment strategies work best, under which conditions.

• The evidence base for designing effective packages of national programs against HIV is still thin.

3. Gold standard debate in impact evaluation of HIV response

• What is “evidence”?

• Methodological orthodoxy versus appropriateness, especially for complex multi-dimensional programs

• Evaluation of HIV combination prevention requires the use of variety of different methods.

Results of 40 interventions 38 RCTs to prevent sexual transmission of HIV

Type of Intervention HIV prevention efficacy Total

Positive effect

Adverse effect

No effect

Behavioral & Microfinance

--- --- 8 8

Diaphragm --- --- 1 1

Vaginal Microbicides 1 1 11 13

Male Circumcision 3 --- 1 4

HIV Treatment, PrEP --- --- 1 1

STI Treatment 1 --- 8 9

Vaccines 1 --- 3 4Total 6 1 33 40

Hayes R et al. AIDS, 2010

Why flat trials?

1. Invalid concept or inadequate intervention

2. Limitations in the delivery or intensity of interventions

3. Issues in trial design or conduct

• limited duration of follow-up

• over-riding influence of cultural norms and socio-economic

factors

• inadequate power (poor adherence, losses to follow up)

• high intensity of intervention in control arms; Contamination

between arms

• Biases associated with self-reported sexual behaviour.

Combination prevention & Evaluation