![Page 1: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/1.jpg)
Centre for University Continuing Education under mandate from the Federal Office of Public Health
Universität Bern
Evaluation in Public Health –Lessons Learned and Future Directions
Monitoring and evaluating HIV
Michel Caraël, Université Libre de Bruxelles
17 June, Zentrum für universitäreWeiterbildung, Bern,
![Page 2: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/2.jpg)
![Page 3: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/3.jpg)
World time
2 milestones in HIV evaluation at global level
• 1993-94. At the peak of global vulnerability to HIV: Sense of failure to mobilize and respond to a global threat. Need to increase program effectiveness.
• 2001-2002. Redefinition of AIDS as a human security and development issue at global level. Massive investment in HIV response generated strong accountability demands.
![Page 4: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/4.jpg)
Global program on AIDS M&E pioneers searching for a case study
Pablo Picasso
![Page 5: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/5.jpg)
30
20
10
0
35
25
15
5
Million
1980 1985 1990 1995 2000 2005
1.1
The first momentum
o 1993,124 countries with national AIDS program & medium term plan & 80 external national program reviews conductedo 1994, Evaluation unit created in WHO GPAo 1994, Methods package : Evaluation of national AIDS program; prevention of HIV infection
People living with HIV
2010
![Page 6: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/6.jpg)
Lessons learned in GPA
1. Recognition of the need to use standardized approaches in the evaluation of AIDS program
2. Bringing in behaviors and integrate HIV surveillance with behavioral surveillance
3. Tension between M&E for reporting and for program improvement
4. Fragmentation of M&E efforts across many agencies: need for harmonization and coordination
5. Implementation of M&E by National programs requiring strong capacity building
![Page 7: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/7.jpg)
30
20
10
0
35
25
15
5
Million
1980 1985 1990 1995 2000 2005
1.1
2d momentum: HIV as global priority
• 2000, UN Security Council resolution on AIDS2000, UN Security Council resolution on AIDS• 2001, UNGASS declaration on HIV2001, UNGASS declaration on HIV• 2001, The Global Fund2001, The Global Fund• 2003, The three ones principles: one M&E framework2003, The three ones principles: one M&E framework
People living with HIV
2010
![Page 8: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/8.jpg)
Estimated global resources for HIV% of countries with M&E systems
N° of global M&E guides & indicators
0
8 billion US $
1996 1998 2000 2003 2005
Declaration of Commitment on
HIV/AIDS
1,000
75% with M&E Systems
![Page 9: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/9.jpg)
The M&E army
UNAIDS, WHO, UNICEF, and the US Government have placed over 80 M&E experts in priority countries.
Role is to assist national governments in the implementation of unified M&E systems.
Focus is on capacity-building of national counterparts in evaluation.
![Page 10: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/10.jpg)
Interconnected world
156 million hits for « HIV »46 millions hits for « HIV
evaluation »
![Page 11: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/11.jpg)
![Page 12: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/12.jpg)
Challenges and directions for the future in HIV
1. M&E system for program improvement and/or accountability?
2. M&E: a need to strengthen E ?
3. Evaluation of complex systems: are gold standards still gold?
![Page 13: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/13.jpg)
1. Reporting, accountability and/or management
• Accountability demands tend to overthrow program improvement.
• Most M&E frameworks are not designed as management tool.
• Tension between bottom-up and top-down design
![Page 14: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/14.jpg)
2. Still great evaluation needs
• Some saturation of HIV information
• Continuous need for strategic data for policy level
• Knowledge is lacking on which combinations of prevention and treatment strategies work best, under which conditions.
• The evidence base for designing effective packages of national programs against HIV is still thin.
![Page 15: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/15.jpg)
3. Gold standard debate in impact evaluation of HIV response
• What is “evidence”?
• Methodological orthodoxy versus appropriateness, especially for complex multi-dimensional programs
• Evaluation of HIV combination prevention requires the use of variety of different methods.
![Page 16: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/16.jpg)
Results of 40 interventions 38 RCTs to prevent sexual transmission of HIV
Type of Intervention HIV prevention efficacy Total
Positive effect
Adverse effect
No effect
Behavioral & Microfinance
--- --- 8 8
Diaphragm --- --- 1 1
Vaginal Microbicides 1 1 11 13
Male Circumcision 3 --- 1 4
HIV Treatment, PrEP --- --- 1 1
STI Treatment 1 --- 8 9
Vaccines 1 --- 3 4Total 6 1 33 40
Hayes R et al. AIDS, 2010
![Page 17: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/17.jpg)
Why flat trials?
1. Invalid concept or inadequate intervention
2. Limitations in the delivery or intensity of interventions
3. Issues in trial design or conduct
• limited duration of follow-up
• over-riding influence of cultural norms and socio-economic
factors
• inadequate power (poor adherence, losses to follow up)
• high intensity of intervention in control arms; Contamination
between arms
• Biases associated with self-reported sexual behaviour.
![Page 18: Centre for University Continuing Education under mandate from the Federal Office of Public Health Universität Bern Evaluation in Public Health – Lessons](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e1b5503460f94b0a1cf/html5/thumbnails/18.jpg)
Combination prevention & Evaluation