Assessment of Program Evaluation Activities in Tuberculosis Control
Programs — United States, 2009–2010
Silvia M. Trigoso, MPHFellow, Public Health Prevention Service (PHPS)
TB Program Evaluation Network Annual ConferenceSeptember 21, 2011
National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention
Division of Tuberculosis Elimination
Today’s Presentation
Purpose Objective Methods Results Limitations Conclusions Recommendations Questions
Purpose of the Assessment
To understand the status, progress, and challenges in planning and
implementing program evaluation activities in TB control programs.
Objective
Assessment addressed 3 evaluation questions:
What is the evaluation focus area(s) reported in the CoAg and interim reports and is it in alignment with National TB Program Objectives and Performance Targets for 2015?
To what extent does the TB control program’s evaluation plan incorporate elements of the CoAg outlined evaluation guidance?
What are challenges and opportunities reported in implementing proposed PE activities?
Methods
Variables were created and database developed for data collection and management.
Data sources included grantees’ program evaluation plans and interim reports submitted to FSEB as part of the 2010-2014 CoAg.
Reviewed program evaluation sections submitted as part of the 2010-2014 CoAg FOA and interim reports submitted in August 2010.
Data analysis was conducted and results were obtained.
Results
No Focus Area Reported At Least 1 Focus Area0
10
20
30
40
50
60
70
10
58
Number of Grantees Reporting a Focus Area for Evaluation
in 2010 Interim Report(N = 68 grantees)
# of
Gra
ntee
s
Only 1 Focus Area 2 or More Focus Areas
Outside of Scope0
5
10
15
20
25
30
35
40
45
50
43
15
10
Number of Focus Areas for Evaluation Re-ported by Grantees in 2010 Interim Report
(n = 58 grantees)
# of
Gra
ntee
s
Evaluation Focus Areas Reported by Grantees, 2010 Interim Report (n = 58)
Completion of Treatment 6
TB Case Rate 4
Contact Investigation 24
Laboratory Reporting 1
Treatment Initiation 1
Sputum Culture Conversion 8
Data Reporting 2
Recommended Initial Therapy 2
Universal Genotyping 2
Known HIV Status 13
Evaluation of Immigrants and Refugees 4
Sputum-Culture Reported 3
Program Evaluation/Evaluation Focal Point
1
Human Resource Development 1
Training Focal Point 0
Focus Area # of Grantees
Contact Investigation/Elicitation
(22%)
Contact Investiga-tion/
Evaluation(27%)
Contact Investigation/Tx Initiation
(18%)
Contact Investigation/Tx Completion
(33%)
% Of Grantees Reporting Contact Investi-gation
by Contact Investigation Category, 2010 IR (n = 24 grantees)
Unchanged Focus Area67% (42)
Changed due to NTIP Data 11%(7)
Changed due to program evaluation representative
technical assistance 9% (6)
Changed due to program consultant technical as-
sistance 6% (4)
Changed due to limited staff/less labor intensive 5% (3)
Evaluation plan in development w/PET TA2% (1)
Change in Reported Focus Area for Evalu-ation,
2009 CoAg & 2010 Interim Report (n = 62)
Pre-Planning11% (7)
Planning16% (10)
Implementation45% (28)
Effects2% (1)
Cannot be De-termined26% (16)
Stage of Development for Program Evaluation, 2010 Interim Report
(n = 62 grantees)
Increased in Level Decreased in Level Unchanged0
5
10
15
20
25
30
35
40
6
35
21
Difference in Program Evaluation CoAg Guidance Reported in Interim Report
(n=62 grantees)
Gra
nte
es
Limited staff/resources for evaluation activities
Contacts' interest/ability to come to PH office for evaluation
TB Staff Turnover
Topographical & transportation barriers
TB cases low
Cultural & Linguistic competency
Patients' value and understanding of LTBI tx when asymptomatic
Social factors impacting TB treatment delivery
Discontinuation of TIMS
CI in detention centers (evaluation and tx of contacts)
Demonstrating PE as useful
Outreach to rural areas
Stigma attached to TB disease and LTBI/hinders follow-up & mgmt.
Decentralized PH system/lack of admin or regulatory authority to comply with nat'l objective
0 5 10 15 20 25 30 35
33
External Challenges Reported in Interim Report
(n =40 grantees)
Difficulty with data analysis
Health care provider/lack of TB education
Timely data entry
Record keeping/organization
Lack of experience in survey design
Lack of cooperation in data collection
Lack of experience with survey tools
Lack of participation in survey completion
0 1 2 3 4 5 6
Internal Challenges Reported in Interim Report
(n = 14 grantees)
Limitations
Assessment was limited to what grantees reported on submitted reports.
Not all grantees had program evaluation plans.
Could not obtain interim reports for all grantees.
Grantee’s interim reports did not reflect an update of PE activities conducted between reporting time period.
Conclusions The three most frequently reported focus areas for
evaluation are: (1) contact investigation; (2) known HIV status; and (3) sputum culture conversion.
Over 80% of grantees report at least more than 1 focus area. 10 grantees report a focus area not in alignment with National TB Objectives for 2015.
Over 65% of grantees did not change their focus area for evaluation. For some grantees who did change focus areas it was reported to be because of technical assistance provided by FSEB program evaluation representatives or program consultants.
There is a difference in PE CoAg guidance reported in interim report with most grantees decreasing in level.
The most frequently reported external challenge reported by grantees is limited staff and resources for conducting evaluation activities.
Recommendations
Stronger program evaluation reporting guidance is needed. Reporting formats vary across regions.
Identify areas for program evaluation team (PET) to provide TA.
Understanding challenges grantees face in implementing PE activities (e.g. , limited staff/resources + TB staff turnover) provides PET opportunity to be a valuable and useful resource.
Encourage use of monitoring tools to guide focus areas for evaluation and impacting TB programs (i.e. local data, NTIP, ARPEs).
Acknowledgements
Brandy Peterson & Awal Khan Program Consultants Field Services and Evaluation Branch Tom Chapel, CDC Chief Evaluation Officer
Contact Information
Silvia M. TrigosoPublic Health Prevention Service Fellow
Email: [email protected]
For more information please contact Centers for Disease Control and Prevention
1600 Clifton Road NE, Atlanta, GA 30333Telephone: 1-800-CDC-INFO (232-4636)/TTY: 1-888-232-6348E-mail: [email protected] Web: http://www.cdc.gov
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
National Center for HIV/AIDS, Viral Hepatitis, STD , and TB Prevention
Place Division name here