std/hiv prevention operational and comparative effectiveness research kees rietmeijer, md, phd...

STD/HIV Prevention Operational and Comparative Effectiveness

Research

Kees Rietmeijer, MD, PhD

Colorado School of Public HealthUniversity of Colorado Denver

Contact: kees.rietmeijer@dhha.org

Principal’s Course 2010Seattle, July 21, 2010

Resources

www.stdpreventiononline.org

Efficacy

How an intervention works under optimal conditions

Effectiveness

How an intervention works in the real world

Efficacy vs EffectivenessEfficacy

• Setting: research/study

• Resources: high

• Priority: high

• Staff: well trained and well-paid; focused on intervention

• Quality assurance: high

• Research subjects selected and motivated

• Focus on internal validity

• Project time is limited

Effectiveness• Setting: practice

• Resources: medium-low

• Priority: medium-low

• Staff: insufficiently trained, low-paid; multiple competing tasks

• Quality assurance: low

• Unselected and less-motivated clients

• Focus on external validity

• Sustainability issues

Evaluation• Process

– Monitoring• Quantity and quality of provider-delivered interventions

– Evaluation• Was intervention implemented as intended?

• Outcome – Monitoring

• Changes in behavior or incident STD/HIV among clients receiving the intervention

–Evaluation• Same as above, but including comparison/control groups

• Impact– Evaluation

• Overall impact on morbidity and mortality of the intervention at the community level

Normalizing HIV Testingin an STI Clinic Setting

CDC Recommendations

• “All patients seeking treatment for STDs, including all patients attending STD clinics, should be screened routinely for HIV during each visit for a new complaint, regardless of whether the patient is known or suspected to have specific behavior risks for HIV infection.”

8

MMWR 2006 / 55(RR14);1-17

HIV Testing at Denver STD ClinicBefore November 2003

• General consent for all procedures and testing, except HIV testing, obtained at registration

• HIV testing offered by clinician during the clinic visit, based on risk assessment

• Blood drawn for syphilis and HIV (if accepted) testing during the clinic visit

• HIV test used: standard EIA

9

HIV Testing at Denver STD ClinicAfter November 2003

• November 2003: Rapid HIV testing (OraQuick) offered –First as optional alternative to standard EIA

–Routine after July, 2004

• May 2004: Change in testing logistics

• March 2005: Introduction electronic medical record and switch from opt-in to opt-out HIV testing

10

HIV Testing at Denver STD ClinicChange in Testing Logistics

• To avoid adding another 20 minutes to the visit, prior to clinic encounter:–Draw RPR blood before clinician sees

patient

–Offer HIV testing routinely by clerical/veni-puncture staff

–Use RPR blood draw to collect extra tube for rapid HIV test

11

Evaluation

• Inclusion/Exclusion criteria–New problem visits

–RPR performed

–Previously known HIV+ excluded

• Main outcome: HIV/RPR ratio–RPR used as the gold standard of routine

testing

12

Impacts of Rapid Testing

13

Denver Metro Health Clinic

Percentage of patients who received their positive test results:

Before: After:

66 % 100%

14

HIV/RPR Ratio and HIV Positivity Rate by Period

January 2005 - October 2006

0.79

0.86

0.92

0.96

0.5%

0.8%

0.6%0.7%

0.60

0.80

1.00

Period I: Jan 2003 -

Nov 2003

Period II: Dec 2003 -

May 2004

Period III: Jun 2004 -

Mar 2005

Period IV: Apr 2005 -

Oct 2006

Period

Ra

tio

(HIV

:RP

R)

0.0%

1.5%

HIV

(+)

Ra

te

HIV:RPR Ratio

HIV+ Rate

Brooks et al. Sex Transm Dis 2009

Express / Fast-TrackClinic Visits

Express / Fast Track Visits

• Purpose–To enhance clinic efficiency and patient

satisfaction through a process by which qualifying patients are offered a testing-only, no-exam visit

16

Express / Fast Track Visits

• Who is eligible?–Asymptomatic patients regardless of

• Demographics

• Risk or contact status

• Sexual preference

17

Express / Fast Track Visits

• How is it done?–All patients go through triage after registration

–Triage staff determines eligibility

–Patients can opt for comprehensive visit if they desire regardless of eligibility for express visit

–Patients are discouraged to opt for express visit if symptomatic, but are not denied

18

Express / Fast Track Visits

• What is done?–HIV rapid test (unless opt-out)

–RPR

–CT and GC NAAT• Males: urine

• Females: self-obtained vaginal swab

–Gonorrhea cultures among at-risk MSM• Anal swab

• Pharyngeal swab

19

20

Shamos et al. Sex Transm Dis 2008;35:336-340.

21

Shamos et al. Sex Transm Dis 2008;35:336-340.

Time & Motion Study

• Median visit duration men–Comprehensive: 85

–Express: 52 minutes

–Reduction: 39%

• Media visit duration women–Comprehensive: 105

–Express: 46

–Reduction: 56%

22

Shamos et al. Sex Transm Dis 2008;35:336-340.

Providing STD Test ResultsOnline

Results Online• How it works:

– Program started June 2008 as an alternative to in-person and telephone results

– Results available online after 5 days

– Originally, patients opted in and had to create their own (strong) password (opt-in)

– Since December 2008, patients are given a login code and a predetermined password (opt-out)

– Patients are given instructions in case of positive results

– Access of online results automatically registered in patient medical record

24

25

26

27

28

0

10

20

30

40

50

60

70

80

Decem

ber '0

7

Janua

ry '0

8

Febru

ary

Mar

chApr

ilM

ay

June

July

August

Septe

mbe

r

Oct

ober

Novem

ber

Decem

ber

Janua

ry '0

9

Febru

ary

Mar

ch

April

% of patients receiving results via the telephone % of patients receiving results online

DMHC Online Results Evaluation: 12/07 – 04/09

Ling et al. Sex Transm Dis 2009; In Press.

Providing Test Results OnlineLessons Learned

• Proportion of patients receiving results online–Opt-in Phase: 25%

–Opt-out Phase: 50%

• No net increase or decrease of patients receiving results

• Substantial savings in clerical time spent on answering phone calls

29

Ling et al. Sex Transm Dis 2009; In Press.

So, you can study it and you can even publish it…..

But is it Science?

• “Generalizable Knowledge”– Internal validity: Does it work under optimal

circumstances?

–Methodology: RTC is gold standard

• “Generalized Knowledge”–External validity: Does it work in the real world?

–Face validity: “Looks like a winner”

–RTCs often not feasible

–Adoption only in part dependent on evidence

No Respect

33

Project RESPECT(1993 – 1995)

• Multi-site, randomized controlled trial involving patients of 6 STD clinics in the US

• Studied the efficacy of different counseling intervention strategies during HIV pre- and post test sessions (1 week apart)– 2 x 5 minutes educational messages

– 2 x 20-minutes client-centered counseling

–1 20 minute session followed by 3 1-hour enhanced counseling modules

35

Project RespectMain Results

• Compared to standard education messages, client-centered counseling resulted in overall STD reduction of 30% after 6 months and 20% after 12 months

• 2-session prevention counseling was as effective as the 4-session enhanced counseling

Kamb et al. JAMA 1998;280:1161

Kamb et al. JAMA 1998;280:1161-1167

37

Project Respect

Relative effectiveness was greatest among those at highest risk for STI

# STI prevented per 100 persons counseled

– 20 years and younger 9.1– Exchange sex for money or drugs 5.9– STD at baseline 5.3– Lower education (<12th grade) 4.3– Female 3.9– African American 3.2

Bolu et al. Sex Transm Dis 2004;31:469.

The bad news is…

...that the good news is old news…

39

Prevention Counseling the main Challenge

• How to implement prevention counseling in the busy practice setting?–Competing needs

–Resource constraints

–Lack of provider buy-in

–Lack of supervisory buy-in

Online PN - InSpot

Project Description

• inSPOT Colorado went live in January 2006

• 2 interventions to enhance use of the site– State-level: October 2008

• Banner ads on ManHunt

• Public Service Announcements on popular radio station

• Local newspaper ads aimed at at-risk populations

– Denver Metro Health Clinic: August 2008• inSPOT palm cards to persons who are traditionally given

contact cards

• Poster/flyers in waiting room

• Link provided on STI clinic website

Project Evaluation• Clinic-level

– Purpose: to assess if there was an increase among visitors who accessed the clinic as a result of notification through inSPOT

– Method: Waiting room surveys before and during/after clinic-based interventions• Systematic sample of ~500 clinic visitors

• Time frames: – Survey 1: 10/2008 – 5/2008

– Survey 2: 8/2008 – 2/2009

Project Evaluation

• State-level

– Purpose: to assess visits to the Colorado inSPOT site before and during/after state-based and clinic-based interventions and specific STIs for which the service was used

– Method: monthly site statistics provided by inSPOT owner (ISIS)

ResultsClinic - Level

Survey 1 Survey 2

Total Respondents 453 481

N % (95% CI) N % (95% CI)

Heard of inSPOT 22 4.9 (3.0 – 7.4) 28 5.8 (3.8 – 8.4) Ever Sent eCard 1 0.2 (0 – 0.7) 10 2.0 (0.9 – 3.7)

Ever Received eCard 2 0.4 (0.05-1.4) 5 1.0 (0.3 – 2.3)

ResultsClinic - Level

• If you were diagnosed with an STI, how would you tell your partners?

SURVEY 2 N % 95% CI

Face-to-Face 430 89.4 (86.7 – 91.1)Telephone 176 36.6 (32.3 – 40.9)Written note or letter 18 3.7 (2 – 5.4)Text message 53 11.0 (8.2 – 13.8)E-mail / Internet 23 4.8 (2.9 – 5.7)Would not tell 12 2.5 (1.1 – 3.9)Ask someone else to tell them 8 1.7 (0.6 – 2.8)

PLoS Publication

Results – State Level

eCards Sent

2008 2009

ResultsState-Level (1/2008 – 6/2009)

Chlamydia 297 15.8 (14.2 – 17.4)

Gonorrhea 307 16.3 (14.7 – 17.9)

Syphilis 225 11.9 (10.4 – 13.4)

Crabs/Scabies 537 28.5 (26.5 – 30.5)

Hepatitis A 73 3.9 (3.1 – 4.7)

Molluscum Contagiosum 85 4.5 (3.6 – 5.4)

Non-gonococcal urethritis 41 2.2 (15.5 – 2.9)

Shigella 23 1.2 (0.7 - 1.7)

Other STD 297 15.7 (14.1 – 16.3)

Total Cards 1885

STI Identified on eCARD N % (95% confidence interval)

Conclusions Clinic - Level

• Small proportion of clinic visitors mention the Internet PN service as reason for visit to the clinic

• Realistic/feasible clinic-level interventions did not appear to increase use of the service

• Large majority of clinic patients prefer face-to-face or telephone to disclose STI status to partners

ConclusionsState - Level

• Site use sensitive to public campaigns and/or awareness–However, use wears off over time

• STI-specific analyses suggest that site is used disproportionally for scabies and crabs indicating possible abuse–However, no data to suggest (at clinical level)

that site abuse has detrimental effects

Limitations

• Before-after analysis with inherent biases• Predominantly heterosexual population • Insufficient intervention intensity and reach• Limited access to Internet services for

target population • inSPOT referrals may have preferentially

sought care at other clinics• Single-site evaluation: not necessarily

generalizable to other settings

RE-AIM in Effectiveness Trials• Reach

– Broad, heterogeneous, representative sample; often use a defined population

• Efficacy/Effectiveness– Brief, feasible interventions not requiring great expertise; adaptable to

setting; randomized time-series, or quasi-experimental designs

• Adoption/Adaptation– Appeal to and work in multiple settings; able to be adapted to fit

setting

• Implementation– Implemented by variety of different staff with competing demands,

using adapted protocol

• Maintenance– Major issues; setting-level maintenance is as important as individual-

level maintenance

Glasgow et al. American Journal of Public Health 2003;93:2161-1267

Study Rationale

19,000,000 incident STDs annually

STD clinics provide access to men and women likely to be infected and to acquire new infections over time

Yet behavioral interventions with counseling or multiple sessions are difficult to implement in busy medical settings

Recent interest in simple, easy to use, and low cost interventions for waiting rooms

Rationale continued Previous research suggests benefits of video-

based approaches, but subject to limitations:

Controlled research settings Tailored videos Single site Inclusion of group counseling

Effectiveness of stand-alone video in ‘real-world’ setting is unknown

Safe in the City Project Overview

5-year CDC-funded multi-site study

Develop a brief video-based STD clinic waiting room intervention to reduce (or eliminate) STI and risky sexual behavior

Evaluate effectiveness in 3 publicly funded STD clinics in Denver, San Francisco, and Long Beach, CA.

Denver Waiting Room

2nd TV2nd TV

San Francisco Waiting Room

Long Beach Waiting Room

Intervention Development Considerations

Waiting rooms in medical settings provide an underused opportunity to reach patients who are thinking about their health.

Yet to be feasible and sustainable, interventions must:

Be easy and inexpensive to administer

Result in minimal interruption of patient flow

Require few clinic resources, especially staff time

Be acceptable to diverse clients

Formative Process

Identification of intervention medium, theoretical framework, and key messages by research team

Collaboration with award-winning film maker to integrate framework in an appealing product

Multi-step participatory process involving target audience, clinic staff, and community advisors

Intervention research study in 3 STD clinics

Intervention Development:

Integrated Theoretical Framework

Core constructs grouped into interconnected elements

→ HIV/STD risk, knowledge, perception→ Positive attitudes toward condom use

→ Self-efficacy/skills for condom negotiation, acquisition, use→ Modeling of appropriate behaviors

Theory of Planned Behavior

Social Cognitive Theory

Information Motivation

Behavior Model

Focus Groups

3 sites held 12 focus groups with 176 participants

3 different stages of video development:

Story line development Script development Post-production editing

What Is the Intervention?

23-minute video

3 story lines

2 cartoon animations Condom variety and

selection Instructions for use

Posters in waiting and exam rooms

Story Line – Paul and Jasmine

Things are getting more serious between Paul and Jasmine, but Paul “slips” and has a sexual encounter with Teresa. Teresa

gets an STD and tells Paul. Now Paul has to tell Jasmine.

Story Line – Rubén, Tim and Christina

Rubén’s girlfriend Christina doesn’t know about his interest in men. Rubén and Tim have a casual sex encounter after

meeting in a bar. Days later, Christina suspects something is wrong.

She insists on a visit to the STD clinic.

Story Line – Teresa and Luis

Teresa has recently met Luis. After her STD scare with Paul, Teresa is serious about wanting to use condoms.

Now she has to convince Luis.

Safe in the CityEvaluating Effectiveness of the Intervention

Maximizing Intervention Delivery

Identify environmental characteristics of waiting rooms

Observe waiting room flow

Determine appropriate playback frequency

Identify factors to assure viewership (goal: 80%)

Assess and adjust to clinic staff acceptance of video

Monitor viewership and audience response

Video Viewership

01020304050

60708090

100

Intervention Cycles

% V

iew

ersh

ip

Viewership Goal

Viewership as defined by watched most or all of the video + identified a main message

Overview of Study Design Population: =~40,000 patients attending 3 STD clinics

from December 2003 – August 2005

Study design: 2 arm non-randomized controlled trial

Arm assignment: alternating 4-week control & intervention periods

Data collection: Passive review of clinic data & external surveillance records to ascertain new STI diagnoses*

* gonorrhea, chlamydia, trichomoniasis, syphilis, and HIV* gonorrhea, chlamydia, trichomoniasis, syphilis, and HIV

Overall Effect of the Intervention on Laboratory-Confirmed Infection*

Hazard Ratio (95% CI)

All patients 0.91 (0.84-0.99)

* = 9% reduction in STI incidence* = 9% reduction in STI incidence

Warner et al. PLoS Med. Jun 24 2008;5(6):e135.

Intervention Effect, by Characteristic

Hazard Ratio (95% CI)STI at index visit

Yes 0.86 (0.75-0.99)

No 0.93 (0.84-1.04)

Sex

Males 0.87 (0.78-0.96)

Females

Sexual orientation

Heterosexual

MSM

Age

<25

>25

1.06 (0.89-1.25)

0.84 (0.71-0.98)

0.90 (0.79-1.03)

1.02 (0.88, 1.17)

0.85 (0.77, 0.95)

Warner et al. PLoS Med. Jun 24 2008;5(6):e135.

Advantages of Study Design

• Evaluation in “real-world” setting

• Use of STI outcomes

• Inclusion of all clinic patients, wide generalizability

• Controlled trial

Conclusions

This reduction can have significant public health benefit, with wide adoption by clinics.

This relatively easy-to-implement, low cost intervention has the potential to reach large numbers of STD clinic patients and other high-risk populations.

Safe in the City intervention was associated with a nearly 10% reduction in new STI

Safe in the City intervention was associated with a nearly 10% reduction in new STI

www.safeinthecity.org – since launch in June 2008: >10,000 Page Visits

SITC viewed on DEBI site over 14,000 times, including over 9,000 views of online kit request form.

Safe in the City

Kit Materials DVD with video

and pdf posters User’s Guide Now Showing

poster CDC documents

Safe in the City Kit Requests (June 24, 2008 – March 25, 2009)

Agency type # of kit requests /shipped

STD Clinics 424 / 347 (82%)

Health Services Clinics 245 / 212 (87%)

Community-Based Organizations (CBOs)

242 / 0 (--)

Health Departments 252 / 153 (61%)

Family Planning Clinics 169 / 131 (78%)

Others (e.g., Hospitals, Univ. Research,

Univ. Student Health Centers) 369 / 141 (38%)

Total 1,701 / 984 (58%)

Implementation Evaluation

• Survey among:–A random sample (N=70) of STD clinics

ordering the video through August (N=216) –All STD clinics who had ordered the video and

requested technical assistance (N=58)

• Clinics surveyed between 3 and 6 months of ordering the video

• Survey response: 66% (N=85)• Average clinic population: 4,170 pt/yr

Implementation Evaluation

• Of 85 respondents:– 31 (33.7%) were actually playing the video

• 18 (34.6%) from random sample

• 13 (32.5%) from TA sample

– Time from receipt to implementation • 7 immediately

• 7 within a week

• 8 within 2 - 3 weeks

• 9 longer than 1 month

– An additional 5 clinics indicated they would start playing the video within a month

Thanks!Slide set available at:

www.stdpreventiononline.org