Vaccine Safety Datalink (VSD) Project and Monitoring of

Pandemic Influenza Vaccines

Aug. 21, 2008Pandemic Influenza Vaccine: Doses

Administered and Safety Training Conference

Eric Weintraub, MPHJames Baggs, PhD

CDC/OCSO/ISO

Goals of Talk

Background of the Vaccine Safety Datalink (VSD) Project

Background on VSD’s Rapid Cycle Analysis (RCA) Project

Plans for monitoring safety of pandemic influenza vaccines within the VSD

Vaccine Safety Datalink (VSD): Background

Established in 1990 A collaborative project among CDC and 8 managed

care organizations (MCOs) Allows for planned immunization safety studies as well

as timely investigations arising from– hypotheses from medical literature and pre-licensure – reports to the

Vaccine Adverse Event Reporting System (VAERS), – changes in immunization schedules, or the

introduction of new vaccines.

VSD Population

Collects medical care and vaccination data on more than 8.8 million members annually (3% of the US population)

As of 12/31/2006: 2,365,978 children (<18) enrolled

– 3.2% of US population

6,450,704 adults (≥18) enrolled – 2.9% of US population

Average yearly birth cohort = 94,701

VSD Sites: 2008Group Health Cooperative

Northwest Kaiser Permanente

No. CA Kaiser Permanente

So. CA Kaiser Permanente

Kaiser Permanente Colorado

Health Partners

Marshfield ClinicHarvard Pilgrim

CDC

VSD Project

Utilizes administrative data sources from health plans– Provides medical and immunization histories on >8.8 million

people annually Data:

– Demographic and enrollment– Vaccination (vaccine type, date of vaccination)– Medical outcomes (outpatient, inpatient, ER, Procedures)– Birth data– Geocoding and census– Pregnancy (developing)

The VSD Distributed Data Model

CDC

Hub

“Direct”

“Indirect”

SAS Programs,

Logs, Output, & Analytical Datasets

VSD History of Success

France EK, Safety of the trivalent inactivated influenza vaccine among children: a population-based study.* Archives of Pediatrics & Adolescent Medicine 2004;158(11):1031–1036.

Hambidge SJ,. Safety of trivalent inactivated influenza vaccine in children 6 to 23 months old. Journal of the American Medical Association 2006;296(16):1990–1997.

VSD Rapid Cycle Analysis

A new approach to surveillance that takes advantage of VSD’s strengths

Alternative to traditional post-licensure vaccine safety study methods, which generally take years to complete

VSD now updates data on all vaccines and all outcomes every week

We conduct updated analyses every week

Basics of VSD Rapid Cycle Analysis

Choose specific outcomes to monitor Each week, evaluate the number of events in vaccinated

persons Compare it to the expected number of events based on a

comparison group– Historical, concurrent

Adjust statistically for multiple looks

Sequential Analysis Methods Used in RCA

Each week, our analysis includes data from all previous weeks

Problem: Repeated testing of the same data increases the chance of false-positive results

Need to adjust for this statistically

Solution: Two types of methods used:

1. Poisson MaxSPRT• Uses a fixed rate as a comparison

2. Flexible Exact Sequential Analysis• Uses concurrent comparison group • Allows for matching for the entire population

VSD and Monitoring Pandemic Influenza for Vaccine Safety

1. Developing capabilities:• Incorporating data from state and local registries to health plan

vaccine registries

2. Identification of cohorts:• Pre-event:

• Individuals most likely to be vaccinated (first responders and health care workers)

• Event:• Individuals who received vaccine

3. Analytical Methods and Data Collection:• Establish alternative methods to conduct ad-hoc case control

methodologies• Phone, web, other survey methods

• Develop chart review instruments for potential vaccine-related outcomes

• Calculate background rates for potential vaccine-related outcomes• Develop modifications to existing seasonal influenza RCA study

Option A: Vaccine distributed outside MCOs– Estimate background rates for potential adverse events– Use alternative data collection methods to identify MCO

members who receive vaccines– Monitor rates of identified adverse events in its Dynamic

Data Files.– Conduct ad-hoc case control studies– Conduct other ad-hoc studies (e.g. Diary card type studies)

Option B: Vaccine distributed within MCOs– Conduct RCA on pandemic influenza vaccine– Monitor uptake and compliance of two-dose vaccine series

VSD and Pandemic Influenza Vaccine

VSD Rapid Cycle Analysis for Seasonal Influenza Vaccine

Simulated real-time monitoring of adverse events after 1st dose TIV

– 2005-06 season

– 2006-07 season

Piloted during 2007-08 season and next influenza season

Monitoring 12 adverse events after TIV:

– Neurologic: GBS, seizures, meningoencephalitides, Bell’s palsy, other cranial nerve disorders, demyelinating disease, peripheral nervous system disorders, ataxia, hemorrhage stroke, ischemic stroke

– Allergic: anaphylaxis, allergic reactions

VSD Investigators At CDC and MCOsCenters For Disease Control and PreventionJulianne Gee

Kaiser Permanente of No. California (NCK),

Oakland CARoger Baxter, MDNicky Klein, MD, PhDNed Lewis

Group Health Cooperative (GHC), Seattle WALisa Jackson, MD, MPHDarren Malais

Northwest Kaiser Permanente (NWK),

Portland OR Allison Naleway, PhDJohn Mullooly, PhDKaren RiedlingerLois Drew

Harvard Pilgrim /Harv. Vanguard (HAR)

Boston MATracy Lieu, MD, MPHRichard Platt, MD, MScRichard Fox

Marshfield Clinic Rsch. Foundation (MFC)

Marshfield WIEdward Belongia, MDJames Donahue, MDJeremy McCauley

Health Partners Rsch Foundation (HPM)

Minneapolis MNJim Nordin, MDAmy Butani

Kaiser Permanente of Colorado (KPC)

Denver, COSimon Hambidge, MD, PhDJason Glanz, MS, PhDDavid McClure, PhDChristina Clarke

So. California Kaiser Permanente (SCK), CA

Los Angeles, CA Steven Jacobson, MD, PhDWansu Chen, MS

Sites include > 125 staff working on VSD

Acknowledgements

We thank the principal investigators of participating VSD sites, members of the VSD Rapid Cycle Analysis working group, and members of the VSD project for their contributions to this study.

*The findings and conclusions in this presentation are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.

Additional Slides

VSD Acronyms

Vaccine Safety Datalink VSD Distributed Data Model DDM Dynamic Data Files DDF Rapid Cycle Analysis RCA

Group Health Cooperative, WA GHC Harvard Pilgrim Health Care, MA HAR HealthPartners Research Foundation, MN HPM Marshfield Clinic, WI MFC Kaiser Permanente of Colorado, CO KPC Northern California Kaiser Permanente, CA NCK Northwest Kaiser Permanente, OR NWK Southern California Kaiser Permanente SCK

VSD: Strategic Priorities

1. Evaluate the safety of newly licensed vaccines

2. Evaluate the safety of new vaccine recommendations for existing vaccines

3. Evaluate clinical disorders following immunizations

4. Assess vaccine safety in special high risk populations

5. Develop and evaluate methodologies for vaccine safety assessment

VSD Basic Tables

YEAR 2003 2004 2005 2006

<18 Enrolled in VSD1 2,371,861 2,289,419 2,314,208 2,365,978

≥18 Enrolled in VSD1 3,174,192 3,166,829 3,211,866 6,450,704

Total Enrolled in VSD1 5,546,053 5,456,248 5,526,074 8,816,682

Birth Cohort in VSD2 97,378 91,635 95,953 93,837

US Pop Estimate3 290,796,023 293,638,158 296,507,061 299,398,484% of US Pop Enrolled VSD 1.91% 1.86% 1.86% 2.94%

≥18 US Pop Estimate3217,710,885 220,343,552 222,972,821 225,662,922

<18 US Pop Estimate3 73,085,138 73,294,606 73,534,240 73,735,562

US Births4 4,089,950 4,112,052 4,138,349 4,265,996≥18 % of US Enrolled VSD 1.46% 1.44% 1.44% 2.86%<18 % of US Enrolled VSD 3.25% 3.12% 3.15% 3.21%% US Births 2.38% 2.23% 2.32% 2.20%

3 = US Population Estimates obtained from the US Census Website

Table 7: Number Enrolled in the Vaccine Safety Datalink (VSD) Compared to the US Census Population Estimates from 2003 to 2006

5 = For Cycle2005, a new method to determine the Enrolled Cohort was created due to changes in data structures resulting in slightly different numbers for previous cycles

1 = Enrolled Cohort is defined as the number of STUDYIDs enrolled for at least one day in the current cycle year2 = Birth Cohort is defined as the number of STUDYIDs in the CONSTANT file whose BRTHDATE is within 60 days

4 = US Birth Estimates obtained from the NCHS Website, NA = Not Available

RCA Collaborators – partial list

James Baggs, CDC Roger Baxter, NCK Bob Davis, CDC Bruce Fireman, NCK Rich Fox, HAR Paul Gargiullo, CDC Julianne Gee, CDC Jason Glanz, KPC Sharon Greene, HAR Nicky Klein, NCK Margarette Kolczak, CDC

Martin Kulldorff, HAR Ned Lewis, Kaiser Renny Li, HAR Dave McClure, KPC Jennifer Nelson, GHC Rich Platt, HAR Irene Shui, HAR Eric Weintraub, CDC Katherine Yih, HAR Ruihua Yin, HAR

GHC, Group Health Cooperative; HAR, Harvard; KPC, Kaiser Permanente Colorado; NCK, Northern California Kaiser

VSD Basic Tables

Cycle2003 Cycle2004 % Increase Cycle2005 % Increase Cycle2006 % Increase

Total Persons Ever1 11,329,296 11,289,281 (0.35%) 11,915,853 5.55% 18,212,726 52.84%

Total Vaccination Doses2 49,474,572 54,266,719 9.69% 60,980,854 12.37% 78,757,819 29.15%

Total Hospitalizations3 25,495,803 28,839,731 13.12% 32,725,645 13.47% 61,154,572 86.87%

Total Births4 1,110,677 1,203,370 8.35% 1,302,780 8.26% 1,354,435 3.96%

Total Deaths5 239,401 263,412 10.03% 317,010 20.35% 647,036 104.11%

Total Records in ALL Files6 354,905,437 398,558,460 12.30% 410,480,801 2.99% 916,096,862 123.18%

1=Estimated as the number of records with an unique CONSTANT record.2=Estimated as the number of records in the VACCINE file3=Estimated as the number of records in the INPT file4=Estimated as the number of records in the birth cohort for 1991 through end of that year5=Estimated as the number of records in the MORT file6=Number of records in the eight basic tables plus the historical tables

Table 8: Fun Number Facts About the Vaccine Safety Datalink (VSD)

Constant File

Study ID

VSD Site

Birth Date

Sex

Enrollment File

Study ID

Start Date

End Date

Vaccine File

Study ID

Vaccination Date

Vaccine

Facility

Site of Introduction

Manufacturer

Vaccine Lot #

Inpatient File

Study ID

Care Date

Length of Stay

ICD-9 Code - DX

Type - Primary/Secondary

Facility

Location

Outpatient File

Study ID

Care Date

Department

ICD-9 Code - DX

Type - Primary/Secondary

Facility

Location

Rule-out DX

Birth File

Study ID

Mother ID

Facility

Gest Age

Birth Weight

Apgar1

Apgar5

Race

Birth State

Birth County

Mother's Year of Birth

Mortality File

Study ID

Death Date

Exact Date

Mort Cause of Death File

Study ID

Cause of Death - ICD9/10

Cause of Death Type

Geocode File

Study ID

Date of Address

Census Tract

County

State

Medicaid File

Study ID

Monthly Medicaid Status

VSD Data Dictionary

Setting Up A Rapid Cycle Analysis

Choose outcomes to monitor

Choose comparison method(s) – e.g., historical, concurrent

Set the upper limit for when to stop

Why We Need Early Detection Systems in

Vaccine Safety Rare adverse events may be impossible to detect in pre-

licensure studies

Reports to passive surveillance systems (e.g., the Vaccine Adverse Event Reporting System) often need rapid follow-up

Follow-up studies can take months to years using traditional approaches

Maximized Sequential Probability Ratio Testing

(maxSPRT) (Kulldorff et al., 2004)

A refinement of a classical statistical method (Wald, 1945)

Null hypothesis – No excess risk

Alternative hypothesis – Increase in risk

The test statistic is the log likelihood ratio -- depends on the observed vs. expected number of events

Historical Comparison Method

Uses incidence rates from historical data

– Advantage: Knowing the historical rate of rare events allows earlier recognition that a small number of cases is unusual

– Example: 4 cases of Guillain-Barre syndrome in vaccinees, 0 expected

– Limitation: Background rates may vary over time (secular trends)

Concurrent Comparison Method

Uses matched controls, e.g., patients making preventive visits

– Advantage: Avoids false signaling or missed signals due to secular trends

– Limitations:

Need to define an appropriate control group – not simple!

Vaccines may be adopted rapidly, leaving few controls

Example: Rotashield® vaccine and intussusception (historical analysis)

-2

0

2

4

6

8

10

12

Jan Mar May Jul Sept Nov

Log likelihood ratio

1999

Vaccine licensed Aug 9815 VAERs reports through Jul 99

Vaccinesuspended Withdrawn

Criticalvalue= 3.3

MaxSPRT analysiswould have signaledin May 1999

What Happens When a Signal Occurs?

Rapid cycle analysis methods detect signals – values above specified statistical thresholds

Not all signals represent a true increase in risk

When a signal occurs, we conduct a series of evaluations using traditional epidemiologic methods

How We Evaluate Signals – 1

1. Check data quality

2. Check whether comparison groups are defined appropriately

3. Conduct the analysis using a different control group (e.g., concurrent vs. historical) or different vaccine

How We Evaluate Signals – 2

4. Conduct a temporal scan to see if outcomes cluster during a post-vaccination time window

5. Conduct a definitive study using logistic regression analysis

6. Review charts to confirm or exclude cases as true cases

ReferencesDavis RL, Kolczak M, Lewis E, et al. Active surveillance of vaccine safety: a system to detect early signs of adverse events. Epidemiology 2005;16:336-41

Lieu TA, Kulldorff M, Davis RL, et al. Real-time vaccine safety surveillance for the early detection of adverse events. Med Care 2007;45:S89-95

Kulldorff M, Davis RL, Kolczak M, et al. A maximized sequential probability ratio test for drug and vaccine safety surveillance. Unpublished data being submitted for publication.