Matching key safety questions with

appropriate algorithms for

appropriate corrective actions

Lionel Van Holle (GSK Vaccines)

DSRU 10th June 2015

Table of content

� Introduction

�The key Safety questions

�The appropriate corrective actions

�The role of algorithms screening SRDs�Disproportionality

�Time-to-onset signal detection

�Logistic regression

�Tree-based scan statistics

�The full quantitative signal detection toolkit

�Future developments needed

�Conclusion

Introduction

• Disproportionality algorithms have been used for screening spontaneous report databases (SRDs) for more than a decade.

• Are they enough or should we develop otheralgorithms?

• What for?

– Better answering the same safety question as before?

– Answering other safety question?

The Key Safety Questions

1. Does the product cause adverse reactions?

�Product-related safety issues

2. Does an ingredient of my product cause adverse reactions?

�Ingredient-related safety issues

3. Does a subset of manufactured products cause adverse reactions?

�Manufacturing-related safety issues

Safety issues Examples

Product-related Thalidomide drug for morning sickness -> birth

defects, limb malformations in ~ 10,000 children

worlwide

Ingredient-related E-ferol: an injectable preparation of alpha-tocopherol

(vitamin E) for parenteral nutrition was recalled from

the market because of unusual liver and kidney

syndromes with 38 deaths reported among treated

low birthweight infants -> syndrome most likely due

to a combination of alpha-tocopherol, polysorbates,

contaminant.

Manufacturing-related Cutter incident -> some lots of the Cutter vaccine

(polio) were not properly inactivated and contained

live polio vaccine -> 120,000 doses distributed;

40,000 developed abortive poliomyelitis; 56

paralytic; 5 deaths

The appropriate corrective actions

• ? Safety profile re-evaluation

• ? B/R re-evaluationProduct-related

safety issue

• ? Strategy of ingredientsubstitution/removal

Ingredient-relatedsafety issue

• ? Recall of manufacturing lot(s)

• ? Development of new QC/QA tests

Manufacturing-related safety issue

The role of algorithms screening SRDs

Due to high number of spontaneous reports

preventing individual medical assessment

All product-event pairs

First-pass

screening

Causality

Assessment

Algorithms that do not require

prior medical assessment

Association

Temporality

Specificity

Consistency

Biological gradient

Experimentation

Plausibility

Analogy

SAFETY SIGNALS

Disproportionality

When disproportionality is used in routine, it

compares the observed number of reports for a

given product-event to what is expected from

other/all products.

Event of interest Other (or all) events

Product of interest A B

Other (or all) products C D

��� �

�

��

�∗�

∗��

�∗�

����

���

Time-to-onset signal detection

Kolmogorov-Smirnov tests

If the distance between

Cumulative distributions is

unexpected

Van Holle L et al, Using time-to-onset for detecting safety signals in spontaneous reports of adverse events

following immunization: a proof of concept study. PDS 2012; 21: 603-610. DOI: 10.1002/pds.3226

Logistic regression

• Determines if the reporting pattern (in terms

of causality criteria/strength of evidence) of a

product-event pair is similar or not to the

reporting pattern of a positive reference set

(i.e emerging signals or listed events)

Van Holle L et al, Use of logistic regression to combine two causality criteria for signal detection in vaccine spontaneous

report data. Drug Safety (2014) 37:1047-1057.

Caster O et al, Improved statistical signal detection in pharmacovigilance by combining multiple strength-of-evidence

aspects in vigiRank. Drug safety (2014) 37: 617-628.

• Integrate more causality criteria in the first-passscreening

BUT

• Routine signal detection methods (DPA or more advanced ones: TTO, LogReg, Supervised methods) use other products as comparator and are consequentlyinappropriate for detecting ingredient-based or manufacturing-based safety issues.

• It leads to potential non-detection of these issues or worse, detection at the wrong level (product).

Role of these methods?

Relevant Action performed ?Safety profile update

• LIKELY if product related

• POSSIBLE if large fraction of lots

Ingredient substitution

• UNLIKELY if ingredient shared(Rely only on qualitative assessment or ad hoc analysis)

Lot(s) recall

• UNLIKELY if small fraction of lots (Rely only on qualitative assessment or ad hoc analysis)

Algorithms

Routine Disproportionality [+ TTO, LogReg]

Safety issue

Product-related Ingredient-related Manufacturing-related

Tree-based scan statistic

Originally used in disease surveillance: e.g.

investigating ‘death from silicosis’ (event of

interest) incidence among different occupations

or group of occupations (exposure).

Potential cuts in the tree structure

symbolizing a scanning window

investigating combinations of

occupations

• For each tree cut, the rate of the event of interest (λ)

in the window scan (G) or not (R) is calculated. Total

number of cases of interest if fixed (c).

• Null hypothesis (H0) is that the rate of the event-of-

interest is the same in the window scan than outside.

• Alternative hypothesis (H1) is that the rate of the

event-of-interest is higher in the window scan than

outside.

• A likelihood ratio can be built (H1/H0)

Kulldorf M et al, A tree-based scan statistic for database disease surveillance. Biometrics (2003) 59, 323-331.

• The cut with the maximal likelihood ratio constitutes

the test statistic

• Significance is measured through Monte Carlo

simulations

• It adjusts for multiple testing across the multiple cuts

in the tree structure.

It can be adapted from disease surveillance to adverse

reaction surveillance if we link spontaneous report data

with hierarchical data representing an exposure of

safety relevance.

Kulldorf M et al, A tree-based scan statistic for database disease surveillance. Biometrics (2003) 59, 323-331.

Has the potential for being an algorithm for detecting

manufacturing-related issues & identifying most likely

manufacturing step

Specific

to SRDs

Has the potential for being an algorithm for detecting

ingredient-related issues & identifying most likely

ingredient

Relevant Action performed ?

Safety profile update

LIKELY if product related

Ingredient substitution

LIKELY if ingredient-related

Lot(s) recall

LIKELY if manufacturing-related

Algorithms

Routine Disproportionality [+ TTO, LogReg]

Tree-based scan (linked to product dictionary)

Tree-based scan (linked to manufacturing data)

Safety issue

Product-related Ingredient-related Manufacturing-related

Full quantitative SD toolkit

Future developments needed?

• Spontaneous report database (SRD) need to be linked to external information:

– A product dictionary with a hierarchical structure allowing to see the different ingredients of the product (non-independence of the products)

– A manufacturing database with a hierarchicalstructure allowing to see the differentmanufacturing steps (non-uniformity in production)

>< DPA approach with standalone SRD, flexible enough softwares?

• Define which events to monitor for manufacturing-related or ingredient-relatedsafety issues:– All MedDRA PTs as for product-related safety? (no a

priori)

– A subselection based on biological plausibility?

– A grouping of terms?

– …

• Extend the scope of tree-based scan statistic to allow integration of other causality criteria (thannumbers) as for product-related safety issues?

• Need a zero-pass screening to determine the most likely scenario (product, manufacturing, ingredient)?

Conclusion

• Quantitative signal detection toolkit shouldcontain algorithms of signal detection able to detect different types of safety issues (product, ingredient, manufacturing)

• The tree-based scan statistic is a good candidate for filling the current gap that preventsappropriate corrective actions

• Creation of a product dictionary & a manufacturing hierarchy database will berequired.