An agency of the European Union

Signal detection: Utilisation des bases de l’OMS et de la FDA dans le contexte de la detection des signauxNe soyez pas dupes … je vais vous donner MON point de vue

Presented by: François MAIGNENPrincipal scientific administrator (PhvRM)

Introduction & disclaimers

- Background (main objective of seminar)- Conflicts of interests & disclaimer- Apologies for the lack of French- Learning objectives:

- What are the principles of disproportionality analysis +++- Knowing your own data (size, characteristics incl. type of products, patients,

age of the database, quality and quantity of information, terminologies, etc …)

- Knowing the other databases (characteristics, strengths, limitations)

2

Disproportionality analysis: it is a question of data AND background!!!

33

Drug 1 All other medicinal products

Total

Event 1 a c

All other reaction terms

b d

Total N = a + b + c + d

c + d

a + c

a + b

Disproportionality analysis

• Refers to a particular method• However, it is crucial to keep in mind that for a given drug-event

pair the result of the DA will be different from a database to an other database

• The result of DA is valid AT A GIVEN POINT IN TIME• Benefits of quantitative methods (threefold):

• Operational: easier to screen large databases• Safety net against human error (to a certain extent): systematic way of

reviewing the information received in your database• Time benefit in terms of discovery of the signals (possibly)

4

What are the elements to consider when doing signal detection (in part. Quantitative meths)?It is fundamental to know the database on which you are actually doing the

signal detection:- Size: the benefit of using quantitative methods in small databases is

unclear- Age: New or old database (type of products, terminology used, legacy

data, data fields and data model quantity of information stored in the database, duplicates)

- Source of the data: spontaneous vs RCTs (or other sources), type of reports (serious / non-serious, HCP vs patients)

- General characteristics: type of products (vaccines, biologicals), patients, etc …

5

Why using other databases?

• Other databases = other data sources (increase the likelihood of finding new signals)

• Small databases: provide an opportunity to use an external (spontaneous reporting) database as a background to implement a DA approach

• Use of longitudinal (observational) databases (THIN, GPRD)• Limitations: in particular cost, availability of information (FOI),

data management associated to it.

6

7

8

FDA AERS

• One of the oldest SRS database in the world: 1969• Very large database: approx. 5 million reports• HCP/consumers/Pharmaceutical Companies• Post-authorisation• 60% serious reports (i.e. 40% non serious)• NO VACCINES

9

10

11

VAERS

- Established in 1990- Medium size database: approx. 200,000 reports (30,000

reports / year)- Maintained by FDA/CDC- ALMOST UNIQUE: EXCLUSIVELY VACCINES- Serious reports: 13%- Hcp/Patients – spontaneous/RCTs

12

13

WHO Vigibase

- The oldest database: established in 1968- The largest: 7 million reports- Mostly spontaneous reports incl. a small number of reports

involving vaccines- Serious reports: < 10%

14

15

EudraVigilance- The newest of these databases, limited FOI- Formally established in 1995, started in December 2001.

Implementation of mandatory electronic reporting in November 2005.- Large SRS: 2,200,000 reports- Almost exclusively serious (EU), serious unexpected (non-EU)- Contains both post-authorisation (spontaneous, observational,

registries, compassionate use) and clinical trials (RCTs) modules- New products incl. anticancer, antiretrovirals, biologicals. Over

representation of CAPs

16

Other (non spontaneous) databases …• Longitudinal databases (GPRD, THIN, EU-ADR, etc …)• Very useful for signal confirmation / signal strengthening: Recent

example (EMA): biphosphonates and risk of cardiovascular valve disease

• Can be expensive, sometimes country specific• Might “only” capture data from ambulatory patients (no

hospitalisation data)• A lot of data management, not easy to analyse• Lack of standardisation, pb of compatibility with SRS databases

(medical terminology, medicinal products)

17

1818

Use of longitudinal databases for signal detection

Use of longitudinal databases (record linkage and electronic health records – OMOP / Noren / Callreus) ~ incidence rate ratio• Same patients different time windows (A. Bate)• Hospital records of different patients (T. Callreus)

1919

Merci Andrew

2020

2121

Interpretation / Limitations

Patients prescribed a PPI are associated withacute pancreatitis in the month after thePrescription (ICdiff positive)But graph shows that these patients are generallikely to have acute pancreatitis around the timeof PPI prescribing (In agreement with confounding by indication)

2222

Record linkage

2323

2424

Interpretation / limitations

Bias associated with the hospitalisation (confirmed by later events which occurred remotely after the administration of the vaccine)Spurious / unexplained associationsRely heavily on temporal association (Post hoc ergo propter hoc)

2525

Longitudinal health records

Powerful to detect some associationsEven if reporting artefacts do not influence this method, not devoid of other biases (selection, protopathic, misclassification, etc …)Much more complicated to implement (very large datasets, confidentiality aspects, linkage of records and interoperability of databases, etc …)

2626

Future directions (?)

•Performances in a prospective setting, added value of clinical judgement needs to be quantified•Other approaches to signal detection (use all the information included in the reports, PK/PD properties of the substances)•Surveillance networks (incl. spontaneous reporting) (Sentinel initiative, ENCEPP, EU-ADR, etc …)

Summary

• Different databases: completely different characteristics• Age (1968 – 2005)• Type of products (vaccines, mixture of older and newer

products, overrepresentation of new products)• Seriousness of the reports (<10% - 90%) • Different types of databases: different types of signals,

different types of results for the DA• CRITICAL TO KEEP THESE CHARACTERISITCS IN MIND WHEN

PERFORMING THE SIGNAL DETECTION ACTIVITIES +++27

Mercifrancois.maignen@ema.europa.eu

http://uk.linkedin.com/in/francoismaignen@EMA_News

@FrancoisMaignen

@FrancoisMaignen/medicines

28

Conclusions

• Using external sources of information can be a (very) useful adjunct to the signal detection activities

• The type of database (spontaneous reporting / longitudinal database) important to define

• Know the characteristics of the database• Operational issues: cost, compatibility between databases

29

Quizz 1

Question: Je suis en train d’etudier un produit specifique dans la base de donnees de l’organisation pour laquelle je travaille (approx. 10 nouvelles observations / semaine). Je n’ai pas recu de nouveaux rapports d’effets indesirables pour ce produit depuis plus d’un an. Le PRR de ce produit pour l’effet que j’etudie calcule sur cette base de donnee est-il ?

- Reste inchange depuis la derniere annee- A diminue- A augmente- A change

30

Quizz 2

Question: La limite inferieure de l’intervalle de confiance (95%) du PRR pour un produit et une reaction (10 observations d’effets indesirables) que j’etudie est egale a 0.85. Que puis-je en conclure ?

- Je n’ai pas de signal associe a ce produit- Je n’observe pas de signal de disproportionalite- Je ne dois rien faire de plus avec cette combinaison produit-

reaction

31

Quizz 3

Question: Un collegue qui travaille pour une societe specialisee (mais pas uniquement) dans des produits oncologiques se demande pourquoi il n’observe jamais de SDR associe a des agranulocytoses dans sa base de donnees. Que puis-je lui conseiller ?

32

Quizz 4Question: Je travaille pour une societe importante specialisee dans

les produits de biotechnologies. Je n’ai qu’un total de 30 observations spontanees pour un de mes produits dans ma base de donnees (il s’agit d’une enzyme recombinante donnee dans une maladie orpheline tres rare).

- J’aurais des problemes pour utiliser le PRR pour m’aider a faire la detection des signaux pour ce produit

- Vu le nombre de notifications spontanees, ol est peu probable que je trouve des signaux importants en terme de sante publique

- Les type de produits que j’ai dans le reste de ma base de donnees n’aura aucune influence sur la detection des reactions d’immunogenicite avec le PRR.

33

Quizz 5

Question: J’ai un probleme pour detecter et distinguer les interactions medicamenteuses avec un produit dont je suis responsible en utilisant le PRR. En pratique, quelles sont les methodes qui sont a ma disposition pour essayer d’ameliorer cette detection?

34