spontaneous adverse event signaling methods: classification and

Epidemiologic ReviewsCopyright © 2001 by the Johns Hopkins University Bloomberg School of Public HealthAll rights reserved

Vol. 23, No. 2Printed in U.S.A.

Spontaneous Adverse Event Signaling Methods: Classification and Use withHealth Care Treatment Products

John A. Clark,1 Stephen L. Klincewicz,2 and Paul E. Stang34

Since the systematic recording of health care treatment-associated adverse events (AEs) was first proposed byFinney after the thalidomide tragedy (1-3), regulators, pub-lic health organizations, and manufacturers have been facedwith the difficult task of interpreting postmarketing AE sig-nals. A reexamination of the methodology supporting thisprocess is now occurring as a result of several importantdevelopments, including guidances by the InternationalConference on Harmonisation (4), directives from theEuropean Committee for Proprietary Medicinal Products (5,6), new signaling procedures for screening multiproductreporting systems (7), and an Institute of Medicine report inthe United States that documented the need for enhancedpreventive measures aimed at AEs (8). Such initiativesreflect the critical role of pharmaceutical interventions inhealth care, which includes consistent and systematic AEmonitoring followed by regular assessments of the benefit-risk balance of a given product. These recent changes alsosuggest that future public health surveillance of AEs willinvolve increased cooperation among manufacturers, regu-lators, academicians, and other interested parties and willrequire better-defined methods that are applied to globallygenerated surveillance data.

Public health surveillance can be defined as the system-atic collection, analysis, interpretation, and timely dissemi-nation of health data to assist in public planning and policy.This concept of surveillance is not a new one—it is particu-larly well-entrenched in infectious disease, cancer, and envi-ronmental/occupational epidemiology. While the role ofpublic health surveillance in the detection and monitoring ofmedication and device effects has received less attention,like its disease counterparts, it has evolved steadily over the

Received for publication November 8, 2000, and accepted forpublication September 18, 2001.

Abbreviations: AE, adverse event; FDA, Food and DrugAdministration; FPS, flank pain syndrome; PRR, proportionatereporting ratio; SRR, standardized reporting ratio; WHO, WorldHealth Organization.

1 Centocor, Inc., Malvern, PA.2 Janssen Research Foundation, Titusville, NJ.3Gait Associates, Inc., Sterling, VA.4 Adjunct—University of North Carolina at Chapel Hill School of

Public Health, Chapel Hill, NC.Reprint requests to Dr. Paul Stang, 21240 Ridge Top Circle, Suite

140, Sterling, VA 20166.

past 4 decades into a well-defined discipline. PostmarketingAE reporting systems are analogous to existing systems ofdisease notification in which a voluntary report of an illness(or a precursor condition) is forwarded by a clinician to apublic health authority. In such systems, single reports andreport series may trigger intense follow-up with case ascer-tainment, surveillance measures, and formal intervention.Similarly, AE reports associated with the use of a medicalproduct can also initiate a high level of scrutiny and fre-quently result in regulatory responses that range from noti-fication of the prescribing community to removal of theproduct from the market.

Health care product surveillance programs are based oninformation from four major sources: spontaneous report-ing, medical literature case reports or case series, humanstudies, and pharmacologic or toxicologic experiments (9,10). Of these, spontaneous reports typically contribute themost data (11, 12). The term "spontaneous" refers to "vol-untary anecdotal" reports from providers (and, in the UnitedStates, includes consumer reports) regarding individualpatients; these reports do not arise from formal studies orcase reports in the literature. The collection of spontaneousreports has become the foundation of postmarketing surveil-lance programs largely because of the high volume of infor-mation they supply, their low maintenance costs, and theirdemonstrated usefulness when supervised by experiencedevaluators (13). Despite the central role of spontaneousreporting in AE surveillance, scant attention has been paid tothe development of a classification scheme that allows forthe systematic evaluation of spontaneous reports to deter-mine whether they suggest a "signal" of a potential issue.

This review attempts to address this need by 1) clarifyingbasic definitions pertaining to AE signaling, 2) outliningfour critical steps in the generation of a spontaneous reportsignaling argument, 3) identifying three fundamental datastrategies by which spontaneous report signaling is carriedout, 4) deriving a functional classification scheme for spon-taneous report signaling methods, and 5) describing themethodological groupings that comprise the classificationscheme from the perspective of an AE surveillance programdesigner.

While a large body of literature pertaining to spontaneoussignaling is available for review, relatively few detailedexamples of spontaneous AE signaling have ever been pub-lished. Many signaling exercises arise as regulatory obliga-tions or are implemented during the course of confidential

191

Downloaded from https://academic.oup.com/epirev/article-abstract/23/2/191/510322by gueston 08 April 2018

192 Clark et al.

product monitoring by a manufacturer. Thus, spontaneoussignaling techniques may be used thousands of times in anygiven year, but rarely come to public attention.

SPONTANEOUS AE SIGNALS, SIGNALING METHODS,AND SIGNALING PROGRAMS

Rationale for spontaneous AE signaling

A large body of postthalidomide safety experience sug-gests that spontaneous reporting programs can contribute toa better understanding of the risk-benefit ratio of a productas it is actually used in medical practice (14-17). However,spontaneous report evaluations are speculative at their out-set. Consequently, prior to wide acceptance, AEs noted inspontaneous reports require demonstration of consistencyacross multiple techniques and may require confirmationfrom nonspontaneous data sources. Methods applied tospontaneous reports should not be regarded as generatingprecise estimates (the calculation of incidence rates, forexample), but, rather, as providing the constituents of a sig-naling argument suggesting a product-AE relation (18) (fig-ure 1). In the public health literature, this concept has beenreferred to as "preepidemiology" (19).

Spontaneous AE signals

The primary focus of spontaneous safety data review isthe detection of AE signals. An AE signal is defined as apotential product-AE relation that deserves further attention.This broad view encompasses earlier requirements that theAE signal identify a previously unknown or incompletelydocumented product-AE pair (20, 21). The definition of sig-naling used here (which includes developing more detailedinformation about an already known product-AE relation)is similar to that recently suggested by World Health

Organization (WHO) researchers (22). While spontaneousAE signals may identify new causal relations, spontaneousreports are not restricted to identifying causal arguments.Other potentially useful observations derived from sponta-neous sources include the identification of noncausal asso-ciations (23), the clinical spectrum of a drug-AE pair (24),the patient subtypes and medical circumstances associatedwith a product-induced adverse reaction (25), clues to themechanism of action by which product exposure leads to anadverse reaction (26), and factors associated with the initia-tion of reporting behavior (27, 28).

Spontaneous AE signaling methods

Spontaneous AE signals are generated when statistical ornonstatistical methods are applied to spontaneous AE data(29). Although the manner in which a particular drug-AEpair comes to attention (spontaneous AE signaling method)has sometimes been used synonymously with the content ofthe AE (spontaneous AE signal) (29, 30), signaling methodsare better defined as procedures that are independent of con-tent. This is important because particular spontaneous AEsignals are often generated simultaneously by more than onesignaling method (20, 31). AE signaling methods can beapplied to report/case sets of one to thousands, range inapplicability from extremely limited to nearly universal, andencompass a variety of functions from simple sorting proce-dures for creating workable subsets of reports (32, 33) tocomplex analyses of case series that eventually come to beregarded as definitive (34).

Spontaneous AE signaling programs

An AE signaling program is initiated when one or moresignaling methods are applied systematically to the sponta-

SPONTANEOUS DATA

SORTINGREPORT-BASED -<STEPS

CASE-BASED -^STEPS

a

*IDENTIFICATION

lr 4

CASE SERIESFORMATION

-4CASE SERIES

CHARACTERIZATION

Places reports into meaningfulsubsets of workable size basedon content and/or number

Uses an explicit or implicitstatistical model to identifyproduct-AE pairs of interest

Applies a case definition andcase series methods to createan argument in favor of association

Describes the characteristics andpatterns of product-AE case series,focussing on reporting, demographics,risk factors, and possible mechanismsof action

SPONTANEOUS SIGNALLING ARGUMENT

FIGURE 1. Steps in the spontaneous adverse event signaling process.

Epidemiol Rev Vol. 23, No. 2, 2001


Spontaneous Adverse Event Signaling Methods 193

neous AE reports for a particular product (35, 36). As withmany public health surveillance systems, spontaneous sig-naling programs are implemented by international agencies(e.g., the WHO), by national bodies (e.g., individual countryregulators), by regional surveillance programs (e.g.,regional pharmacovigilance units in France and Spain), andby manufacturers (e.g., manufacturer-based drug safetydepartments). In the past, such signaling programs have usu-ally been administered as standardized regimens that wereapplied uncritically to multiple drugs. This occurred despitethe recognition that AE signals must be interpreted in thecontext of single product use (3) and that AE signaling pro-grams are essentially product specific ("product" means oneor more dosage forms that are monitored together as a dis-tinct entity). In general, the inclusion of a signaling methodin a spontaneous AE signaling program should address a rel-evant epidemiologic, medical, or regulatory design principleand should be consistent with the International Conferenceon Harmonization E2A and E2C guidances (9, 10, 37).

SPONTANEOUS SIGNALING PROCESS AND DATASTRATEGIES

Spontaneous signaling arguments

It is useful to classify spontaneous report signaling meth-ods around the several broad functional steps that comprisesignaling arguments: sorting, identification, case series for-mation, and case series characterization (figure 1). The idea ofclassification by functional step is consistent with recent con-sensus documents that describe the AE monitoring of healthcare products (i.e., the Council for InternationalOrganizations of Medical Sciences IV report (17) and variousEuropean Committee for Proprietary Medicinal Productsdirectives (5, 6)). The first two steps (sorting and identifica-tion) are based on reports, while the second two (case seriesformation and characterization) are based on only thosereports that fit a case definition. Stepwise report-based sig-naling procedures followed by case-based signaling proce-dures correspond roughly to the public health concepts of"hypothesis generating" and "hypothesis strengthening."

intraproduct and qualitatively, the statistical model that hasbeen proposed is the single-report Bayesian odds model; forintraproduct and quantitative application, Poisson-based sta-tistical models have most often been suggested, and forinterproduct and quantitative application, several 2 x 2 con-tingency tables and Bayesian data-mining models have beenproposed. The majority of the published statistical researchconcerning AE signaling methodology has been directedtoward developing methods that are used at this step(38-40).

Case series formation step. Case series formation pro-cedures, which test the potential association between a prod-uct and an AE, rely upon a case definition to create a caseseries. Case definitions specify the minimum quality anddiagnostic criteria that are needed to designate reports ascases. Depending on the approach used, case definitions canrange from the acceptance of reports at face value to detailedclinical algorithms. The case-series formation step of spon-taneous report signaling evaluates at least four kinds ofinformation that are provided by a product-AE case series:1) the number of cases; 2) the aggregate diagnostic featuresof a (usually) multiple case series; 3) the degree of clinicalsimilarity exhibited by two or more cases; and 4) the pres-ence of similar experience from different sources (i.e., inde-pendence and repeatability of cases).

Case series characterization step. Once a product-AEpair has been identified, it can be characterized further togain additional insights. Case series analyses of the distri-bution of elements within a drug-AE case series have beencalled "characterizations" or "identification of risk factors"and have been well described as AE signaling methods (24,41, 42). Case series characterizations usually focus onclinical features, age, gender, dose/duration of therapy,reporting dates, or the geographic origin of reports. Suchprocedures are used to refine and expand an existing AE sig-naling argument by, for example, demonstrating a reporteddose relation, providing clues to potential explanatorymechanisms, identifying subpopulations at risk, or describ-ing reporting patterns that help distinguish biologic fromreporting phenomena.

Functional steps in spontaneous signaling

The sorting step. Sorting procedures are data manage-ment screens that create report sets of regulatory and/or epi-demiologic interest. Historically, sorting methods have beenconsidered a form of signaling (13), although the results arealmost always reexamined through the application of addi-tional techniques. Sorting is probably the most commonkind of signaling procedure used in health care product sur-veillance. The careful use of sorting techniques in product-specific surveillance programs is generally believed to beimportant in enhancing their effectiveness (10).

The identification step. Identification procedures applyexplicit or implicit statistical models to one or more sponta-neous reports to find product-AE pairs of interest. Like sort-ing, the identification step is carried out using reports, notcases, and therefore produces findings that are usually fur-ther refined using case-based methods. When applied

Spontaneous signaling data strategies

The development of signaling arguments can be accom-plished by using at least three different data strategies:intraproduct qualitative methods, intraproduct quantitativemethods, and interproduct quantitative methods. Qualitativemethods assess one or more content elements of a particularreport or case series, while quantitative methods apply pro-cedures involving numbers, proportions, or rates to groupsof reports or cases. Both qualitative and quantitative methodsare ultimately concerned with identifying and describingpotentially associative (but not necessarily causal) relations,but do so by using distinct logical frameworks (43).

Quantitative data strategies can be further subdivided intointraproduct and interproduct procedures. Intraproductmethods generate signals by examining spontaneous reportsin the context of the monitored (index) product and its userpopulation, while interproduct methods work by comparing



194 Clark etal.

spontaneous reporting for the index product with sponta-neous reporting for other products (13). Unlike intraproductmethods, which can be either qualitative or quantitative, theinterproduct methods that have been proposed to date are allquantitative. Intra- versus interproduct logic is a fundamen-tal feature of spontaneous report signaling method designand has been a factor throughout the history of its develop-ment (24).

Classification and review of spontaneous reportsignaling methods

In this article, spontaneous report signaling methods areclassified first according to signaling step and then accord-ing to data strategy (table 1). This sequence produces 11groupings of spontaneous report signaling methods (thereare no published sorting methods that use interproduct quan-titative comparisons). In the remainder of this article, eachof these 11 methodological groups are described in detail inthe same order in which they appear in table 1, i.e., bydescending row from left to right.

SPONTANEOUS SORTING METHODS

Qualitative sorting (key content) methods

Qualitative sorting by key content selects subsets ofreports for further examination that contain information ele-ments of interest from a regulatory, medical, or report-qual-ity perspective (44). Perhaps the most important qualitativesorting methodology is the well-known "alert" (or 15-day)

report, which depends on the AE meeting regulatory defini-tions for the "seriousness" and "expectedness" (whether theevent is mentioned in the package insert) (36). Key contentmethods often focus as well on the chronologic evolution ofclinical events (e.g., positive dechallenge or rechallenge),important outcomes (32, 36) (e.g., fatal, life-threatening),previously unreported AE types (35, 45), or AE types ofhigh interest (32,44), such as those included in the Food andDrug Administration's (FDA) partial listing of designatedmedical events (46). Qualitative sorting allows an evaluatorto reduce a large reporting universe into smaller, moremeaningful parcels, each of which can then be considered indetail.

Quantitative sorting methods

Subjective judgment. The first published quantitativesorting method was based on a subjective impression ofexcessive report numbers (the "pigeon hole" signal ofNapke (47)). Although a formal evaluation of subjectivequantitative signaling has not been published, the same lim-itations seen with subjective imputation of report content,i.e., increased intra- and interobserver variability, would beexpected to apply (see imputation screening methodsbelow).

Cutoff criteria. Cutoff signaling methods refer to pro-cedures that detect an arbitrary number of reports of aproduct-AE pair. While unrefined, cutoff criteria have longbeen regarded as useful mechanisms for directingresources (22, 29, 32, 35, 44). These simple approachesmay be the only reasonable quantitative strategy for drug-

TABLE 1. Classification of spontaneous report signaling methods by functional step and data strategy

Functional stepsData strategies

Qualitative intraproduct Quantitative intraproduct Quantitative interproduct

Sorting

Identification

Case series formation

Case seriescharacterization

Qualitative sorting methodsKey content methods

Quantitative sorting methodsSubjective judgmentCutoff criteria

Imputation screening assessments Intraproduct quantitativeSubjective judgmentRule-based methods

(standardized assessmentmethods)

Bayesian probability calculations

Intraproduct qualitative case seriesformation methods

Subjective case seriesRule-based case seriesRetrospective Bayesian analysis

identification methodsSerial (increased frequency)

identification methodsTemporospatial cluster

identification methods

Intraproduct quantitative case seriesformation methods

Standardized reporting ratioAssociative case-control study

Interproduct identification methods2 x 2 table count methodsBayesian data-mining methods

Interproduct case series formationmethods

Spontaneous cohort designSpontaneous case-control

design

Intraproduct qualitative case series Intraproduct quantitative case series Interproduct case seriescharacterization methods

Descriptive case distributioncharacterization methods

Numerator-only case distributionRate/proportion-based case

distributionCharacterization case-control

study

characterization methodsComparative case distribution




specific AE surveillance programs that accumulate fewreports, since, in this setting, more elaborate proceduresare often not feasible.

Use of sorting signaling methods

These methods rely on applying particular criteria to AEreports that segregate them into more meaningful groups.The expedited (serious + unexpected) report-sorting proce-dure and several time period-specific line listings are man-dated by regulation. Sorting methods based on designatedmedical events (mentioned above) also appear to be appro-priate for many spontaneous report data sets, given the reg-ular appearance of a relatively short list of AE typesthroughout the history of health care product monitoring.For products that give rise to small numbers of reports, oneor two sorting methods alone (i.e., with no identificationmethods) will typically be sufficient tools for screeningaccumulated spontaneous AE reports.

SPONTANEOUS IDENTIFICATION METHODS

Imputation screening (causality) assessments

Imputation screening assessments use single-report eval-uations to identify a signal (i.e., they implement the signal-ing process by finding a "good case"). When used as aspontaneous report signaling tool, imputation is not so mucha causality assessment measure as it is a hypothesis genera-tion tool for locating "interesting" product-AE pairs for fur-ther examination (20, 48, 49). Three important issues havebeen raised about the applicability of imputation screeningmethods to spontaneous reports: 1) the effect of missing,unobtainable, or erroneous data elements (50-52); 2) theidentification of a single product-AE pair out of a list ofmultiple potential causes (50); and 3) the value of imputa-tion in generating signals of previously undescribed prod-uct-AE pairs (49, 53).

The first issue, data integrity, is especially pertinent tospontaneous report-by-report assessment because sponta-neously forwarded information is often incomplete or inac-curate (54-56). Additionally, many reports are not usefulsources because the product-AE pair is not amenable to clas-sical imputation procedures (e.g., dechallenge evaluation inthe presence of a nonresolving AE) (49). The second issue,the difficulty of causal attribution in the presence of multiplepotential background contributors, is frequently encounteredin individual patient reports, including those arising fromspontaneous report sources (50, 57). In general, imputationmethods tend to have less value as the complexity of the dis-ease/treatment background increases. The third issue, initialdiscovery value, arises because causality assessments alwaysdepend to some degree on expectations shaped by priorexperience. However, it has been well documented thatimportant new spontaneous report signals can originate fromsettings where prior experience is limited or misleading,thereby resulting in a delay in signal recognition (58).

Imputation screening methods have undergone extensiverefinement over the past 4 decades (40, 43, 59), borrowingheavily from advancements in public health. Starting from

pure subjectivity, they evolved into formalized proceduresbased on rules and then into more specific probabilistic cal-culations derived from Bayes' theorem. More complexapproaches, such as decision support algorithms, have alsobeen suggested (60); however, essentially no experienceexists in which they have been used to systematically screenspontaneous report databases for product-AE signals.

Subjective judgment. In the simplest form of imputol-ogy, subjective assessment, or "global introspection" (57),an evaluator assigns a causal rating to an individual sponta-neous AE report based on his or her own medical diagnosticexperience (61). Subjective assessments have usuallyinvolved classification into multiple categories, such as thedesignations "documented," "probable," "possible," and"doubtful" proposed in the 1960s and still used in modifiedform in AE evaluations today (62, 63). Unfortunately, sub-jectively generated causality assessments are associatedwith high levels of intra- and interrater variability (64-68)and produce results that can differ substantially from impu-tation methods based on more explicit methods (69). Theimprecision of global introspection stems in large part fromthe multifactorial nature of AE causality, which makesreproducible evaluations difficult to carry out in the absenceof a formal assessment procedure (57). In spite of its well-documented limitations, global introspection is probably themost widely used method for evaluating spontaneous reportcausality (44, 50, 70).

Rule-based methods. With rule-based methods (alsocalled standardized assessment methods or standardizeddecision aids (71)), evaluation of an individual AE report isscored by using a questionnaire or an algorithm (13, 40, 70,72). Over the last 2 decades, a large number of such instru-ments have been published (68, 73-75). All rule-basedmethods contain three basic components that have beendesigned by an expert: 1) a set of structured responses; 2) aweighting algorithm that translates the responses into ascore; and 3) an algorithm that equates scoring ranges withimputation categories (probable, possible, etc.). Evidencehas been published that rule-based methods can reduceintra- and interrater variability compared with subjectivejudgment, thereby addressing one of the objections to theuse of global introspection (68, 76, 77).

Concerns have been raised about the proper role of rule-based methods based on both applied and theoretical con-siderations (59, 78). In particular, the scoring of product-AEtest sets by several raters can differ widely, even when thesame rule-based procedure is being used (79). This residualinterrater variability appears to be attributable to questionsthat require more subjective input on the part of the user(80). One approach to this issue has been to decrease sub-jectivity by developing algorithms that are specific to par-ticular AEs (81, 82). Such AE-specific, rule-based methods(20) are designed by a consensus of expert clinicians,thereby increasing response specificity, but at the cost ofmarkedly restricting the applicability of the procedure to anarrow range of safety outcomes (82). To our knowledge, noliterature has been published in which rule-based methodswith this kind of narrow scope have been systematicallyused in product-specific AE surveillance programs.



196 Clark et al.

Unlike results obtained in optimal settings, the use ofrule-based methods in actual surveillance environments maynot significantly improve interrater variability beyond thatobtained with global introspection (83-86). This may relateto the inability of rule-based methods to discriminate reli-ably between proximate imputation categories (e.g., proba-ble vs. possible) (53), suggesting that categorical adjacencyis an important limit for these instruments. In addition, whenapplied to the same product-AE test set, the use of differentrule-based methods can create widely divergent scores (87,88), largely because of the use of different weightingassumptions for the various causality criteria (89). The mostprominent theoretical objection to rule-based methods hasbeen the impossibility of validation or intermethod transla-tion of assessment scores derived from different instru-ments, since a "gold standard" cannot be defined for thesepurposes (72). This observation also underscores the inap-propriateness of the word "standardized" in this context,since such instruments do not result in "standard" output,but, instead, represent the results of similar, but indepen-dently designed, expert systems.

Bayesian probability calculations. The most individual-ized formal imputation method devised thus far for evaluatingsingle-report content, Bayesian AE analysis, was introducedinto the medical literature by Lane (90) and Auriche (91)(figure 2). In the odds form of the Bayesian expression, theprior odds (the best initial estimate among exposed patientsof the ratio of probabilities favoring product causality vs.noncausality) is multiplied by one or more likelihood ratios.Likelihood ratios are formed by dividing the probability fora risk factor or test result (usually a chronologic test) underthe condition of product causality by the probability for thesame risk factor/test result under the condition of productnoncausality. The product of the prior odds and all pertinentlikelihood ratios results in a posterior odds for productversus nonproduct causality that incorporates all known rel-evant public health and report information. Thus, for a spon-taneous report, the posterior odds can be interpreted as thebest estimate for the odds in favor of product causality, tak-ing into account the limitations of external data sources,report content, and report quality (40, 90-95).

While the Bayesian odds model has the advantage of asound basis in probability theory, it has features that couldrestrict its application to spontaneous report data sets. First,Bayesian assessments are time intensive, requiring a largenumber of assumptions and calculations (70). However,Naranjo and Lanctot (59) have pointed out that the Bayesianodds model, once populated with reference data, is straight-forward to implement in subsequent drug problems involv-ing the same AE type. Second, the results of Bayesian case

P(D—•AE|B ,C) P(D—•AE|B) P(C|D—•AE)_ x

P(Dyv AE|B,O

FIGURE 2. Bayesian imputation. P, probability; D-»AE, productcaused AE; D^AE, product did not cause AE; |,given that; B, back-ground conditions; C, case conditions.

report evaluations have been compared with laboratory evi-dence (lymphocyte toxicity assay) in patients who experi-enced dermatologic and/or hypersensitivity reactions. In thiscross-comparison, The Bayesian odds assessment appearedto be characterized by a high sensitivity (i.e., high true-positive rate), but an unknown specificity (i.e., true-negativerate) for detecting allergic persons (96). Third, and perhapsmost important, the Bayesian odds model is individualizedaccording to each situation in which it is applied and, there-fore, has limited usefulness as a screening device. The find-ing that Bayesian evaluations exhibit relatively lowcorrelations to rule-based methods (97) is a reflection of thisattribute, since Bayesian methodology does not use aweighting algorithm, but instead relies upon report-specificepidemiologic estimates and case data modeling that arestrongly influenced by empirical observation.

Intraproduct quantitative identification methods

Intraproduct quantitative identification methods (serial(increased frequency) methods and temporospatial clusteridentification methods) enumerate groups of reports of simi-lar content for the same product over time or time-space inorder to identify an AE signal. The statistical models that havebeen used for serial identification methods have often beenbased on Poisson or various nonparametric models, whiletemporospatial cluster identification methods use an array ofestablished cluster procedures to identify localized increasesin the reporting rate of a product-AE pair. It has long beenunderstood that spontaneous report quantitative signalingmethods, including those based on statistical theory, are toolsfor generating "suspicions" (i.e., interesting product-AEpairs) and are not exercises in hypothesis testing (32, 98, 99).

Serial methods have been discussed extensively in thedrug safety literature. Past experience indicates that, whenthese methods are applied routinely to spontaneous reportdata, knowledge of their limitations is important. A goodexample of this occurred in the United States, where a reg-ulatory requirement for serial signaling was eventually with-drawn as a result of high "false alarm" rates (100, 101). Incontrast to serial methods, procedures that focus on tem-porospatial cluster identification have received little atten-tion from the AE signaling community, although there areboth theoretical and practical bases for the application ofthese techniques to product-specific surveillance programs(102-105). AE clustering methodology takes essentially thesame approach as that commonly used by public healthagencies in the investigation of both product- and non-prod-uct-induced disease outbreaks (102).

Serial identification methods. Spontaneous report serialmethods monitor the number, proportion, or rate of a prod-uct-AE pair over time (32, 35, 38, 39, 41, 45, 105-108)(table 2). The first such method (Patwary signaling (38, 105,108)) was based on a two-period comparison (current vs.historical) of the proportion of index-product-attributed AEreports to all-product-attributed AE reports for a particularAE type. This "AE-specific, proportion of all products"strategy was subsequently changed to the "product-specific,proportion of all AEs" strategy, which is commonly used




today. The same statistical procedures can be used to carryout either calculation.

Serial signaling procedures were originally derived froma reporting rate comparison between current and extensivehistorical time periods (unequal time period methods) (38),whereas subsequent methods emphasized comparisonbetween two successive equal time periods (equal timeperiod methods) (109-112). Other unequal time periodapproaches that have been suggested involve comparingcurrent with all (i.e., current + past, rather than just past) his-torical experience (113). Methods have also been publishedin which a comparison is made over contiguous time inter-vals (trend methods) (38, 113-116), including the checkingof ongoing experience vis-a-vis an externally specified stan-dard (cumulative sum methods) (105). The latter two typesof procedures are aimed at ascertaining incrementalincreases over time and cumulative upward divergence froma preset level, respectively. Two acknowledged methods fortwo equal-period serial testing are the conditional binomialmethod of Norwood and Sampson (111) and the normalapproximation for a difference between two proportionsdescribed in the US FDA 1991 reporting guidelines (112).Tsong (39) published an evaluation of six equal-periodincreased frequency methods using simulated false-alarmrates that showed the FDA's 1991 method to be the optimalprocedure of those that were investigated, while Hillson etal. (117) suggested a method in which serial comparisonsare adjusted for variations in the lag time between the datesof occurrence and reporting.

Spontaneous report serial methods that assume a classicalstatistical distribution are limited by the presence of geo-graphic clustering (104, 105). Specifically, when widely dis-tributed products are monitored by using a Poisson-based test,geographic clustering of the reports can be shown to provideevidence that the Poisson assumption is incorrect. Thus, in thepresence of geoclustering, an evaluator should considerreplacing a Poisson-based (or similar) serial test with a pro-cedure that identifies high probability clusters. Other method-ological approaches include region-based serial testing, inwhich numerator counts are made of reporters and/or institu-tions instead of reports (104), and the use of specializedprobability distributions for reports (105, 108, 113, 116). Thelatter strategies take geoclustering into account either by elim-inating it from the calculation or by incorporating its effectsinto the assumed probability distribution.

Temporospatial cluster identification methods. Nowhereis the influence of public health investigation more apparentthan in the assessment of temporospatial clustering. Insteadof accounting for clustering within a serial signaling model,an evaluator may design a method that screens product-specific spontaneous AE reports for the presence of tem-porospatial clusters. A number of product-AE "outbreak"'investigations have been published that came to the atten-tion of public health agencies as a result of initial presenta-tions that were clustered over time and space (118-123).Temporospatial clustering has been seen with lot-associatedand other product defects (119, 120, 122, 123), with local-ized patterns of use/misuse (121), and with events laterdetermined to occur commonly under permissive conditions

(118). AE clustering methodology remains the principalapproach used in pharmacoepidemiology to identify AEsdue to manufacturing or product defects and is evolving asa method for identifying reporting biases in spontaneousreporting.

Interproduct quantitative identification methods

Although first described by WHO researchers in the1970s, only recently have interproduct identification meth-ods become the subject of intense discussion (7, 33,124-126). Such procedures evaluate differences in the pro-portions of events found in multiple product reporting sys-tems to identify suspected product-AE relations. Like otheridentification methods, interproduct identification compar-isons yield product-AE pairs that are then further evaluatedthrough the use of more precise techniques (22, 124, 126).Since an increase in the number of products in the computa-tional universe is associated logically with both improve-ment in estimates for expected reporting behavior and areduction in sensitivity to underreporting, interproduct iden-tification techniques appear to be most useful when appliedto large AE reporting systems, such as those found at theregional, national, or international levels.

2 x 2 table methods. In the early 1970s, Finney (38)described an interproduct identification method (Patwarysignaling) in which the proportion of all index-productreports containing a specific AE type was compared with thesame proportion derived from a multiple product database.They referred to this strategy as "reaction proportion signal-ing" and applied it to the WHO spontaneous reporting sys-tem. In 1994, Amery (33, 41) published a similar procedurecalled the relative adverse drug experience profile, whichwas applied to a multidrug report database at the manufac-turer's level. Amery also suggested a related technique inwhich the proportion of all index product reports containingtwo or more AE types is compared with the same proportioncalculated from the underlying database, a concept aimed atscreening for product-induced syndromes (33, 41). Withthese methods, signal identification occurs when standardstatistical tests indicate that the product-AE to product-all-AE proportion exceeds an expected value derived from thereporting system database, under the null assumption ofproduct-AE independence (33, 41).

The Medicines Control Agency has described a procedureof this type called the proportionate reporting ratio (PRR)method. Like the two previous procedures, the PRR portionof this calculation is based on a comparison of the propor-tion of index product reports that contain a specific AE typewith the same proportion from a multiproduct universe (33,125). The PRR is then defined as the reporting odds calcu-lated from the resulting 2 x 2 table (33, 125). However,unlike the above methods, the PRR methodology selectssuspect product-AE pairs by applying combination criteriaderived from general experience (33, 125), such as a PRRgreater than three plus a chi-square score of greater thanfive, in the presence of at least three product-AE reports (127).

Bayesian data-mining methods. Recently, two Bayesiandata-mining procedures have been published that extend inter-



198 Clark et al.

TABLE 2. Spontaneous serial

Time horizon and year

Two periodEqual

N/A*

methods

Report distribution Test method

Numerical increase

Reference

N/A

Comments

Sometimes referred to as safety "shifttables-

Assumes uniform productusage/reporting

1985

1985

1988

1991

1992

Poisson

Poisson

Poisson

Poisson

Doubling

Normal approximation

Conditional binomial

Normal approximation

Normal approximation withYate's correction

1992

1992

1998

Poisson Normal approximation (log-transformed data)

Poisson Normal approximation (squareroot-transformed data)

Poisson Chi-squared withcontinuity correction

109t 1985 FDA* "arithmetic" method

39, 109f, 111 1985 FDA "Poisson" methodHigh false-alarm rate

39, 111t Exact method, rare event assumption

39, 111, 112t Derived by Norwood and Sampson1991 FDA methodBest false-alarm rate of those tested

39t Variant of normal approximationmethod

39f Variant of normal approximationmethod

39t Variant of normal approximationmethod

High false-alarm rate

113t Also referred to as "pairwisecomparisons"

Proposed for medical device reportsAssumes uniform product

usage/reportingIntended for comparisons when

periods are short

1998

Unequal1969

1976

1998

Poisson Normal approximation with lag-time adjustment

Poisson:): t test

Poisson:): Score standardized to past

117t Adjusts for lag time between eventoccurrence and reporting

38t, 105, 108 Patwary methodComparison of current with past

report proportions

105, 108, 114t Mandel method

Negativebinomial

experience compared withan arbitrary threshold(M-statistic)

Negative binomial exact test

Comparison of current with pastmean report numbers (orproportions)

Specified threshold based onseverity of AE*

113t, 116 Proposed for medical device reportsDetects a cluster of reports during a

specified periodAssumes uniform product

usage/reportingComparison of current with all report

proportionsTable continues

product quantitative identification to MxN tables (7, 22, 33,124,126), where M refers to a large set of AEs, and N refers to

a large set of monitored products (table 3). The first of these,the Bayesian confidence propagation neural network method,



TABLE 2.

Time

Continued

horizon and year Report distribution Test method

Spontaneous Adverse

Reference

Event Signaling Methods

Comments

199

2000 Binomial-Poisson Binomial-Poisson exact test

2000 Zero-truncatedPoisson

Zero-truncated Poisson exacttest

Trend1974

1974

1976

1976

Poisson^:

PoissonJ

Chi-squared

Exact probability for specifiednumber of increases

Linear trend threshold methodof Mandel

Center-batch matrix for trend

1977

1978

1998

2000

Beta-binomial

Exact (or, if appropriate,asymptotic) test based ondistributions derived from acenter-batch matrix

Modified one-sided numericalcumulative sum test

Cox-Stuart nonparametric

Graphic smoothing techniques

116t Proposed for medical device reportsDetects a cluster of reports during a



proportions

116t Proposed for medical device reportsDetects a cluster of reports during a



proportions

38t Comparison of current with >2 pastproportions

38t Nonparametric comparison of currentwith >2 past proportions

114f Assumes uniform productusage/reporting

114t Comparison of geographicallystratified report means orproportions over sequentialbatches

Uses geographic dispersion of datasources to reduce false positives

115t Intended for finding "latent" signals(i.e., detectable centrally but notby peripheral contributors)

Uses geographically dispersed trenddata

Assumes uniform productusage/reporting

105f, 108 Devised by the World HealthOrganization

Specified threshold based onseverity of AE

113t, 116 Proposed for medical device reportsDetects a gradual trend in reportsProper interpretation is based on

knowledge of product usage

1161" Proposed for medical device reportsUse graphic smoothing to visually

present reporting trendsAssumes uniform product

usage/reporting

* N/A, not applicable; FDA, Food and Drug Administration; AE, adverse event.t Indicates primary footnote reference for that method.% This method is consistent with the assumption of a Poisson probability distribution for reporting when event probability is low.

was proposed by Bate et al. (7, 124) from the WHO, the sec-ond was developed by DuMouchel (126) with reference to the

FDA spontaneous reporting system. A stepwise approach tothe development of Bayesian data-mining techniques, as



200

TABLE

Clark et al.

3. Comparison of two Bayesian data mining procedures used in spontaneous

Bayesian confidence propagationneural network method (9)

report signaling

DuMouchelmethod (126)

Product-AE* stratification (tables) Nonstratified

Null model Product-AE independence

Stratified by (at least) gender and year of report

Product-AE independence

Cell-level measurement statistic

Prior Distribution for product-AEpairs (specified for a time point)

Final output

IC*= log2(P [x,y\IP [y\), where P{x) and P(y) PR(xa > xe), where xa is the actual cell count andare the report probabilities for the product and xo is the expected cell countAE whose association is under study PR(xa > xe) is assumed to derive from the

Poisson distribution

Distribution derived empiricallyBased mathematically on a two gamma mixture

distribution

Ranking of product-AE pairs by degree of"interest"

P(x). P(y). and P(x,tf are assumed to derivefrom beta distributions

Single distribution is assumedCan be approximated by the exponential

distribution

Product-AE IC statistics that are sufficiently"interesting"

Time scanning (the IC values are plotted overtime)

* AE, adverse event; IC, information component.

described by Louis and Shen (128), involves 1) the construc-tion of a large M x N table, where cells contain counts of thefirst through Afth AE for the first through Mh product, 2) thecreation of a null model that calculates expected counts foreach cell, 3) derivation of a statistic that measures deviationbeyond the expected value in each cell, and 4) quantificationof the relation between observed and expected in each cell toassist in selecting or evaluating product-AE pairs.

The procedure of Bate et al. (7, 24) has been carried outfor specific product-AE pairs at multiple time points,thereby incorporating interval changes into a time scan. Thisrepeated comparison technique has been proposed as anenhancement of signal detection, since an upward change inthe measurement statistic over time implies that increasing"awareness" of a product-AE pair has developed within thereporting system as time progresses. In contrast,DuMouchel's method uses time stratification in the overallmodel to adjust for secular reporting trends. Its aim is to useall accumulated evidence in the reporting system to rankorder suspect product-AE pairs. Both the methods of Bate etal. and those of DuMouchel explicitly recognize thatBayesian data mining is an identification process only and isnot meaningful unless subsequent examination takes place.

Use of identification signaling methods

The use of automated identification signaling methods isindicated for larger and more complex report databases. Indesigning identification procedures, it is helpful to concep-tualize this step as the use of screening tests to find interest-ing product-AE pairs for further study. The use of effectiveidentification procedures for product monitoring resemblesthe use of screening tests for other public health applicationsand commands special attention to false-positive rates.Experience suggests that, despite past difficulties, both rule-

based imputation screening and serial signaling methods canbe effective in spontaneous AE surveillance, but that therationale for both of these identification methodologies mustbe scrutinized carefully. Screening strategies based on theBayesian odds model are unlikely to be helpful on a wide-spread basis because they are too complex and individual-ized to be effective as screening tests. Although not usedsystematically in the past, screening for temporogeographicclusters may be effective for certain types of safety out-comes, especially those that are related to product prepara-tion and local circumstances of use. At present, interproductidentification methods appear to be promising developmentsfor large, publicly administered databases, but are probablynot feasible for the relatively small and highly selectivedatabases maintained by an individual manufacturer.

SPONTANEOUS CASE SERIES FORMATION METHODS

Intraproduct qualitative case series formation methods

Spontaneous report qualitative association methods canbe defined as the application of an imputation method to oneor more cases to demonstrate that sufficient evidence existsto generate a signal. The concept of case-based signaling,coming after and based on report-based signaling, is inher-ent in much of the spontaneous report methods literature anddates at least to the early writings of Finney (129). In recentyears, this distinction has also been emphasized byMeyboom et al. (20), who has stressed the differencebetween the use of imputation to identify "interesting" AEsversus its use to form case series for the intensive review ofsuspected product-AE pairs.

Like all case series methods, qualitative case series for-mation relies on an evaluator's ability to devise a case defi-nition that is defensible from a medical and public health




perspective (13, 130, 131). Although acceptance of sponta-neous reports at face value may sometimes be reasonable,the importance of more specific case definitions is under-lined by the relatively low rate of case validation at the timeof follow-up observed when spontaneous reports are care-fully scrutinized (54). Qualitative assessments of case seriesallow an evaluator to condense a range of imputation resultsinto a descriptive impression, much as an experienced clin-ician would do when considering multiple cases of a dis-ease. Similar to imputation methods, qualitative associationmethods can be classified as subjective, rule-based, orBayesian.

Subjective case series. With subjective case series, anevaluator includes cases without formally specifying a casedefinition. As a result, the development of such case seriesis strongly influenced by an evaluator's individual medicaland epidemiologic judgment. As with individual reportimputation, subjective assessments of case series are proba-bly the most common way in which associative evidence forproduct-AE relations is summarized.

Rule-based case series. Rule-based case series, can becreated by specifying a minimum case requirement or arange of diagnostic criteria that must be met by the caseseries as a whole. In recent years, several consensus confer-ences have produced well-accepted definitions for certainproduct-induced AE types, the results of which can be usedto construct rule-based, AE-specific case definitions (20).AE-specific, rule-based case series are generated by apply-ing these case definitions to candidate reports. AE-specificcase definitions have typically concentrated on the surveil-lance of AEs known to be frequent complications of healthcare product use, such as drug-induced liver injury or hema-tologic cytopenias (81, 132).

Generalized rule-based methods for constructing caseseries include the monitored adverse reaction and a qualitycriteria grading methodology proposed by Edwards et al.(56), Jones (133), and Turner (134). The monitored adversereaction method allows case series to be created from therange of case diagnostic criteria (as determined by theFDA's rule-based imputation method) that is found in aproduct-AE series of specified length. With the FDA's pub-lished rule-based method, a probable case means that posi-tive dechallenge information is available; a possible casemeans that a "reasonable" temporal sequence is present, anda remote case means that a temporal relation is evident (133,134). A monitored adverse reaction is then said to exist if ahigh-quality case series with at least one of the followingattributes is noted: 1) at least one probable case; 2) at leasttwo possible cases; 3) at least one possible and five remotecases; or 4) at least 10 remote cases (135).

The quality criteria grading system proposed by Edwardset al. (56) of the WHO defines cases in terms of quality cri-teria and key content. An index case means that either posi-tive rechallenge information is available or the case exhibitsno confounding variables, a substantial case means that 11key variables are available for that case, and a feasible casemeans that six key variables are available for that case. Apublishable signal is then said to exist if any combination ofat least three index case equivalents are present, in which

one index case, two substantial cases, or four feasible casesconstitute an equivalent.

Retrospective Bayesian analysis. A spontaneous caseseries approach based on Bayesian methodology has alsobeen described and is referred to as retrospective Bayesiananalysis (136). In the retrospective Bayesian method, a sin-gle summary of the prior odds for the case series as a wholeis calculated by using information provided by the seriesand is assumed to apply to all cases. After examination ofthe included cases, each individual case is then reassessed inlight of the information from all cases, and a calculation ofthe posterior odds for the entire case series is obtained.

Intraproduct quantitative case series formation methods

Intraproduct quantitative case series formation methodscompare reported case numbers with an expected value.With the standardized reporting ratio (SRR), this step isaccomplished by comparing an observed number ofreported cases with the number of cases expected based ona background incidence schedule. With the associative case-control study, this step is accomplished by using cases andcontrols that are based on spontaneous reporting.

SRR. In SRR analyses, the number of spontaneouslyreported cases of a product-AE pair (or the case-reportingrate) is compared with the expected number of backgroundcases for the same product-AE pair (or the background inci-dence rate) to see whether the actual number of reportedcases exceeds the number of cases expected on the basis ofchance (131, 137-139). The SRR is therefore analogous toa standardized incidence ratio, except that the numerator isformed from reported, not incident, cases. The number ofexpected cases is calculated by applying an appropriatebackground rate schedule to the person-time of the patientgroup using the product. While background rates are logi-cally derived from a source of data in which cases are fullyascertained, it can also be modified to take into account thesupposition that background cases, like reported cases, willalso be underreported (140). With the latter modification,the number of reported cases is compared with the numberof background cases that would have been expected to bothoccur by chance and be reported.

The methods used in carrying out SRR calculations canbe classified either as spontaneous rate ratios or as grossnumerical comparisons. With spontaneous rate ratios, thenumber of reported cases is compared with the expectednumber of incident cases by using a Poisson model (131,140), while with gross numerical comparisons, authors pre-fer to interpret the reported and expected case counts (orrates) nonstatistically (138, 139, 141). Methods have alsobeen described in which one or more unknown componentsof the Poisson model are parameterized (142). SRRs rely ona case definition for the AE that is applicable to both spon-taneously reported cases (the numerator) and an externaldata source (the denominator) (140). If such a case defini-tion is not feasible or if a usable external data source is notavailable, the method cannot be used. Additionally, ifexpected reports are made subject to an underreporting fac-tor (which increases the utility of the procedure), a sensitiv-



202 Clark et al.

ity analysis is recommended to see how much the underre-porting assumption affects the results (140). The SRR offersan important advantage over interproduct case series com-parisons (see "Interproduct case series formation methods"below), since any underreporting assumptions made by anevaluator must be explicit. If an evaluator chooses to use afully ascertained background rate, false positives due toclustering and underreporting are unlikely explanations forany resulting signals (131) because the comparison standardis both "completely reported and completely clustered."

Associative case-control study. Intraproduct associativecase-control strategies have also been described in generalterms, the aim of which is to support or refute product-AEassociations (143). To date, very little experience has accu-mulated using spontaneously based case-control designs.

Interproduct case series formation methods

Interproduct case series formation methods evaluate thespontaneous rate or proportion of index product-AE casesvis-a-vis the analogous spontaneous rate or proportion for acomparator product, usually over a single comparable timeframe. Such interproduct methods seek to provide an under-standing of the relative association between product and AEby benchmarking against an "other product" standard. Likeother case series formation techniques, interproduct associa-tive methods assume that reports have been validated to theextent necessary to carry out the analysis (i.e., they have meta case definition). In general, the methods used in two-groupcomparisons of this type can be classified as spontaneouscohort design methods, spontaneous case-control designmethods, and gross numerical comparisons.

In the spontaneous cohort design, the distribution ofreports between two products is compared with a proba-bilistic expectation based on some surrogate of exposure(i.e., unit sales, prescription number, market share, defineddaily doses, or estimated exposed patients (31, 144-148)).This model is conceptually equivalent to the two-groupPoisson model that is well described in standard texts (149).If exact testing is performed, it has usually been carried outusing a conditional binomial procedure (150). A sponta-neous case-control design has also been described in whicha 2 x 2 contingency table is populated by using AE and all-other-AE case counts for the index and other products (41,151-154). A version of this has been referred to as the"case/noncase design" or "ADR reporting odds ratio"(151-153), with quantitative evaluation carried out using astandard odds ratio statistic. Some authors have preferred tointerpret differences between the reporting rates/proportionsof specific AEs to different drugs nonstatistically, i.e., byusing gross numerical comparison (154-158). While com-parative analyses have typically been carried out over com-parable equal-length segments of the marketing cycle,Tsong (159) proposed the use of a Mantel-Haenszel statisticthat allows a single summary comparison over multipleanalogous time periods of two marketing cycles.

In addition to well-described confounders such as age,gender, intended use, duration of use, and concomitantmedications (24, 160), a number of other factors have

been suggested that could affect the interpretability ofspontaneous comparative signals, including the year(s) ofthe marketing cycle to be analyzed (155, 156, 160), secu-lar trends in reporting (155, 156, 160), publicity (160),and the effects of product promotion (160). Begaud andMoride (13), Tuber-Bitter et al. (150), and Begard et al.(161) have emphasized the conservative interpretation ofthe reporting rates for two compared products in properlyevaluating such signals, Sachs and Bortnichak (160, 162)and Lawson (163) have noted the effect that reportingbiases (such as confounding by indication) may haveplayed in misinterpreting spontaneous comparisons of therate of gastrointestinal bleeding in patients who receivethe nonsteroidal antiinflammatory drug piroxicam versusother similar antiinflammatory drugs, and Stang and Fox(164) have emphasized the importance of taking con-founding by indication into account when interpretingspontaneously based analyses.

Use of case series formation signaling methods

When compiling and comparing spontaneous product-AE case series, it is important for evaluators to specifycase definitions carefully and to adapt them to the limita-tions of spontaneous report data. In contrasting such expe-rience with expectation, the standardized reporting ratioand its modifications are particularly valuable. However,this methodological approach requires extensive knowl-edge of background or expected incidence rates that shouldbe confirmed through review of the medical literatureand/or database research. Interproduct comparisons ofspontaneously based case series should be undertaken withcare. Because of the many biases affecting interpretation,these methods are probably best used as a last resort whenbetter data sources are not available in an appropriate timeframe.

SPONTANEOUS CASE SERIES CHARACTERIZATIONMETHODS

Intraproduct qualitative case series characterizationmethods

Summaries of product-AE case series based on sponta-neous report data have long been well accepted as a charac-terization methodology (24). Case-based descriptive statis-tics provide a general sense of the information contained inthe case series as a unit, generally focusing on age, gender,dose, and clinical presentation. Analyses of report contentdistribution for a given drug-AE pair can also provideimportant clues concerning patient subtypes at high risk andthe underlying mechanism of the event and are an importantsource for postmarketing data that may eventually beincluded in the package insert (23). However, authors suchas Rawlins (24) have suggested that proposing a correlationbetween product exposure and a characterized factor is inap-propriate unless exposure for that factor is available (see"Intraproduct quantitative case series characterization meth-ods" below).




Intraproduct quantitative case series characterizationmethods

"Numerator only" case series distribution. Within thereported case series, different clinical outcomes may beassociated with different age groups, gender, dosage, orconcomitant medications (25,41,145). A classic example ofthe sometimes substantial impact of such numerator-only(internal correlative) characterizations was published byInman and Mushin (165) in 1974. In this instance, a caseseries of halothane-hepatitis demonstrated a decreasingtime-to-onset with increasing number of exposures, therebysupporting an associative argument and providing a poten-tial explanatory mechanism (allergy) by which the eventcould occur.

A specialized form of numerator-only case series distribu-tion occurs when clustering methods are used to examineinformation contained in the reported case distribution of anAE type over time and/or space (166, 167). Pure temporalclustering has been associated with publicity-induced report-ing effects (166), while the AE reported case distribution byregion or country within the same time frame (i.e., tests forpure spatial clustering) can also suggest localized reporting orproduct-related phenomena (145). Temporospatial clusteringhas already been addressed as a report-based identificationmethod, but could also become apparent subsequently as partof a product-AE characterization.

Rate/proportion-based case series distribution. If suffi-ciently detailed product use data are available, signalingbased on selected content characteristics can be enhanced bycomparisons with analogous population rates or proportions(24, 168). This results in a more formalized signal statisticthat resembles rate comparison of interproduct case seriesbut is confined to subset comparisons for a single product-AE pair (41, 144). Although this approach could be affectedby the same sensitivity to differential reporting rates and dif-ferential clustering that affects the serial identification andinterproduct case formation methods, the assumption ofsimilar within-product reporting dynamics is probably agood one in many instances because the factors that have thestrongest impact on the propensity to report would often notappear to be altered substantially by the characteristicsunder study. (For example, under the usual circumstances ofuse, the reporting dynamic for a death attributable to an AEis not likely to be altered by its therapeutic dose or the ageof the patient.) Rate/proportion-based case series distribu-tions will have limited usefulness if an evaluator has reasonto believe that the reporting rate for a product-AE pair bycharacteristic is related to that characteristic itself.

Characterization case-control study. In addition to theassociative case-control design described above, sponta-neously reported cases and reporter-selected exposed con-trols can be used to characterize risk factors (169). Thisdesign is spontaneous report in origin (an evaluator obtainsdata for both cases and noncases from self-selectedreporters) and requires that both cases and noncases receivethe product. Between-group comparisons then focus onpotential risk factors for the development of the syndrome.An excellent example of a characterization spontaneouscase-control study, aimed at determining risk factors for the

suprofen flank pain syndrome, was published by Strom et al.(169).

Interproduct case series characterization methods

Comparative case series distribution. Quantitative caseseries characterization methods have rarely been extendedto comparative characterizations of two or more products.Such characterizations have typically focused on thereported outcome (seriousness) profile of a particular prod-uct-AE pair in cases receiving the index versus anotherproduct(s) (170).

Use of case series characterization signaling methods

A review of the literature pertaining to case series charac-terization methods indicates that such data can be helpful toprescribers as long as its limitations are kept in mind. In par-ticular, the possibility of a relation between the character-ized feature and the safety outcome rate or other endpointshould always be formally considered. In presenting suchdata, it should also be remembered that the characteristics ofthe reported case series may be different from similar popu-lation-based case series.

AN EXAMPLE: SUPROFEN FLANK PAIN SYNDROME

Suprofen is a nonsteroidal anti-inflammatory drug thatwas first marketed in Europe in 1982, and in the US inJanuary 1986. In February 1986, the first two of over 300reports of acute renal colic-like pain (later called suprofenflank pain syndrome (FPS) (166, 169) were received by theUS Food and Drug Administration's Spontaneous ReportingSystem. In addition to pain, some patients developed hema-turia and acute renal insufficiency. By March 1986, supro-fen FPS had been recognized in the United States as a sig-nal based on a subjective impression of report number, awidely used sorting method. Although not published, theapplication of imputation screening assessments at the iden-tification step of signaling would also have been effective infinding suprofen FPS, since both dechallenge and rechal-lenge positive reports of FPS were forwarded. In contrast, aserial identification (increased frequency) identificationmethod signaled this relation only after widespread public-ity and appeared to be less effective as an early warning pro-cedure. Case series formation took place quickly after signaldetection and focused on a subjective discussion of theobserved versus expected number of cases (i.e., an SRR-likeprocedure). This comparison was interpreted as evidence infavor of a relation based on the small number of expectedFPS cases in the target population, as well as the close tem-poral relation of the cases to exposure, which furtherreduced the expected case estimate. Subsequently, caseseries characterization was addressed through a spontaneouscase-control study that used suprofen FPS cases reportedthrough August 1986. This study provided evidence thatmale sex, the use of muscle-building exercise equipment,physically active lifestyle, recent sun exposure, residence inthe Sunbelt, and alcohol use were associated with suprofen



204 Clark et al.

FPS. These findings supported a proposed mechanism inwhich patients exposed to a potent uricosuric agent (supro-fen) could develop acute urinary crystallization, especiallyin the presence of dehydration and large muscle mass (i.e.,substantial endogenous uric acid burden). The manufacturerof suprofen subsequently withdrew the product from world-wide marketing. Suprofen FPS illustrates how a signalingargument for a product-AE develops through an orderlysequence (sorting, identification, case series formation, caseseries characterization) from an AE surveillance environ-ment and also demonstrates the rapidity with which suchinformation can be accumulated, analyzed, interpreted, andplaced into a public health context.

SUMMARY

AE signal detection and its techniques are part of the con-tinuum of public health surveillance, borrowing from bothits theory and application (171). Like public health surveil-lance networks, whose major goals are to identify early signsof new outbreaks, pinpoint new organisms, and monitor des-ignated illnesses, AE signaling and surveillance systemsattempt to provide early warnings of previously unsuspectedproduct-AE pairs, hypothesize potential drug-event rela-tions, identify populations "at risk," and facilitate caseascertainment and definition. In both examples, definitiveresearch is often subsequently undertaken to quantify thestrength of relations that may be proposed.

As with any public health surveillance effort, AE surveil-lance provides an infrastructure for the ongoing collectionof health data and its direct integration into the health regu-latory policy (172), including its keystone role in riskassessment and management. However, unlike many sur-veillance systems, postmarketing AE systems collect caseinformation that is often relatively incomplete and imper-fect, estimate exposure based on surrogate values (e.g., salesdata), and are used by both governmental and the privatesector for preventive planning. These factors make AE sig-naling and surveillance more ambiguous, regulatory ori-ented, and complex than its disease counterparts (173).Despite such issues, AE signaling methods continue toevolve in extent, complexity, and acceptance (4, 131, 174).Undoubtedly, this is largely due to the widespread practicalexperience that has been gained with spontaneous reportingsystems over the past 4 decades and the cumulative useful-ness that has been demonstrated.

REFERENCES

1. Finney DJ. An international drug safety program. J NewDrugs 1963;3:262-5.

2. Finney DJ. An international drug safeguard plan. J ChronicDis 1964; 17:565-81.

3. Finney DJ. The design and logic of a monitor of drug use. Aninternational drug safety guard plan. J Chronic Dis 1965;18:77-98.

4. Klincewicz SL, Clark JA, Arnold BDCA, et al. Globalizationof the safety sections of the periodic safety update report.Drug Inf J 1999;33:887-98.

5. European Agency for the Evaluation of Medicinal Products,Committee for Proprietary Medicinal Products. Conduct ofpharmacovigilance for centrally authorised products.London, England: Committee for Proprietary MedicinalProducts, 1997. (publication no. CPMP/PhVWP/183/97).

6. European Agency for the Evaluation of Medicinal Products,Committee for Proprietary Medicinal Products. Notice to mar-keting authorization holders: pharmacovigilance guidelines.London, England: Committee for Proprietary MedicinalProducts, 1999. (publication no. CPMP/PhVWP/108/99).

7. Bate A, Lindquist M, Edwards IR, et al. A Bayesian neuralnetwork method for adverse drug reaction signal generation.Eur J Clin Pharmacol 1998;54:315-21.

8. Institute Of Medicine, Committee on Quality of Health Carein America. To err is human, building a safer health system.Washington, DC: National Academy Press, 1999.

9. Food and Drug Administration. International conference onHarmonisation: guideline on clinical safety data manage-ment: definitions and standards for expedited reporting. FedRegist 1995,60:11284-7.

10. Food and Drug Administration. International conference onHarmonisation: guideline on clinical safety data manage-ment: periodic safety update reports for marketed drugs. FedRegist 1997;62:27470-6.

11. Faich GA, Knapp D, Dreis M, et al. National adverse drugreaction surveillance: 1985. JAMA 1987;257:2068-70.

12. Faich GA. U.S. adverse drug reaction surveillance1989-1994. Pharmacoepidemiol Drug Safe 1996;5:393-8.

13. Begaud B, Moride Y, Haramburu F. Quality improvementand statistical calculations made on spontaneous reports.Drug Inf J 1994;28:1187-95.

14. Carson JL, Strom BL, Maislin G. Screening for unknowneffects of newly marketed drugs. In: Strom BL, ed. Pharma-coepidemiology. New York, NY: John Wiley & Sons, 1994:432^7.

15. Abenheim L, Moore N, Begaud B. The role of pharmacoepi-demiology in pharmacovigilance: a conference at the 6thESOP meeting, Budapest, Hungary, September 28, 1998.Pharmacoepidemiol Drug Safe 1999;8(suppl):Sl-7.

16. Faich GA. Adverse-drug-reaction monitoring. N Engl J Med1986;314:1589-92.

17. Council for International Organizations of Medical Sciences.Benefit-risk balance for marketed drugs: evaluating safetysignals. Geneva, Switzerland: Council for InternationalOrganizations of Medical Sciences, 1998.

18. Goldman SA. Limitations and strengths of spontaneousreports data. Clin Ther 1998;20(suppl C):C40-4.

19. Wartenberg D, Greenberg M. Solving the cluster puzzle: cluesto follow and pitfalls to avoid. Stat Med 1993;12:1763-70.

20. Meyboom RH, Hekster YA, Egberts AC, et al. Causal orcasual? The role of causality assessment in pharmacovigi-lance. Drug Safe 1997;17:374-89.

21. Edwards IR, Biriell C. Harmonisation in pharmacovigilance.Drug Safety 1994;10:93-102.

22. Lindquist M, Edwards R, Bate A, et al. From association toalert—a revised approach to international signal analysis.Pharmacoepidemiol Drug Safe 1999;8(suppl):S 15-25.

23. Wechsler ME, Garpestad E, Flier SR, et al. Pulmonaryinfiltrates, eosinophilia, and cardiomyopathy following corti-costeroid withdrawal in patients with asthma receiving zafir-lukast. JAMA 1998;279:455-7.

24. Rawlins MD. Spontaneous reporting of adverse drug reac-tions. II: Uses. Br J Clin Pharmacol 1988;26:7-11.

25. Bateman DN, Rawlins MD, Simpson JM. Extrapyramidalreactions to prochlorperazine and haloperidol in the UnitedKingdom. Q J Med 1986,59:549-56.

26. Hart D, Ward M, Lifschitz MD. Suprofen-related nephrotox-icity. Ann Intern Med 1987; 106:235-8.

27. Moride Y, Haramburu F, Requejo AA, et al. Under-reportingof adverse drug reactions in general practice. Br J ClinPharmcol 1997;43:177-81.

28. Tubert-Bitter P, Haramburu F, Begaud B, et al. Spontaneousreporting of adverse drug reactions; who reports and what?




Pharmacoepidemiol Drug Safe 1998;7:323-9.29. Venulet J. Adverse reactions to drugs. Int J Clin Pharmacol

1973;7:253-64.30. Venning GR. Identification of adverse reactions to new

drugs. I. What have been the important adverse reactionssince thalidomide? Br Med J 1983,286:199-202.

31. Inman WHW. Role of drug-reaction monitoring in the investi-gation of thrombosis and "the pill." Br Med Bull 1970;26:248-56.

32. Finney DJ. Statistical aspects of monitoring for danger indrug therapy. Methods Inform Med 1971;10:l-8.

33. Amery WK. Signal generation from spontaneous adverse eventreports. Pharmacoepidemiol Drug Safe 1999;8:147-50.

34. Blum MD, Graham DJ, McCloskey CA. Temafloxacin syn-drome: review of 95 cases. Clin Infect Dis 1994; 18:946-50.

35. Royall BW. International aspects of the study of adversereactions to drugs. Biometrics 1971;27:689-98.

36. Venulet J. Possible strategies for early recognition of poten-tial drug safety problems. Adv Drug React Ac Pois Rev 1988;1:39-42.

37. International Society For Pharmacoepidemiology. Guidelinesfor good epidemiology practices for drug, device, and vaccineresearch in the United States. Pharmacoepidemiology 1996;5:333-8.

38. Finney DJ. Systematic signalling of adverse reactions todrugs. Methods Inf Med 1974;13:1-10.

39. Tsong Y. False alarm rates of statistical methods used indetermining increased frequency of reports on adverse drugreaction. J Biopharmacol Stat 1992;2:9-30.

40. Lane DA, Kramer MS, Hutchinson TA, et al. The causalityassessment of adverse drug reactions using a Bayesianapproach. Pharm Med 1987;2:265-83.

41. Amery W. Analysis of the information in a central ADE data-base. Int J Risk Safety Med 1994;5:105-23.

42. Rawlins MD. Spontaneous reporting of adverse drug reac-tions. Q J Med 1986;59:531^.

43. Kramer MS, Lane DA. Causal propositions in clinicalresearch and practice. J Clin Epidemiol 1992;45:639^9.

44. Hartmann K, Ciorciaro C, Kuhn M. Signal generation in a non-EU country. Pharmacoepidemiol Drug Safe 1997;6(suppl 3):S13-19.

45. Royall BW, Venulet J. Methodology for international drugmonitoring. Methods Inf Med 1972; 11:75-86.

46. Food and Drug Administration, Center for Drug Evaluationand Research. The four levels of AERS. Rockville, MD,1998. (http://www.fda.gov./cder/aers/4_level.htm).

47. Napke E. Drug adverse reaction alerting program. Can PharmJ 1968;20:251^1.

48. Venulet J. Role and place of causality assessment.Pharmacoepidemiol Drug Safe 1992; 1:225-34.

49. Stephens MDB. Causality assessment and signal recognition.In: Stephens MDB, Talbot JCC, Routledge PA, eds.Detection of new adverse drug reactions. London, England:Macmillan Reference Ltd, 1999:298.

50. Meyboom RHB, Royer RJ. Causality classification at phar-macovigilance centres in the European Community.Pharmacoepidemiol Drug Safe 1992;l:87-97.

51. Irey NS. Adverse drug reactions and death: a review of 827cases. JAMA 1976;236;575-8.

52. Venulet J. Incomplete information as a limiting factor incausality assessment of adverse drug reactions and its practi-cal consequences. Drug Inf J 1986;20:423.

53. Meyboom RHB. Causality assessment revisited. Pharma-coepidemiol Drug Safe 1998;7:S63-5.

54. Funch D, Dreyer NA, Bassin LG, et al. The validity of casereports in a pharmacoepidemiologic study. Post MarketingSurveill 1993;7:283-9.

55. Sicker T, Hoffmann A, Hoffmann H. Individual case assess-ment of venous thromboembolic events suspected to be asso-ciated with combined oral contraceptives. Experience ofreevaluating individual case histories on Valette.Pharmacoepidemiol Drug Safe 1998;7(suppl):S 109-10.

56. Edwards IR, Lindquist M, Wiholm BE, et al. Quality criteria

for early signals of possible drug reactions. Lancet 199O;336:156-8.

57. Kramer MS. Assessing causality of adverse drug reactions:global introspection and its limitations. Drug Inf J 1986;20:433-7.

58. Inman B. 30 years in postmarketing surveillance. A personalperspective. Pharmacoepidemiol Drug Safe 1993;2:239-58.

59. Naranjo CA, Lanctot KL. Bayesian evaluation of sequentialcases of neutropenia. Pharmacoepidemiol Drug Safe 1993;2:Sl-6.

60. Hoskins RE, Manning S. Causality assessment of adversedrug reactions using decision support and informatics tools.Pharmacoepidemiol Drug Safe 1992; 1:235^*9.

61. MacDonald MG, MacKay BR. Adverse drug reactions: expe-rience of Mary Fletcher Hospital during 1962. JAMA 1964;190:1071-4.

62. Cluff LE, Thornton GF, Seidl LG. Studies on the epidemiol-ogy of adverse drug reactions. I. Methods of surveillance.JAMA1964;188:976-83.

63. Seidl LG, Thornton GF, Cluff LE. Epidemiological studies ofadverse drug reactions. Am J Public Health 1965;55:1170-5.

64. Koch-Weser J, Greenblatt DJ. The ambiguity of adverse drugreactions. (Abstract). Clin Pharmacol Ther 1976; 19:110.

65. Karch FE, Smith CL, Kerzner B, et al. Adverse drug reactions:a matter of opinion. Clin Pharmacol Ther 1976; 19:489-92.

66. Koch-Weser J, Sellers EM, Zacest R. The ambiguity ofadverse drug reactions. Eur J Clin Pharmacol 1977;ll;75-8.

67. Blanc S, Levenberger P, Berger JP, et al. Judgments oftrained observers on adverse drug reactions. Clin PharmacolTher 1979;25:493-8.

68. Naranjo CA, Busto U, Sellers EM, et al. A method for esti-mating the probability of adverse drug reactions. ClinPharmacol Ther 1981;3O:239^5.

69. Miremont G, Haramburu F, Begaud B, et al. Adverse drugreactions: physicians' opinions versus a causality assessmentmethod. Eur J Clin Pharmacol 1994;46:285-9.

70. Jones JK. Determining causation from case reports. In:Strom BL, ed. Pharmacoepidemiology. New York, NY: JohnWiley & Sons, 1994:365-78.

71. Naranjo CA. A clinical pharmacologic perspective on thedetection and assessment of adverse drug reactions. Drug InfJ 1986;20:387-93.

72. Hutchinson TA, Lane DA. Assessing methods for causalityassessment of suspected adverse drug reactions. J ClinEpidemiol 1989;42:5-16.

73. Irey NS. Tissue reactions to drugs. Am J Pathol 1976;82:616^7.

74. Karch FE, Lasagna L. Toward the operational identificationof adverse drug reactions. Clin Pharmacol Ther 1977;21:247-54.

75. Kramer MS, Leventhal JM, Hutchinson TA, et al. An algo-rithm for the operational assessment of adverse drug reac-tions. I. Background, description, and instructions for use.JAMA 1979;242:623-32.

76. Busto U, Naranjo CA, Sellers, EM. Comparison of tworecently published algorithms for assessing the probability ofadverse drug reactions. Br J Clin Pharmacol 1982;13:223-7.

77. Hutchinson TA, Leventhal JM, Kramer MB, et al. An algo-rithm for the operational assessment of adverse drug reac-tions. II. Demonstration of reproducibility and validity.JAMA 1979;242:633-8.

78. Hutchinson TA. Standardized assessment methods foradverse drug reactions: a review of previous approaches andtheir problems. Drug Inf J 1986;20:439^t4.

79. Case B, Oszko MA. Use of an algorithm to evaluate pub-lished reports of adverse drug reactions. Am J HospPharmacol 1991;48:121-2.

80. Hutchinson TA, Flegel KM, HoPingKong H, et al. Reasonsfor disagreement in the standardized assessment of suspectedadverse drug reactions. Clin Pharmacol Ther 1983;34:421-6.

81. Benichou C. Criteria of drug-induced liver disorders. Report ofan international consensus meeting. J Hepatol 1990; 11:272—6.

82. Danan G, Benichou C. Causality assessment of adverse reac-



206 Clark et al.

tions to drugs. I. A novel method based on the conclusions ofinternational consensus meetings: application to drug-induced liver injuries. J Clin Epidemiol 1993;46:1323-30.

83. Louik C, Lacouture PG, Mitchell AA, et al. A study ofadverse reaction algorithms in a drug surveillance program.Clin Pharmacol Ther 1985;38:183-7.

84. Schmidt LG, Dirshedl P, Grohmann R, et al. Consistency ofassessment of adverse drug reactions in psychiatric hospitals:a comparison of an algorithmic and an empirical approach.Eur J Clin Pharmacol 1986;30:199-204.

85. Leventhal JM, Hutchinson TA, Kramer MS, et al. An algo-rithm for the operational assessment of adverse drug reac-tions. III. Results of tests among clinicians. JAMA 1979;242:1991-4.

86. Grohmann R, Dirschedl P, Scherer J, et al. Reliability ofadverse drug reaction assessment in psychiatric inpatients.Eur Arch Psychiatry Neurol Sci 1985;235:158-63.

87. Pere JC, Begaud B, Haramburu F, et al. Computerized com-parison of six adverse drug reaction assessment procedures.Clin Pharmacol Ther 1986;40:451-61.

88. Michel DJ, Knodel LC. Comparison of three algorithms usedto evaluate adverse drug reactions. Am J Hosp Pharmacol1986;43:1709-14.

89. Frick PA, Cohen LG, Rovers JP. Algorithms used in adversedrug event reports: a comparative study. Ann Pharmacother1997;31:164-7.

90. Lane DA. A probabilist's view of causality assessment. DrugInf J 1984;18:323-30.

91. Auriche M. Approache baye'sienne de l'imputabilite desphenomenes indesirables aux medicaments. Therapie 1985;40:301-6.

92. Lane DA. The logic of uncertainty: measuring degree ofbelief. Drug Inf J 1986;20:445-53.

93. Lane DA. The Bayesian approach to causality assessment: anintroduction. Drug Inf J 1986;20:455-61.

94. Kramer MS. A Bayesian approach to assessment of adversedrug reactions: evaluation of a case of fatal anaphylaxis.Drug Inf J 1986;20:505-17.

95. Naranjo CA, Lanctot KL, Lane DA. The Bayesian differen-tial diagnosis of neutropenia associated with antiarrhythmicagents. J Clin Pharmacol 199O;30:112O-7.

96. Naranjo CA, Kwok MCO, Lanctot KL, et al. Enhanced dif-ferential diagnosis of anticonvulsant hypersensitivity reac-tions by an integrated Bayesian and biochemical approach.Clin Pharmacol Ther 1994;56:564~75.

97. Lanctot KL, Naranjo CA. Comparison of the Bayesianapproach and a simple algorithm for assessment of adversedrug events. Clin Pharmacol Ther 1995;58:692-8.

98. Finney DJ. Statistical logic in the monitoring of reactions totherapeutic drugs. Methods Inf Med 1971;1O:237^5.

99. Finney DJ. The detection of adverse reactions to therapeuticdrugs. StatMed 1982;1:153-61.

100. Food and Drug Administration. Postmarketing expeditedadverse experience reporting for human drug and licensedbiological products: increased frequency reports. Fed Regist1996,61:55603-7.

101. Food and Drug Administration: Postmarketing expeditedadverse experience reporting for human drug and licensedbiological products: increased frequency reports. Fed Regist1997;62:34166-8.

102. Jacquez GM, Waller LA, Grimson R, et al. The analysis ofdisease clusters. Part I: State of the art. Infect Control HospEpidemiol 1996; 17:319-27.

103. Jacquez GM, Grimson R, Waller LA, et al. The analysis ofdisease clusters. Part II: Introduction to techniques. InfectControl Hosp Epidemiol 1996;17:385-97.

104. Clark JA, Berk RH, Klincewicz SL. Calculation of the prob-ability of multiplicities in two cell-occupancy models: impli-cations for spontaneous reporting systems. Drug Inf J1999,33:1195-1203.

105. Moussa MAA. Statistical problems in monitoring adversedrug reactions. Methods Inf Med 1978;17:106-12.

106. Praus M, Schindel F, Fescharek R, et al. Alert systems for

post-marketing surveillance of adverse drug reactions. StatMed 1993; 12:2383-93.

107. Lydick E, Blumenthal SJ, Guess HA. Learning to see: twentyyears of renal adverse experience reporting with Indocin. JClin Res Pharmacoepidemiol 1990;4:183-9.

108. Lao CS. Application of CUSUM technique and beta-bino-mial model in monitoring adverse drug reactions. J BiopharmStat 1997;7:227-39.

109. Food and Drug Administration. Center for Drugs andBiologies: guideline for reporting adverse drug reactions(ADR's). Rockville, MD: Center for Drug Evalation andResearch, 1986:11-7.

110. Clark J. Increased frequency reporting. Rockville, MD:Center for Drug Evalation and Research, 1988:4C.

111. Norwood PK, Sampson AR. A statistical methodology forpostmarketing surveillance of adverse drug reaction reports.Stat Med 1988;7:1023-30.

112. Food and Drug Administration. Center for Drug Evaluationand Research (CDER). Guideline for postmarketing report-ing of adverse drug experiences. Rockville, MD: Center forDrug Evalation and Research, 1992:3^:24-8.

113. Lao CS, Kessler LG, Gross TP. Proposed statistical methodsfor signal detection of adverse medical device events. DrugInf J 1998;32:183-91.

114. Mandel SPH, Levine A, Beleno GE. Signalling increases inreporting in international monitoring of adverse reactions totherapeutic drugs. Methods Inf Med 1976; 15:1-10.

115. Levine A, Mandel SPH, Santamaria A. Pattern signalling inhealth information monitoring systems. Methods Inf Med1977;16:138-44.

116. Lao CS. Statistical issues involved in medical device post-marketing surveillance. Drug Inf J 2000;34:483-93.

117. Hillson EM, Reeves JH, McMillan CA. A statistical sig-nalling model for use in surveillance of adverse drug reactiondata. J Appl Stat 1998;25:23-40.

118. Centers for Disease Control. Eosinophilia-myalgia syn-drome—New Mexico. MMWR Morb Mortal Wkly Rep1989;38:765-7.

119. Centers for Disease Control. Unusual syndrome with fatali-ties among premature infants: association with a new intra-venous vitamin E product. MMWR Morb Mortal Wkly Rep1984;33:198-9.

120. Jolson HM, Bosco L, Bufton MG, et al. Clustering of adversedrug events: analysis of risk factors for cerebellar toxicity withhigh-dose cytarabine. J Natl Cancer Inst 1992;84:500-5.

121. Bennett SN, McNeil MM, Bland LA, et al. Postoperativeinfections traced to contamination of an intravenous anes-thetic, propofol. N Engl J Med 1995;333:147-54.

122. Martone WJ, Williams WW, Mortensen ML, et al. Illness withfatalities in premature infants: association with an intravenousvitamin E preparation, E-Ferol. Pediatrics 1986;78:591-600.

123. Centers for Disease Control. Deaths associated with thi-amine-deficient total parenteral nutrition. MMWR MorbMortal Wkly Rep 1989;38:43-6.

124. Bate A, Lindquist M, Ore R, et al. Identifying and quantify-ing signals automatically (Abstract). PharmacoepidemiolDrug Safe 1998;7(suppl 2):99.

125. Davis S, Evans S, Wark S, et al. Signal generation: proceduresfor identification, evaluation and tracking. PharmacoepidemiolDrug Safe 1998;7(suppl 2):S79.

126. DuMouchel W. Bayesian data mining in large frequencytables, with an application to the FDA SpontaneousReporting system. Am Stat 1999;53:177-90.

127. Waller PC, Lee EH. Responding to drug safety issues.Pharmacoepidemiol Drug Safe 1999;8:535-52.

128. Louis T, Shen W. Discussion. Am Stat 1999;53:196-8.129. Finney DJ. Monitoring adverse reactions to drugs—its logic

and its weaknesses. Proc Eur Soc Stud Drug Toxicol 1966;7:198-207.

130. Clark JA, Zimmerman HJ, Tanner LA. Labetalol hepatotoxi-city. Ann Intern Med 1990;] 13:210-13.

131. Tubert P, Begaud B, Haramburu F, et al. Spontaneous report-ing: how many cases are required to trigger a warning? Br J




Clin Pharmacol 1991;32,407-8.132. Benichou C, Solal Celigny P. Standardization of definitions

and criteria of causality assessment of adverse drug reac-tions. Int J Clin Pharmacol Ther Toxicol 1991;29:75-81.

133. Jones JK. Adverse drug reactions in the community healthsetting: approaches to recognizing, counseling, and report-ing. Fam Community Health 1982;5:58-67.

134. Turner WM. The Food and Drug Administration algorithm.Special workshop—regulatory. Drug Inf J 1984; 118:259-66.

135. Food and Drug Administration, Center for Drugs andBiologies: Reports evaluation branch manual. Rockville,MD: Food and Drug Administration, 1987:23-23a.

136. Naranjo CA, Lane D, Ho-Asjoe M, et al. A Bayesian assess-ment of idiosyncratic adverse reactions to new drugs:Guillain-Barre Syndrome and zimeldine. J Clin Pharmacol1990;30:174-80.

137. Tubert P, Begaud B, Pere JC, et al. Power and weakness ofspontaneous reporting: a probabilistic approach. J ClinEpidemiol 1992;45:283-6.

138. Idanpaan Heikkila J, Alhava E, Olkinuora M, et al.Agranulocytosis during treatment with clozapine. Eur J ClinPharmacol 1977;11:193-8.

139. Wysowski D, Fourcroy JL. Flutamide hepatotoxicity. J Urol1996;155:209-12.

140. Begaud B, Moride Y, Tubert-Bitter P, et al. False-positives inspontaneous reporting: should we worry about them? Br JClin Pharmacol 1994;38:401-4.

141. Wysowski DK, Green L. Serious adverse events in Norplantusers reported to the Food and Drug Administration'sMedWatch Spontaneous Reporting System. Obstet Gynecol1995;85:538^t2.

142. Tubert P, Begaud B. Random models for margins of a 2 x 2contingency table and application to pharmacovigilance. StatMed 1991;10:991-9.

143. Martinez C, Walker A. Hypothesis testing using spontaneousreports as a source for case-control studies. Pharmaco-epidemiol Drug Safe 1995;4:S69.

144. Inman WHW, Vessey MP. Investigation of deaths from pul-monary, coronary, and cerebral thrombosis and embolism inwomen of child-bearing age. Br Med J 1968;2:193-9.

145. Inman WHW, Vessey MP, Westerholm B, et al.Thromboembolic disease and the steroidal content of oralcontraceptives. A report to the Committee on Safety ofDrugs. Br Med J 1970;2:203-9.

146. Bergman U, Boman G, Wiholm B-E. Epidemiology ofadverse drug reactions to phenformin and metformin. BrMedJ 1978;2:464-6.

147. Tubert-Bitter P, Begaud B. Comparing safety of drugs. PostMarketing Surveillance 1993;7:119-37.

148. Lindquist M, Pettersson M, Edwards IR, et al. How does cys-titis affect a comparative risk profile of tiaprofenic acid withother non-steroidal antiinflammatory drugs. An internationalstudy based on spontaneous reports and drug usage data.Pharmacol Toxicol 1997;80:211-17.

149. Greenland S, Rothman KJ. Introduction to categorical statis-tics. In: Rothman KJ, Greenland S, eds. Modern epidemiol-ogy. Philadelphia, PA: Lippincott-Raven, 1998:237-9.

150. Tubert-Bitter P, Begaud B, Moride Y, et al. Comparing thetoxicity of two drugs in the framework of spontaneousreporting: a confidence interval approach. J Clin Epidemiol1996;49:121-3.

151. Egberts ACG, Meyboom RHB, De Koning FHP, et al. Non-puerperal lactation associated with antidepressant drug use.Br J Clin Pharmacol 1997;44:277-81.

152. Moore ND, Begaud B, Wiholm BE, et al. The case/non-casedesign to generate and assess alerts from pharmacovigilancecase-report databases. Pharmacoepidemiol Drug Safe 1995;4(suppl 1):S66.

153. Moore N, Kreft-Jais C, Haramburu F, et al. Reports of hypo-glycaemia associated with the use of ACE inhibitors and otherdrugs: a case/non-case study in the French pharmacovigilancesystem database. Br J Clin Pharmacol 1997;44:513-18.

154. Figueras A, Capella D, Castel JM, et al. Spontaneous report-

ing of adverse drug reactions to non-steroidal anti-inflamma-tory drugs. Eur J Clin Pharmacol 1994;47:297-303.

155. Rossi AC, Hsu JP, Faich GA. Ulcerogenicity of piroxicam: ananalysis of spontaneously reported data. Br Med J 1987;294:147-50.

156. Mason DH, Bernstein J, Bortnichak EA, et al. Spontaneousreporting of adverse drug reactions: diclofenac sodium andfour other leading NSAIDs. Int Med 1990;ll: 103-17.

157. Platt R, Dreis MW, Kennedy DL, et al. Serum sickness-likereactions to amoxicillin, cefaclor, cephalexin, and trimetho-prim-sulfamethoxazole. J Infect Dis 1988;158:474-7.

158. Stahl MMS, Lindquist M, Pettersson M, et al. Withdrawal reac-tions with selective serotonin re-uptake inhibitors as reported tothe WHO system. Eur J Clin Pharmacol 1997;53:163-9.

159. Tsong Y. Comparing reporting rates of adverse events betweendrugs with adjustment for year of marketing and secular trendsin total reporting. J Biopharm Stat 1995,5:95-114.

160. Sachs RM, Bortnichak EA. An evaluation of spontaneousadverse drug reaction monitoring systems. Am J Med 1986;81:49-55.

161. Begaud B, Tubert P, Haramburu F, et al. Comparing toxicityof drugs. Use and misuse of spontaneous reporting. PostMarketing Surveillance 1991;5:69-76.

162. Bortnichak EA, Sachs RM. Piroxicam in recent epidemio-logic studies. Am J Med 1986;81(suppl 5B):44-8.

163. Lawson DH. More about spontaneous reports of suspectedadverse drug reactions. Hum Toxicol 1988;7:3-5.

164. Stang PE, Fox JL. Adverse drug events and the Freedom OfInformation Act: an apple in Eden. Ann Pharmacother 1992;26:238-43.

165. Inman WHS, Mushin WW. Jaundice after repeated exposureto halothane: an analysis of reports to the Committee onSafety of Medicines. Br Med J 1974;l:5-10.

166. Rossi AC, Bosco L, Faich GA, et al. The importance ofadverse reaction reporting by physicians: suprofen and theflank pain syndrome. JAMA 1988;259:1203-4.

167. Clark JA, Gross TP. Pain and cyanosis associated with a,—proteinase inhibitor. Am J Med 1992,92:621-6.

168. Asplund K, Wiholm B-E, Lithner F. Glibenclamide-associatedhyoglycaemia: a report on 57 cases. Diabetologia 1983;24:412-17.

169. Strom BL, West SL, Sim E, et al. The epidemiology of theacute flank pain syndrome from suprofen. Clin PharmacolTher 1989;46:693-9.

170. Carvajal A, Prieto JR, Requejo AA, et al. Aspirin or aceta-minophen? A comparison from data collected by the SpanishDrug Monitoring System. J Clin Epidemiol 1996;49:255-61.

171. Thacker SB, Berkelman RL. Public health surveillance in theUnited States. Epidemiol Rev 1988; 10:164-90.

172. Thacker SB, Berkelman RL, Stroup DF. The science ofpublic health surveillance. J Public Health Policy 1989;102:187-203.

173. Thacker SB, Stroup DF, Parrish RG, et al. Surveillance inenvironmental public health: Issues, systems, and sources.Am J Public Health 1996;86:633-8.

174. Meyboom RH, Egberts AC, Edwards IR, et al. Principles ofsignal detection in pharmacovigilance. Drug Safety 1997; 16:355-65.

APPENDIX

GLOSSARY

Adverse drug reaction (ADR)

An undesirable event after the use of a drug in humans forwhich there is a reasonable possibility of a causal relation.



208 Clark et al.

Adverse event (AE); adverse drug experience/event (ADE) Characterization case-control study

Any undesirable event associated with the use of a drug inhumans, whether or not it is considered to be drug related bya manufacturer. Reporting an AE does not necessarily reflecta conclusion by a manufacturer or regulatory agency that theevent is causally related to the drug.

Adverse event signal

A potential relation between a product and an AE.

Bayesian data-mining methods

Procedures that extend interproduct quantitative identifica-tion to MxNtables, where Mrefers to a large set of AEs andN refers to a large set of monitored products. The first of these,the Bayesian confidence propagation neural network(BCPNN) method, was published by WHO (7, 124); the sec-ond was developed by DuMouchel (126) with reference to theFDA spontaneous reporting system.

Bayesian probability calculations

In the odds form of the Bayesian expression, the priorodds (the best estimate among exposed patients for theratio of the probabilities in favor of causality vs. non-causality) is multiplied by one or more likelihood ratios.Likelihood ratios are formed by dividing the probabilityfor a risk factor or test result under the condition of causal-ity by the probability for the same risk factor/test resultunder the condition of noncausality. The product of theprior odds and all pertinent likelihood ratios results in aposterior odds for product versus nonproduct causality thatincorporates all known relevant epidemiologic and reportinformation.

Case series characterization

Case series analyses of the distribution of selected contentelements within a drug-AE case series. Case series character-izations usually focus on clinical features, age, gender, anddose and duration of therapy.

Case series formation

The use of case definition to create and evaluate a caseseries. Case definitions are the specification of minimum qual-ity and diagnostic criteria needed to designate reports as cases.

Causality assessment

Determination of whether there is a reasonable possi-bility that the drug is etiologically related to an AE.Causality assessment includes, for example, assessment oftemporal relations, dechallenge/rechallenge information,association with (or lack of association with) underlyingdisease, presence (or absence) of a more likely cause,plausibility, etc.

This design is spontaneous in origin (an evaluator obtainsdata for both cases and noncases from self-selectedreporters) and requires that both cases and noncases receivethe product. Between-group comparisons then focus onpotential risk factors for the development of the syndrome.

Comparative case series distribution

Interproduct quantitative case-based methods have beenextended (infrequently) to comparative characterizations ofproducts. Such characterizations have typically focused onthe reported outcome (seriousness) profile of a particularproduct-AE pair in cases receiving the index versus otherproducts.

Cutoff criteria

Cutoff criteria signaling methods refer to procedures usedto detect an arbitrary number of reports of a product-AEpair.

Dechallenge/rechallenge

Dechallenge. Withdrawal of a drug from a patient's ther-apeutic regimen.

Negative dechallenge. Continued presence of an AEafter withdrawal of a drug.

Positive dechallenge. Partial or complete disappearanceof an AE after withdrawal of a drug.

Rechallenge. Reintroduction of a drug suspected ofhaving caused an AE after a positive dechallenge.

Negative rechallenge. Failure of a drug, when reintro-duced, to produce signs or symptoms similar to thoseobserved when the drug was previously introduced.

Positive rechallenge. Reoccurrence of similar signs andsymptoms upon reintroduction of a drug.

Designated medical event (DME)

AE events of high interest to the regulatory communityfor which additional scrutiny is warranted. These events areusually of historic interest and include hepatic events, seri-ous cutaneous reactions, and congenital anomalies. A DMEis similar to the sentinel health event used in conventionalpublic health surveillance systems.

Expected (labeled) AE

AE is listed in the current regulatory agency-approvedlabeling for the drug as a possible complication of drug use.

Identification

Identification procedures apply explicit or implicitstatistical models to one or more spontaneous reports tofind product-AE pairs of interest. The identification step




is carried out by using reports, not cases, and, as a result,spontaneous signals noted at the identification step areusually further refined using case-based methods.

Imputation

When used as a spontaneous signaling tool, imputation isnot so much a causality assessment measure as it is a way tolocate interesting product-AE pairs for further examination.In applying imputation methods on a report-by-report basiswithin spontaneous reporting systems, their primary use isto triage reports for subsequent examination and not to ren-der formal causality assessment opinions. To reinforce thisconcept, the term "imputation screening" can be helpful.

International Conference on Harmonisation of TechnicalRequirements for Registration of Pharmaceuticals forHuman Use (ICH)

The ICH is concerned with harmonization of technicalrequirements for the development and registration of phar-maceutical products among three regions: the EuropeanUnion, Japan, and the United States. The ICH was organizedto provide an opportunity for harmonization initiatives to bedeveloped with input from both regulatory and industry rep-resentatives. The six ICH sponsors are the EuropeanCommission, the European Federation of PharmaceuticalIndustries Associations, the Japanese Ministry of Health andWelfare, the Japanese Pharmaceutical ManufacturersAssociation, the FDA Center for Drug Evaluation andResearch and Biologies Evaluation and Research, and thePharmaceutical Research and Manufacturers of America.

Numerator-only case distribution

The analysis of distributions of characteristics in a caseseries using case information only.

Periodic safety update reports (PSURs)

PSURs present worldwide safety experience of a medici-nal product at defined times postauthorization to 1) reportall relevant new information from appropriate sources, 2)relate these data to patient exposure, 3) summarize marketauthorization status in different countries and any signifi-cant variations related to safety, 4) create periodically theopportunity for an overall safety reevaluation, and 5) decidewhether changes should be made to product information tooptimize use of the product.

Qualitative sorting (key content) methods

In qualitative sorting by key content, subsets of reportsare selected for further examination because they containcontent elements of interest from a regulatory, medical, orreport-quality perspective. Key content methods often focuson time sequences with diagnostic value (e.g., positivedechallenge or rechallenge), important outcomes (e.g., fatalor life-threatening), previously unreported AE types, or AEtypes of high interest, such as those included in the FDA'spartial listing of designated medical events.

Rate/proportion-based case series distribution

The analysis of distributions of characteristics in a caseseries using both case information and population data.

Retrospective Bayesian analysis

A spontaneous case series formation approach based onBayesian methodology, in which a single summary of theprior odds for the case series as a whole is calculated byusing information provided by the series and is assumed toapply to all cases.

Rule-based case series formation methods

Rule-based case series can be created by specifying min-imum case requirement or a range of diagnostic criteria thatmust be met by the case series as a whole, the results ofwhich can be used to construct rule-based, AE-specific(also called etiologic-diagnostic) case definitions.Generalized rule-based methods for constructing caseseries include the monitored adverse reaction (MAR) and aquality criteria grading methodology proposed by Edwardset al. (56).

Rule-based identification methods (standardizedassessment methods (SAMs))

With rule-based methods (also called standardized assess-ment methods (SAMs) or standardized decision aids(SDAs)), evaluation of an individual AE report is scored byusing a questionnaire or an algorithm. All rule-based meth-ods contain three basic components: 1) a set of structuredresponses, 2) a weighting algorithm that translates theresponses into a standardized score; and 3) an algorithm thatequates scoring ranges with imputation categories (proba-ble, possible, etc.).

Pharmacovigilance

Monitoring the population for pharmaceutical, product-related adverse events.

Serial (increased frequency) identification methods

Spontaneous serial identification methods monitor thenumber, proportion, or rate of a product-AE pair over time.

Postmarketing surveillance

Monitoring of populations exposed to a product after it ison the market.

Serious AE (regulatory definition)

An AE that is associated with death, initial inpatient hospi-talization, prolongation of hospitalization, permanent or



210 Clark et al.

severe disability, permanent or severe disruption in the abilityto carry out normal life functions, a life-threatening situation(i.e., the initial reporter believed the patient was at immediaterisk of death from the event as it occurred), congenital anom-aly, cancer, or overdose.

Sorting

Sorting procedures use data management tools to createreport sets of regulatory and/or epidemiologic interest.Historically, sorting methods have been considered to be aform of signaling, although results are almost always fol-lowed by the application of additional techniques.

Spontaneous case-control design

To use this interproduct case series formation approach, a2 x 2 contingency table is populated by using AE and all-other-AE case counts for the index and other products. Aversion of this method has been referred to as the "case/non-case design" or "ADR reporting odds ratio," with quantita-tive evaluation carried out by using a standard odds ratiostatistic.

Spontaneous cohort design

To use this interproduct case series formation approach,the distribution of reports between two products is com-pared with a probabilistic expectation based on unit sales,prescription number, market share, defined daily doses, orestimated exposed patients. This model is conceptuallyequivalent to the two group Poisson model that is welldescribed in standard texts.

Spontaneous reporting systems

Systems that record "spontaneous" reports—that is,provider reports (and, in the United States only, consumerreports) pertaining to individual patients that are not derivedfrom either the medical literature or studies.

Standardized reporting ratio (SRR)

In SRR analyses, the number of spontaneously reportedcases of a product-AE pair (or the case reporting rate) is com-pared with the expected number of background cases for thesame product-AE pair (or the background incidence rate) tosee whether the actual number of reported cases exceeds the

number of cases expected on the basis of chance. The SRR isanalogous to the standardized incidence ratio (SIR), exceptthat the numerator is formed from reported, not incident,cases. The SRR helps an evaluator project the quantitativeminimum impact of a spontaneously reported product-AEcase series on a product-exposed population. Because theSRR uses a "fully reported" incidence rate as a control stan-dard, it is theoretically insensitive to the effects of clustering,selective reporting, and underreporting.

Suspect drug

A drug thought to be associated with an AE.

Temporospatial cluster identification methods

Instead of accounting for clustering within a serial signalingmodel, an evaluator may design a method that screens product-specific spontaneous AE reports for the presence of tem-porospatial clusters. Although no such AE signaling methodshave been formally proposed, a number of product-AE out-break investigations have resulted from initial presentations thatwere clustered over time and space. Temporospatial clusteringhas been seen with lot-associated and other product defects,with localized patterns of use/misuse, and with events laterdetermined to occur commonly under permissive conditions.

2 x 2 table count methods

Interproduct identification methods in which the propor-tion of all index product reports containing a specific AEtype is compared with the same proportion derived from amultiple product database.

Unexpected (unlabeled) AE

An AE not listed in the current regulatory agency-approvedlabeling for the drug. This includes an AE that may differ froma labeled reaction because of greater severity or specificity(e.g., abnormal liver function vs. hepatic necrosis). AEs listedas occurring with a class of drugs but not specifically men-tioned with a particular drug are considered unlabeled (e.g.,rash with antibiotic X would be unlabeled even if the labelingsaid "rash may be associated with antibiotics"). This isbecause the labeling does not specifically state "rash is asso-ciated with antibiotic X." Reports of death from an AE areconsidered unlabeled unless the possibility of a fatal outcomefrom that AE is stated in the labeling.



spontaneous adverse event signaling methods: classification and

Documents