-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
1/49
1
Risk Assessment
Principles for the
Product QualityInitiatives
H. Gregg Claycamp, Ph.D., CHP
Center for Veterinary MedicineOffice of New Animal Drug [email protected] 25, 2003
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
2/49
2
Outline
Overall Premise and Questions
Basics of Risk Analysis
Possible Stages of Risk Assessments for PQ
Initiatives
Risk Ranking Model for a PQ Initiative?
Pilot Scale
Conclusions
The opinions and ideas presented here are those of the author and do not
represent policy or opinion of the FDA. This material is intended for
discussion purposes only.
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
3/49
3
Premise: Links Among Process (GMP)Risks and Patient Risks are Lost
RISK
P
rocessInspection
Risks
cGMP
RISK
Quality(Pa
tient)
Factors
Patient
Corr
elations?Models?
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
4/49
4
Goal: Re-Link cGMP (PQ) Risks with Actual
Risks to the Patient
RISK
Pro
cessInspection
Risks
RISK
Quality(Patient)
Factors
cGMP Patient
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
5/49
5
The Question
Can Risk Management theory, tools, practice andphilosophy be employed to re-link risks to the patientwith the risks identified, perceived or otherwise
implicated in product quality terms?
How can we share a common language about risk, riskmanagement,and science-based decision making so that
we can focus on developing a high-quality riskmanagement model for product quality?
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
6/49
6
Getting Started
What theories, tools and lessons learned in riskanalysis can help address these questions?
Given the need for a significant shift in the
approach to risk management, how do we beginthe change process?
Are there off-the-shelf models and tools thatmight be used, i.e., at a pilot-scale?
What kinds of RM processes can be used tofoster changes needed both the regulatoryand industrial spheres?
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
7/49
7
Basic Risk Analysis
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
8/49
8
Starting with the Some Basics
Riskis intuitive and familiar to everyone, yet few
among us define risk carefully and formally
enough for complex risk analysis.
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
9/49
9
Risk = exposure to a chance of loss(or, Risk = chance of losing something
we value)
Risk = Hazard x Exposure
RiskConsequence = Hazard x Exposure
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
10/49
10
Contemporary Risk Analysis
Includes four major activities:
Hazard Identification
Risk Assessment
Risk Management
Risk Communication
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
11/49
11
Risk Assessment Precedes Risk
Management
Risk assessment is not a single process, but a
systematic approach to organizing and analysing
scientific knowledge and information to support a riskdecision. NRC (1994)
Various paradigms exist for the execution of a risk
assessment in public health; however, all paradigmshave in common fundamental scientific principles.
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
12/49
12
Risk Assessment Asks:
What can go wrong?
What is the likelihood it would go wrong? What are the consequences?
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
13/49
13
Risk Management Asks:
What can be done?
What options are available?
What are risk trade-offs in terms of risks,benefits and costs?
What are the impacts of currentmanagement decisions on future options?
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
14/49
14
Roles/Tasks (--short list)
Pose the risk question.
Charge the RiskAssessors with the RiskAssessment Task.
Convene stake holders.
Analyze decision options. Make/recommend the
decision.
Identify data and gatherinformation on the nature,
extent, magnitude anduncertainty of the risk.
Write the RiskAssessment.
Recommends changes toRM questions.
Risk Managers Risk Assessors
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
15/49
15
Risk Assessments
Regulatory PolicyRisk Assessment: (e.g.,Biotechnology RA to determine the need for riskmanagement regulation.)
Applied Risk Assessment: To determinecompliance with a regulation or policy.
Safety Assessments: Highly defined riskcalculations. Usually under a bright line safetypolicy.
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
16/49
16
Safety vs. Risk
Risk Limit
e.g., Limit exceeded
10 times in 100
Safety Limit
UnsafeSafe
e.g., Declared unsafe
Estimates of risk
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
17/49
17
Risk Analysis in a Democracy
Risk assessmentsprovide the facts for riskanalysis.
Risk
Risk
Risk
Risk
RiskRisk
RiskRisk
Assessment
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Wors
t
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
18/49
18
Risk Analysis in a Democracy
The risk managementdecisions about whichrisks to manage are value-laden decisions.
Risk
Management
Risk
Risk
Risk
Risk
RiskRisk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Risk
Values
Costs
Wors
t
RiskM
an
ag
emen
t
Rank
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
19/49
19
Translating Risk Analytic Paradigms
Risk Analysis
Risk Assessment
Release Assessment
Exposure Assessment
Consequence Assessment
Risk Estimation
Risk Management
Risk Communication
Hazard Identification
What can go wrong? What are the consequences?
What is the likelihood that it
would go wrong?
What can be done?
What are the trade-offs in termsof costs, benefits and risks?
What is the impact of decisions
on future RM options?
Risk Assessment
PQ Failures
Exposure Assessment
Consequence Assessment
Risk Estimation
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
20/49
20
Possible Stages of RiskAssessment for WorkPlanning
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
21/49
21
Hazard Identification
What can go wrong? Identify hazards: events
Identify hazardous agents (chemical, biological,physical)
How severe are the potential consequences?Given the event occurs, is the consequence
catastrophic? Mildly annoying?
How likely are the events to occur? Essentially a crude risk estimate for initial prioritization
purposes.
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
22/49
22
Exposure Assessment
Release Assessment: How much of thehazardous event occurs?
Example: Does a non-sterile event involve 1
or 10,000 vials?
Pathway analysis: If the hazardous eventoccurs, what pathways are there that expose
humans to the hazard?
Extent of exposure: If a hazardous event occurs,how many people are potentially exposed?
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
23/49
23
GMP Failure (Release) Assessment
How frequent are the identified PQ events(hazards)?
Boundary of release? Process line, plant,warehouse, distributor?
Release rates (PQ Faults) are obtained in fault
tree assessments, empirically, historical data,
expert analyses.
Example: FMEA
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
24/49
24
Consequence Assessment*
Given exposure to the hazardous event/agent,what is the likelihood of harm under a pre-defined endpoint?
Endpoint examples: Death Illness
Worry
OAI
*A.K.A. Dose-Response Assessment (see next slide)
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
25/49
25
Consequence Assessment
Quant i tyof contamination (non-sterility)
i.e., in bacteria counts per vial
P
roportionofex
posed
personswhobec
omeill
100%
50%
0%ED50
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
26/49
26
Qualitative Consequence Assessment
High
Medium
LowRelativeEffect/Impact
Quantitative
relationships knownin fewcases
Low Medium High
(Exposure or Dose Metric)
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
27/49
27
Risk Estimation
Bring together the information about
the hazard,
the extent of exposure to the hazard,
the consequences of exposures, and then
estimate the risk.
Includes a critical analysis ofuncertainty inboth the data and risk assessment models.
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
28/49
28
Uncertainties in Risk Assessment
Knowledge Variability
Data
Parameters
Model
Temporal
Spatial
Inter-individual
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
29/49
29
Conceptual Models for
RM in PQ Initiative
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
30/49
30
The PQ Risk Management Problem
Diverse PQ failure (hazards) are identified.
Wide-ranging risk (= chance that exposure to thehazard will result in harm [adverse outcome]).
Wide-ranging consequences (death to worry).
Quantitative risk analysis hazard-by-hazard toovast an undertaking.
Ranking of risks for re-linking worst PQ riskswith worst health risks, etc.
How can we ob ject ively rankapples and oranges among the
potatoes and beans?
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
31/49
31
From the Beginning
Is risk analysis for each hazard
independentlyfeasible?
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
32/49
32
Bulb
Fails
Noelectricity
Power Plant
Fails
Power Line
Fails
GlassBroken
FilamentBroken
Connector
Corroded
VacuumLeak
Tree BreaksLine
Wind BreaksLine
Impurities Vibrations
Fault Trees for each process?
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
33/49
33
Faults Magnified N-fold for a Simple
Manufacturing Process
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
34/49
34
Decision Analyses for Each Hazard MultipliesComplexity!
e.g.,
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
35/49
35
Solution? A Multifactor Approach to PQ
Risk Management Multifactor methods already exist.
Some tools (software) already developed.
Appropriately-scaled approach to
the question,
the data quality, the nature of the decision, and
the understanding of the overall process.
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
36/49
36
State the Assumptions
E.g., assume that health risks were linked toPQ compliance risks previously, i.e., thehistorical basis of regulation.
Historically based assumption:compliance Health riskquality
Given the assumption, can GMP compliance
risk be modeled as a surrogate of health risk?
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
37/49
37
Identify the PQ Failures (Hazards)
What can go wrong?
Top level organization of hazards:
Health | Compliance | Resources | Sociopolitical
Second level (detail) organization:
Sterility (microbial contamination)
Dose (formulation)
Toxicity (chemical contamination)
Physical hazards (physical contamination/defect)
Fine detail: risk factor event descriptors.
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
38/49
38
Sort the Hazards/Risks by Major
Categories Start with assumptions.
State questions to be answered.
Sort under the questions. Re-sort if new patterns emerge.
For example, (next slide)
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
39/49
39
Risk
Health Compliance Resource
Death
Chronic Illness
Acute Illness
Mental Health
VAI
OAI
Human
Inspection $
Socio-Political
Public
Industry
The Hill
Organizing a Multi-factorial Risk Model
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
40/49
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
41/49
41
Risk factors for a given endpoint
Health
Death
Sterility
Lyophilization
Final Sterility
etc.
Compliance
OAI
Sterility
Lyophilization
Final Sterility
etc.
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
42/49
42
Estimate the Prevalence
The prevalence of inspection findings for agiven type of event are initial estimates ofprobabilities necessary for risk
management modeling.
Failure analysis in plant.
Failure in compliance inspections. Human adverse events.
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
43/49
43
For each hazard
Health
Endpoint
Probability of Occurrence
Very
Low Low Medium High
Very
High
Death Medium Medium High High High
Chronic
IllnessLow Medium Medium High High
Acute
IllnessLow Low Medium Medium High
Worry Low Low Low Medium Medium
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
44/49
44
The modelers view (for example)
Health
Endpoint
Probability of Occurrence
Very
Low Low Medium High
Very
High
Death 5 4 3 2 1
Chronic
Illness6 5 4 3 2
Acute
Illness7 6 5 4 3
Worry 8 7 6 5 4
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
45/49
45
For each hazard
Compliance
Endpoint
Prior History of Actions
Never
Violations
Few
Viol.
Average
Viol.
Some
Viol.
Many
Viol.
OAI Medium Medium High High High
VAI Low Low Medium High High
Other? Low Low Low Medium High
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
46/49
46
Scoring, then prioritize multiple hazards
Endpo
int
Probability of Occurrence
Very
Lo
w Low
Medi
um
Hi
gh
Very
Hig
h
Death
Me
dium
Med
iumHigh
Hi
gh
Hig
h
Chron
icIllness
Lo
w
Med
ium
Medi
um
Hi
gh
Hig
h
AcuteIllness
Low
LowMedium
Me
dium
High
WorryLow
Low LowMediu
m
Medium
Endpoint
Probability of Occurrence
Very
Low Low
Medium
High
Very
High
Death
Me
dium
Medium
HighHigh
High
Chronic
Illness
Low
Medium
Medium
High
High
AcuteIllness
Low
LowMedium
Mediu
m
High
WorryLo
wLow Low
Mediu
m
Med
ium
Endpo
int
Probability of Occurrence
Very
Lo
w Low
Medi
um
Hi
gh
Very
Hig
h
Death
Me
dium
Med
iumHigh
Hi
gh
Hig
h
Chron
icIllness
Lo
wMed
ium
Medi
um
Hi
gh
Hig
h
Acute
Illness
Lo
wLow
Medi
um
Me
dium
Hig
h
WorryLow
Low Low
Me
dium
Medium
Endpo
int
Probability of Occurrence
VeryLo
w Low
Medi
um
Hi
gh
Very
Hig
h
DeathMediu
m
Medium
HighHigh
High
Chronic
Illness
Lo
wMed
ium
Medi
um
Hi
gh
Hig
h
Acute
Illness
Lo
wLow
Medi
um
Me
diu
m
Hig
h
WorryLo
wLow Low
Me
dium
Med
ium
1. PQ Fault A
2. PQ Fault T3. PQ Fault C
4. PQ Fault D
5. PQ Fault X
6. PQ Fault M
Scored and Prioritized
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
47/49
47
Risk Ranking & Filtering Model
Health
Compliance
Other
1. PQ Fault M2. PQ Fault T
3. PQ Fault C
4. PQ Fault D
5. PQ Fault X
6. PQ Fault A
Scored and Prioritized
Under Multiple Criteria
(Risk Ranking and Filtering)
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
48/49
48
Risk Analysis Cycle
Multi-Factorial
Risk ModelRisk Ranking
and Filtering
Work Planning
cGMP/Compliance
Inspections
Assessments
(Data Bases)
Risk ManagementRisk Assessment
Start
-
7/28/2019 FDA Risk Assessment Principles for the Product Quality Initiatives
49/49
Conclusions
Risk Assessment provides a process fororganizing information in support of risk-baseddecision making.
Risk assessment is one of the tools available forRisk Management, the activity in which theoptions for controlling risks are examined in lightof costs, benefits and risk trade-offs.
Multifactor Risk Ranking and filtering approachmight be robust enough to employ in the GMPInitiative.