a reviewer's perspective on the use of constrained versus

A Reviewer’s Perspective on the Use of Constrained Versus Unconstrained Models to Calculate Relative PotencyEvangelos Bakopanos, Ph.D.Senior Biologist/Evaluator, Monoclonal Antibodies DivisionCentre for Evaluation of Radiopharmaceuticals and BiotherapeuticsBiologics & Genetics Therapies Directorate

CASSS BIOASSAYS 2016April 4-5, 2016Silver Spring, Maryland

• The views expressed in this presentation are those of the presenter and do not convey official Health Canada policy.

Disclaimer:

• Scope of the issue

• Relative Potency & Similarity

• Comparison of constrained and unconstrained analysis of data sets:

o Case Study # 1o Case Study # 2o Case Study # 3

• Conclusions

Outline

Analysis of Bioassay Data

Data

Fit Statistical Model

Assess System Suitability

Assess Sample Suitability

Calculate Relative Potency

Reportable Value

𝐘𝐘 =𝐀𝐀 − 𝐃𝐃

(𝟏𝟏 + 𝐗𝐗𝐂𝐂

𝐁𝐁 + 𝑫𝑫

• Unconstrained (full, unrestricted): fit independent 4PL curves to the dose-response data.

• Constrained (reduced, restricted): fit 4PL curves having common A, D and B parameters

ρ = 𝑬𝑬𝑬𝑬𝟓𝟓𝟓𝟓 𝑹𝑹𝑹𝑹𝑬𝑬𝑬𝑬𝟓𝟓𝟓𝟓 𝑹𝑹

Unconstrained vs Constrained 4PL models

• USP <1034> Analysis of Biological Assay:

For those Test samples in the assay that meet the criterion for similarity to the Standard (i.e., sufficiently similar concentration–response curves or similar straight-line subsets of concentrations), calculate relative potency estimates assuming similarity between Test and Standard, i.e., by analyzing the Test and Standard data together using a model constrained to have exactly parallel lines or curves, or equal intercepts.

Calculation of Relative Potency

• Relative potency is calculated using an unconstrained model.

• The use of the unconstrained model may yield variable & questionable relative potency estimates.

• Precludes any reasonable assessment of the potency data contained in the submission.

• Requires re-analysis of the potency data which hinders the review and approval process.

Recurring Issue with Submissions

Approval Year # of NDS affectedby Unconstrained

Issue

% of Approved NDS

2013 1 13%

2014 2 14%

2015*2* 13%

Monoclonal Antibodies Division

*Also had one SNDS case (excluded from table).

• Unable to find research comparing constrained and unconstrained analysis of data sets and there appears to be little consideration of the effect constraining has on the data.

Typical Feedback from Sponsors

• Scope




• Conclusions

Outline

• Can only be strictly defined between two perfectly parallel curves.

Relative Potency

Unconstrained model

Constrain curves in order to enforce parallelism

Unconstrained Constrained

RP = 127 % RP = 95 %

• Hypothesis Testing:o F-Testo Chi-Square

• Equivalence Testing:o Ratio of parameters (A, B, D).

Criteria based on acceptable range for ratio.

Criteria based on acceptable range for Confidence Intervals of ratio.

Assessment of Similarity (parallelism)

67% of

cases

33% of unconstrained cases

• Scope




• Conclusions

Outline

• ELISA assay using unconstrained model to estimate relative potency.

• Sponsor was requested to implement the use of constrained model & re-analyse method validation data.

• Equivalence study was performed to statistically evaluate the difference between the constrained and unconstrained model after both the sample data and reference data pass parallelism.

• Study included method validation, comparability, tech transfer, release & stability data (~200 samples).

Case Study # 1

Case Study # 1 ResultsHISTOGRAMS OF RELATIVE POTENCY RESULTS

• The distributions of relative potency results generated by constrained and unconstrained methods were almost identical with very small variability.

Case Study # 1 Conclusion

An assay displaying very small variability

RP = 109 % RP = 106 %

Case Study # 2• Bioassay using unconstrained 4PL model to calculate

relative potency.

• Three replicate values (i.e., plates) for each sample.

• Observed increased bioassay variability over time.

• More frequent results close to control limits.

• 60 valid assays were selected randomly and re-analyzed using constrained 4PL model.

Case Study # 2 Results

HISTOGRAMS OF MEAN RELATIVE POTENCIES

Case Study # 2 Results (continued)

HISTOGRAMS OF %CVs OF MEAN RELATIVE POTENCIES

• The averages of the distributions were comparable (108% and 106% for constrained and unconstrained respectively).

• The variability of the results from the constrained analysis was 39 % lower than the variability of results from the unconstrained analysis (standard deviations of 8.6 and 14.2 respectively).

• The use of the constrained analysis reduced the inter-plate variability.

Case Study # 2 Conclusions

• Health Canada’s concerns:

o Bioassay using unconstrained 4PL model to calculate relative potency.

o HC’s analysis of raw assay data (provided as part of the consistency lot testing assessment) using a constrained 4PL model yielded significantly different results.

Case Study # 3

• HC’s analysis of the raw assay data.

Case Study # 3

• Sponsor’s Response:o Bioassay data were reprocessed using a constrained

curve fit for method validation results, drug substance and drug product release lots, as well as drug product stability results.

o Based on comparison of the data, an approximate two-fold change in the method’s precision capability was observed using the constrained curve fit (i.e. % CV of < 21 for constrained versus < 11 for unconstrained).

o The curve fitting model used in the validated potency method with the appropriate system suitability criteria generates accurate relative potency values.

Case Study # 3

• Decision:o Sponsor had already developed a new bioassay to

replace the current one & intended to file a post approval supplement immediately after NDS approval.

o New bioassay was validated using a constrained model.

o Bridging study demonstrated that the relative potency values reported by the current (unconstrained) & new method were statistically equivalent.

o Product assigned to Lot Release Evaluation Group 3; required to assess potency using the current method (constrained & unconstrained) as well as the new method.

Case Study # 3

• Bioassay parallelism criteria:

o Acceptable range for upper asymptote ratio

o Acceptable range for slope ratio

o Acceptable range for effective asymptote ratio

CASE STUDY # 3

• Hypothesis Testing:o F-Testo Chi-Square

• Equivalence Testing:o Ratio of parameters (A, B, D).

Criteria based on acceptable range for ratio.

Criteria based on acceptable range for Confidence Intervals of ratio.

Assessment of Similarity (parallelism)

• HC’s analysis of the raw assay data.

Case Study # 3

?

CASE STUDY # 3

Constrained

RP = 101% RP = 62 %

CASE STUDY # 3

RP = 101%

CASE STUDY # 3

Constrained

RP = 70 % RP = 38 %

CASE STUDY # 3

RP = 70 %

• Relative potency estimates are based on the assumption that test samples and standard behave similarly in the assay system.

• This assumption is verified by assessing the similarity or parallelism of the dose response curves.

• For those test samples that meet the criterion for similarity to the standard, relative potency estimates are calculated using a constrained model in order to enforce similarity.

Conclusions

• The effect that constraining has on a given data set highly depends on the variability of the assay and quality of the acceptance criteria used to define similarity. Therefore, it should be evaluated on a case by case basis & during method development.

• When data suggests that dose-response curves of a test sample and standard are not similar, regarding them as similar and estimating relative potency accordingly may lead to incorrect conclusions about the potency of the sample.

Conclusions (continued)

a reviewer's perspective on the use of constrained versus

Documents