Utilization of Assay Performance Characteristics to Estimate Hemoglobin A1c Result Reliability

A. Woodworth, N. Korpi-Steiner, J.J. Miller, L.V. Rao, J. Yundt-Pacheco, L. Kuchipudi, C.A. Parvin, J.M. Rhea, and R. Molinaro

August 2014

www.clinchem.org/content/60/8/1073.full

© Copyright 2014 by the American Association for Clinical Chemistry

© Copyright 2009 by the American Association for Clinical Chemistry

Introduction – Quality Control and RiskIntroduction – Quality Control and Risk Analytical Quality Control (QC) Programs are developed

in all CLIA Certified Laboratories to rapidly identify and correct errors.

CMS Recently Introduced a new type of QC Plan – Individualized QC Plan (IQCP) Laboratories can decide to use traditional QC approaches, or Risk management strategies

The risk of reporting an unreliable patient result can be determined using analytical performance requirements and characteristics of an individual assay.

© Copyright 2009 by the American Association for Clinical Chemistry

IntroductionIntroduction A risk based IQCP for Hemoglobin (Hb) A1c

Ideal to pilot a risk analysis because:− Harmonized and Standardized− Defined performance characteristics− Both lab and POC platforms− Large body of clinical information guides testing

This Hb A1c risk assessment utilizes:

− Assay performance characteristics in clinical settings− Three different acceptance limits (±5%, 6%, and 7%)− Standard QC practice (1-2S with 2 levels of QC)− Same number of patients between QC events (100)

© Copyright 2009 by the American Association for Clinical Chemistry

Quantitative Tools Utilized for Risk Quantitative Tools Utilized for Risk AssessmentAssessment Assay imprecision (%CV) and accuracy (%Bias)

Allowable Total Error (TEa) - Quality specifications for acceptable differences between the true and measured results for a given assay

Patient Weighted Sigma Metrics – Sigma value calculated and averaged over the observed Hb A1c patient distribution, where σ = [(TEa -%Bias)/CV]

E(Nuf) – The number of final patient results with

errors > TEa each time there is a QC out of range

In-Control % Unreliable - Probability (%) of producing results with errors > TEa when the assay is in-control

© Copyright 2009 by the American Association for Clinical Chemistry

IntroductionIntroductionAim of this Study

To evaluate the risk of reporting an unreliable Hb A1c result when using currently available NGSP-Certified Hb A1c

methods.

The risk assessment employed:− 6 different methods across 4 academic medical centers

See Editorial: Little, RR. Performance of Hb A1c Assay Methods: Good

Enough? Clin Chem 2014;60:8:1031.

© Copyright 2009 by the American Association for Clinical Chemistry

QuestionQuestion

Besides analytical error, what other sources of error should be considered when evaluating the risk of reporting an unreliable result? How are these errors detected?

© Copyright 2009 by the American Association for Clinical Chemistry

Materials and Methods- Hb AMaterials and Methods- Hb A1c1c Assays Assays

Instrument Manufacturer Methodology

Variant II Turbo (version 1.0)

BioRad Ion-Exchange HPLC

Variant II BioRad Ion-Exchange HPLC

Tosoh G8 Tosoh Ion-Exchange HPLC

Capillarys 2 SebiaCapillary

Electrophoresis

Cobas Integra 800 Roche Immunoassay

DCA Vantage-1 (old cal)

Siemens Point of Care

DCA Vantage-2 (new cal)

Siemens Point of Care

© Copyright 2009 by the American Association for Clinical Chemistry

Materials and MethodsMaterials and Methods

(n=40/lab)

© Copyright 2009 by the American Association for Clinical Chemistry

QuestionsQuestions

What fixed conditions were used for the assessment of risk of reporting an unreliable result for each assay?

What different TEa settings were used and why?

© Copyright 2009 by the American Association for Clinical Chemistry© Copyright 2009 by the American Association for Clinical Chemistry

Results – Results – Figure 2Figure 2

The percentage bias of all methods was calculated using the linear regression relationships over a range of NGSP target value-assigned Hb A1c concentrations. These biases varied widely across the different platforms with the greatest variability in the Variant II and Integra 800 assays at low Hb A 1c concentrations. Precision was determined using vendor specific QC material and ranged from 1.28% - 2.97% and 0.8% - 2.65% at the low and high levels, respectively (For more detail see table 1).

© Copyright 2009 by the American Association for Clinical Chemistry© Copyright 2009 by the American Association for Clinical Chemistry

The predicted change in the expected number of unreliable patient results prior to an accepted QC event E(Nuf), represented in the y axis and computed over a range of possible out-of control conditions [systematic error (SE)] shown on the x axis for each Hb A1c assay platform evaluated. Each assumes a 1-2s QC rule with 2 QCs and a mean of 100 Hb A1c examinations between QC events and an Allowable Total Error (TEa) of 6%. The lines for the Capillary 2 (3D) and DCA Vantage Lot 2 (3G) are almost flat because the E(Nuf) is minimal.

Results – Figure 3Results – Figure 3

© Copyright 2009 by the American Association for Clinical Chemistry© Copyright 2009 by the American Association for Clinical Chemistry

Results – Table 2Results – Table 2

Three different measures of risk of reporting an unreliable result were calculated for each assay at 3 different Allowable Total Errors (TEa = 5%, 6% and 7%) including the patient weighted sigma, the % of unreliable results while the assay is in control (In-control % unreliable), and the maximum number of unreliable results due to an out of control condition (Max E(Nuf)). Each was calculated assuming a 1-2s QC rule with 2 QC events per day and 100 patient samples between QC events. The Capillarys 2 had the lowest and the Integra 800 had the highest number of expected unreliable results while the assay was in control and out of control at 5% TEa.

© Copyright 2009 by the American Association for Clinical Chemistry

QuestionsQuestions

Which was more influential on method performance (precision or bias) and how can this be checked regularly?

Based on the findings of this study, Is your laboratory running an appropriate amountof QC for your Hb A1c method?

© Copyright 2009 by the American Association for Clinical Chemistry

Summary and ConclusionsSummary and Conclusions Three types of analytical characteristics can be used to assess

risk of reporting an unreliable Hb A1c result: Assay performance characteristics (precision and bias) QC Strategies (# of QC samples + rules) Clinical performance requirements (TEa goals)

Risk estimates for reporting unreliable results based upon analytical performance alone varied almost 500 fold across the 6 Hb A1c assay platforms.

At a ±6% TEa budget, all but one Hb A1c assay in this study requires the “max affordable” QC be run.

Risk estimates for individual laboratories’ Hb A1c methods can be utilized to assess QC practices and residual risk for reporting an unreliable Hb A1c result.

© Copyright 2009 by the American Association for Clinical Chemistry

Thank you for participating in this month’sClinical Chemistry Journal Club.

Additional Journal Clubs are available atwww.clinchem.org

Download the free Clinical Chemistry app on iTunes for additional content!

Follow us