resmeth(bias ppt) arianne letada

8/12/2019 Resmeth(Bias Ppt) Arianne Letada

1/35

Use of recovery and bias information inanalytical chemistry and estimation of itsuncertainty contribution

Thomas P.J.Linsinger

Trends in Analytical Chemistry

Arianne G. Letada

MS-Chemistry


2/35

BACKGROUND-LAB METHOD FLOW

Method

Validation

Method

Transfer

Method

Development

Approved


3/35

Definitions

Validation is the processof demonstrating orconfirming the performance characteristicsof amethod of analysis.

A process of evaluating method performanceand demonstrating that it meets a particularrequirement.

Validation applies to a specific operator,

laboratory, and equipmentutilizing the methodover a reasonable concentration range andperiod of time.


4/35

Why Method Validation?

To minimize analytical and instrumental errors

To give reliable and reproducible results in

accordance with the given specifications of the

test method

To ensure the quality of the test results

To meet accreditation requirement

Objective evidence for defense againstchallenges

To be assured of the correctness of results


5/35

Method bias is generally recognized as intrinsic

part of method validation

There are different points of view on how to

estimate potential bias

whether to correct for it how to accommodate it

its uncertainty in the uncertainty budget for a

particular measurement.


6/35

Introduction

It is a truism in the analytical community that

not every method that should work in principlewill also work in practice and deliver accurate

results.

Analytical methods therefore have to be

validated (either in-house or by laboratoryintercomparison) to demonstrate the reliability

of the results obtained by these methods.


7/35

Parameters for Method Validation

Accuracy

Precision (repeatability, reproducibility) Specificity

Limit of detection

Limit of quantitation

Linearity and range


8/35

Definition

IUPAC

Trueness: Closeness of the

agreement between the

average value obtained froma large series of test results

and an accepted reference

value

Bias: The differencebetween the limiting mean

and the true value

ISO Guide 99

Trueness: Closeness of

agreement between the

average of an infinite number

of replicate measured

quantity values and a

reference quantity value.

Bias: Systematic

measurement error or its

estimate, with respect to a

reference quantity value.


9/35

This review will focus on the estimation of bias

and its uncertainty, essential for assessment ofthe significance of an eventual bias, and the

use of uncertainty in the estimation of

measurement uncertainties.

Particular emphasis is given not only to peer-reviewed literature but also to guidelines

issued by international organizations and

accreditation bodies.


10/35

Estimation of bias

b = xmeasxref eq. 1

This bias may be a function of the analytecontent.

b = xmeas / xref eq.2

The methods of bias estimation differ in thesource of the reference value.


11/35

Requirements in selecting the

materials for bias estimation

Materials shall resemble real-life samples as

closely as possible Materials with different analyte levels shall be

available

The reference values shall be reliable and

their uncertainties as small as possible.


12/35

Certified reference materials (CRMs): well-designedintercomparisons or measurements with another method ofdemonstrated accuracy

Advantage

high reliability of the

reference values with low

uncertainties.

Therefore recommended

for bias estimation (e.g.,IUPAC, ISO 17025 and

Eurachem)

Disadvantages

only in few cases are

materials at different

concentration levelsavailable for a certain

matrix.

due to the longer storage

of CRMs, trade-offsbetween stability and

realistic presentation

must frequently be made


13/35

Harmonised guidelines for the use of recovery informationin analytical measurement: use of surrogates (spikes) asoptions for estimating recoveries

Advantages

materials with different

analyte concentrations

are readily available the uncertainties in the

analyte contents are

generally low

Disadvatages

spikes (isotopically

labeled or not) generally

do not reach anequilibrium in the spiked

samples and hence result

in heterogeneous

samples

The recovery of the surrogate is

therefore likely to be greater than

that of the native analyte, so a

bias in an estimated recovery may

arise


14/35

Meija and Mester Isotope-dilution mass-

spectrometry: their review showed that equalbehavior cannot be expected for trace metalsor organometallic substances.

Analytical Methods Committee: which

recognized that all methods for estimatingrecovery are unsatisfactory in some

circumstances


15/35

ISO TS 21748: Guide to the use of repeatability,reproducibility and trueness estimates in measurementuncertainty estimation

Advantages

only possible for a single

laboratory to demonstrate

bias control with respectto other laboratories, as it

usually has to assume

that the results of the

other laboratories are

correct.

Disadvantages

the lower reliability of the

assigned values


16/35

Testing for significance

If the bias obtained is smaller than the expanded uncertainty of

this bias, there is in reality no evidence of a bias. However, even if

found statistically significant, a bias may still be deemed

practically insignificant if it is small compared to the measurement

uncertainty of the measurement in question.

The Eurachem Guide on measurement uncertainty defines this

small as not larger than one-third of the largest uncertaintycomponent.

This general recommendation has been confirmed by Monte Carlosimulations showing that uncertainties are significantly

underestimated if a bias larger than one third of the other

uncertainty contributions remains uncorrected.


17/35

Uncertainty of bias

ucorr = u2meas+ u2b eq.3ub= u2meas,b+ u2ref eq.4

These two equations illustrate the basic

principle of estimating the uncertainty of biasnamely that it can never be smaller than the

uncertainty of the reference value.

It also breaks down the estimation of bias

uncertainty into two sub-problems, namely

determination of the uncertainty of the

reference value and estimation of the

measurement variation.


18/35

Disadvantages

First, it treats uncertainties of recovery as

combined uncertainty, thus forgoing anydeeper insight into source of the potential bias;

and,

second, it implicitly assumes that the results

obtained from the samples during the study arevalid for future, yet unknown, samples.


19/35

Bettencourt et al., showed that deeper insight

of the source of bias can also be gained fromvalidation data. They applied an intricate analytical design that aimed at

separating uncertainties from re-extraction of the extracted

sample, sample-processing recovery and extraction

recovery. In this way, they managed to identify the main sources of

bias, which subsequently could be used for a targeted

method optimization. Their approach is close to the

bottom-up approach of uncertainty estimation

Maroto et al., who constructed Youden plots forsamples of various weights used to obtain an estimate

of the constant bias, which later on were combined

with the uncertainty from the proportional bias.


20/35

The second problem is not specific to biasestimation but extends to all aspects of method

validations; samples always have to be chosen

to reflect samples encountered in daily use.

The issue of bias is in this respect no differentfrom determination of repeatability,

intermediate precision or limit of quantification.


21/35


22/35

Barwick and Ellison discussed how to

accommodate variation betweenconcentrations, matrices and spike vs. incurred

samples in the general uncertainty model. They gave guidance on how to include additional

uncertainty sources corresponding to differences betweenrecovery of the spiked and the incurred analyte and

changes of recovery with concentration and matrix.

This is done by estimating uncertainty contributions for all

of these variations and including them in the overall

uncertainty model

Eurachem Guide on measurement uncertainty

recommended inclusion of these uncertainties


23/35

Bot et al. used a nested design to evaluate meanuncertainty of recovery, variation of recovery due to

difference in matrices and variation of recoverydepending on the content of endocrine disruptors in a

sample (sediments and waters). Their uncertainties are

estimated from ANOVA. They generally found that

variation of recovery depending on the analyte contentwas negligible.

The problem of matrix mismatch can sometimes be

overcome by using materials from proficiency tests

(PTs) for assessing method trueness. PT samples are frequently closer to real-life samples

than spikes or CRMs, and the problem of matrix

mismatch is hence less prominent.


24/35

Desenfant and Priel explained the use of PT results

for the estimation of uncertainty of bias. The

uncertainty of bias is the standard deviation of thebiases in the various rounds of the PT scheme and

must be included in the overall uncertainty.

If the average bias is not zero, an additional

uncertainty contribution relating to this bias must beadded. However, Desenfant and Priel discouraged

this latter approach. While the use of PT data certainly

has its advantages, one main disadvantage is that

assigned values are often less reliable.


25/35

Correction versus non-correction of

bias

There is evidence that recovery correction

leads in most cases to better comparable

results, but legislation is sometimes unclear or

even explicitly states that results should not be

corrected for recoveries.

Recoveries in organic analysis are oftenregarded as acceptable if they are 70110%


26/35

It is understood that any correction of bias

should be a method of last resort only. Removal of a bias should always be preferred

over accommodation of a bias.

Magnusson and Ellison gave four criteria that

need to be fulfilled to warrant bias correction:

(1) evidence of a significant effect;

(2) a causal relationship;

(3) the estimation of the bias must be sufficiently

accurate; and,

(4) correction must reduce the uncertainty.


27/35

Guide to the Expression of Uncertainty in

Measurement(GUM) explicitly states thatknown biases must be corrected for, and only

in exceptional circumstances is it acceptable to

increase the uncertainty to allow for the bias.

Eurachems translation of the GUM foranalytical chemistry also states that known

biases must be corrected for.

IUPACalso follows this policy. Its guide on theuse of recovery information distinguishes

between rational and empirical methods


28/35

TheAsia Pacific Laboratory Accreditation Cooperation (APLAC)

states that results must be corrected for recovery. Results not

corrected for recovery are only traceable to the specific workinginstruction.

TheAMCstates even more drastically that results obtainedwithout correction for recovery are necessarily empirical

The same conclusions are drawn by theAustralian National

Pathology Accreditation Advisory Council (NPAAC). Eurolab,Nordtest, the Nordic Committee on Food Analysis (NMKL),

European Accreditation (EA) and the American Association for

Laboratory Accreditation (A2LA) agree that bias must be

corrected for.

This unanimous agreement between the institutions in Europe,Asia and the Americas that results must be corrected for

recoveries is encouraging with respect to the ultimate goal of

achieving comparability of measurement results world-wide.


29/35

Use of the uncertainty of bias

According to the law of error propagation,

uncertainties in the basic analytical procedure

and the uncertainty of bias/recovery must be

combined to obtain the full measurement

uncertainty.

There is agreement among international bodiesthat the uncertainty of the bias correction is a

part of the measurement uncertainty.


30/35

The EA Guidelines for uncertainty estimation

state explicitly that in general, the uncertaintyassociated with the determination of the bias isan important component of overall uncertainty.

Lyn et al. showed that ignoring bias, in this

case from sample preparation, underestimateduncertainty.

Feinberg and Laurentiefound that inclusion of

uncertainties of the recovery factors increaseduncertainties.


31/35

Guidance exists on the issue of how uncorrected bias

should be included in an uncertainty statement.

Magnusson and Ellison devoted an entire review articleto the treatment of uncorrected bias. Also, Phillips and

Eberhardt, ODonnell and Hibbert and Synek

investigated several options for allowing for

uncorrected bias. A recovery of 100% does not eliminate the recovery

factor from the basic measurement equation. The

problem basically is that, if the variation in results of a

method is high enough or if the uncertainties of thecertified values of the CRMs used are large enough,

any bias will remain undetected.


32/35

Maroto et al. found that ignoring non-significant

bias results in underestimations of theuncertainty, if the uncertainty of the bias is

large and if the bias contributes significantly to

the overall uncertainty.

Ellison and Barwick also explicitly stated that arecovery of 100% has an uncertainty, but they

also warned of double counting so they

advised planning validation experiments

carefully.


33/35

IUPAC, Eurachem and Nordtest follow this line by

stating explicitly that an uncertainty of recovery needs

to be included in the overall uncertainty even for 100%

recovery or a bias of zero.

The NPAACguide qualifies this point of view by

recommending inclusion of the uncertainty of bias only

when significant.

TheISO explicitly assumes that the uncertainty of the

bias check is negligible compared to the other

uncertainty sources, so does not have to be included. If

this is not the case, an additional uncertaintycontribution must be added.


34/35

Hasselbarththe uncertainty of bias is not a part of theoverall measurement uncertainty. The condition is that

a full uncertainty budget is available and bias is testedas final step in an exhaustive evaluation of

measurement uncertainty.

In this case, all uncertainty sources are already

included and no additional uncertainty needs to beadded when no significant bias is found.

He agrees that, whenever bias is tested in a within-

laboratory or a between-laboratory validation

procedure, the model still includes the recovery factorand its uncertainty must be included even if recovery is

found to be 100%.


35/35

Conclusions and suggested procedure

1. Materials that are sufficiently close to real-life samples with

sufficiently accurately assigned reference values shall be selectedfor the assessment of method bias.

2. Method bias shall be estimated.

3. The uncertainty of the bias shall be estimated. This estimation

includes in all cases the uncertainty of the assigned value and a

contribution of the variation of the measurement results used forthe bias estimation.

4. The significance of the bias is tested by comparing the bias

determined with its expanded uncertainty estimated. If the bias is

found to be significant and no further method optimizations are

performed, results, in general shall be corrected for this bias.5. The uncertainty of the bias is included in the uncertainty budget

for measurements from this method, regardless of whether or not

the bias was found to be significant. If no bias correction is applied

for insignificant bias, an additional uncertainty contribution

accounting for this uncorrected bias has to be added

resmeth(bias ppt) arianne letada

Documents