Clin Chem Lab Med 2015; 53(6): 887891

*Corresponding author: Rainer Haeckel, Bremer Zentrum fr Laboratoriumsmedizin, Klinikum Bremen Mitte, 28305Bremen,Germany, Phone: +49 412 273448, E-mail: rainer.haeckel@t-online.deWerner Wosniok: Institut fr Statistik, Universitt Bremen, Bremen, GermanyThomas Streichert: Institut fr Klinische Chemie, Universittsklinik Kln, Kln, Germany

Rainer Haeckel*, Werner Wosniok and Thomas Streichert

Optimizing the use of the state-of-the-art performance criteriaDOI 10.1515/cclm-2014-1201Received December 5, 2014; accepted February 9, 2015; previously published online March 20, 2015

Abstract: The organizers of the first EFLM Strategic Con-ference Defining analytical performance goals identi-fied three models for defining analytical performance goals in laboratory medicine. Whereas the highest level of model 1 (outcome studies) is difficult to implement, the other levels are more or less based on subjective opinions of experts, with models 2 (based on biological variation) and 3 (defined by the state-of-the-art) being more objective. A working group of the German Society of Clinical Chemistry and Laboratory Medicine (DGKL) proposes a combination of models 2 and 3 to overcome some disadvantages inherent to both models. In the new model, the permissible imprecision is not defined as a constant proportion of biological variation but by a non-linear relationship between permissible analytical and biological variation. Furthermore, the permissible imprecision is referred to the target quantity value. The biological variation is derived from the reference inter-val, if appropriate, after logarithmic transformation of the reference limits.

Keywords: biological variation; permissible uncertainty; reference intervals.

IntroductionThe organizers of the first EFLM Strategic Conference Defining analytical performance goals identified three models for defining analytical performance goals in lab-oratory medicine. Whereas the highest level of model

1 (outcome studies) is difficult to implement, the other levels are more or less based on subjective opinions of experts, with models 2 (based on biological variation) and 3 (defined by the state-of-the art) being more objective. The benefits and disadvantages of models 2 and 3 have already been discussed elsewhere [1, 2].

Problems with the biological variation model are the large variability between studies (e.g., 2.3%31.9% for triglycerides [3]), data are often generated from relatively young and healthy subjects, dependence on the time span studied (hours to years), and possible effects of measur-and concentrations. Owing to the great diversity of litera-ture reports, many authors consider biological variation not suited to set metrological requirements. Problems with the state-of-the-art model are lack of scientific reasoning, often based on old data, which may be outdated, lack of transparency, lack of neutrality (dependency on indus-try), and the lack of relationship between what is achiev-able and what is needed clinically.

A working group of the German Society of Clinical Chemistry and Laboratory Medicine (DGKL) has devel-oped a combination of models 2 and 3 to overcome some disadvantages inherent to both models [2].

New concept for defining permissible performance criteriaThree types of biological variation (CVB) have been described: (1) intra-individual variation (CVI), (2) inter-indi-vidual variation (CVG), and (3) combined CVB (combined CVI and CVG). As a surrogate for the biological variation [2], the empirical (biological) coefficient of variation (CVE) has been proposed [1, 2]. According to Simundic etal. [4], CVE could be termed CVA+I+G.

The empirical biological variation CVE derived from the reference interval (RI) was proposed because labora-tories are obliged to provide RI for all measurands, must check their transferability (if taken from external sources), and should review the RI periodically as required by ISO 15189 [5]. Furthermore, reference limits (RL) usually vary less than data on biological variation in the literature and often are from consensus groups.

UnauthenticatedDownload Date | 5/19/15 5:06 AM

888Haeckel etal.: Optimizing the state-of-the-art performance criteria

RLs reflect the biological variations, including the analytical variation. If values are normally distributed, the empirical (biological) standard deviation, sE, can be esti-mated by (upper RLlower RL)/3.92 [2]. However, a true empirical normal distribution does not exist in laboratory medicine. For small reference ranges and relatively high mean values (e.g., sodium and chloride concentrations in plasma or hematological quantities), the distribution usually looks close to normal, but a log-normal distri-bution has the same shape under these conditions. For relatively large reference ranges (e.g., thyreotropin, tri-glycerides, enzymes in plasma), it is obvious that the data are not normally distributed. Generally, it may be assumed that laboratory data follow a power normal distribution, i.e., (x1)/ has a normal distribution, where the coeffi-cient controls the shape of the distribution. Important special cases are =1, indicating a normal distribution, and =0, the lognormal distribution. The value for a specific data set can be estimated by numerical methods.

If is unknown and cannot be estimated from data, e.g., because only RLs are given, but there are no individ-ual values, the assumption of a logarithmic distribution was recommended [6]. Assuming a lognormal distribu-tion, the empirical standard deviation (sE) is calculated from the lower reference limit (RL1) and the upper refer-ence limit (RL2) by Eq. (1), as recently explained [2]:

E ,ln 2 1s (lnRL lnRL ) / 3.92.= (1)

Then, the CVE of a logarithmic normal distribution expressed on the linear scale (CVE*) can be calculated from Eq. (1), as explained in the Appendix: 2 0.5E E,lnCV (exp(s ) 1) 100.

= (2)

CVE* can be considered as a surrogate for the conven-tional biological variation (CVB), although it also con-tains the analytical variation. The CVE* values were compared (Figure 1) with the combined CVB values [CVB=(CVI2+CVG2)0.5] for 64 quantities (in blood, serum, and plasma) listed in the RiliBK [8]. The CVB data were taken from Ricos et al. [7]. Although this list is already very comprehensive, CVB data are available for only 80% of the RiliBK measurands. As shown in Figure 1, CVB and CVE* correlate quite well. The greatest divergences (above the regression line=black line in Figure 1) are obtained with C-reactive protein, CA 19-9, and prostate-specific antigen (PSA). In the Ricos etal. [7] list, a relatively high CVG value of 130% is presented for CA 19-9. Erden etal. [9] more recently published a CVG=64.2%, a value that appears more realistic. In Table 1, the intra-individual biological variation (CVI) of PSA is compared from several reports. The most comprehensive survey was published

CA 19-9

Troponin

CRPPSA

Estradiol

-20

0

20

40

60

80

100

120

140

0 10 20 30 40 50 60 70

CVB,

R

icos

et a

l.

CVE*

Figure 1:Comparison of CVE* values with combined CVB values taken from Ricos etal. [7].PSA, prostate-specific antigen; CRP, C-reactive protein.

Table 1:The intra-individual variability of plasma PSA reported by several researchers.

CVIa CVGSletormos etal. [10] (survey of 13 studies) 2.122.9Further reports (not mentioned by Sletormos etal.) Ricos etal. [7] 18.1 72.4Fraser [11] 14.0 72.4Deijter etal. [12] 17.6Erden etal. [13] 20.0Schifman etal. [14] 6.2Gurr etal. [15] 7.0

CVE*=52.5, CVB=74.6 [7]. aCVI, intra-individual biological variation; CVG, inter-individual biological variation.

by Sletormos etal. [10] on behalf of the European Group on Tumor Markers. The CVI values from 13 studies varied between 2.1% and 22.9%. These large differences between studies make a choice of the correct CVB value difficult. Ricos etal. [7] selected a CVI=18.1, whereas a CVI of about 13% (close to the mean of the reported span) would cor-relate with the CVE* quite well.

The empirical biological variation CVE* derived from the reference range can be applied to derive permissible analytical uncertainty and to determine quantity quo-tients standardizing reporting laboratory results.

Proposal for determining permissible analytical uncertaintyVarious approaches have been presented to define permissible analytical uncertainty as a proportion of

UnauthenticatedDownload Date | 5/19/15 5:06 AM

Haeckel etal.: Optimizing the state-of-the-art performance criteria889

2 2 0.5A A ApB=(0.5 pCV +0.5 pCV ) 0.7 pCV .= (4)

Then, the permissible limit of uC is (pCVA2+pB2)0.5:

C A1.22 pCV .u =

Considering a 95% probability of uC, the expanded uncertainty can be calculated by Eq. (5):

ApU% 1.96 pu 2.39 pCV ,C= = (5)

where pU% corresponds to the RMSD value of the RiliBK 2008 [8, column 3 in Table B1a]. The pU% in relation to CVE* values were compared with the RMSD for several measurands (Figure 2). Mean limits for 64 plasma quanti-ties were about 5% lower than those required by the new RiliBK [8]. The straight line represents the 0.5 multipli-cation factor for the biological variation as proposed by Fraser [11]. The problem with this more or less arbitrary classification is the difficulty to which a single measurand may be attributed. The expanded uncertainty also leads to a curved relation with CVE* (like pCVA vs. CVE*).

Permissible limits for ring trials (EQAS)Recently, we have proposed that permissible limits in external quality assessment schemes (EQAS) may be derived of pCVA, respectively, of pU% values [18]. Analo-gous to this approach, the 90% interval of the permissible limits for EQAS may be

EQAS ApU % 1.64 pU% 3.92 pCV= = (6)

biological variation. Cotlove etal. [16] proposed to multi-ply CVB with a constant factor of 0.5. Fraser [11] suggested a three-class model with factors 0.25, 0.5, and 0.75, and Haeckel and Wosniok [1] suggested a five-class model. All models use more or less arbitrary factors. If permissible limits derived of the present state-of-the-art [root-mean-square of measurement deviation (RMSD) in Figure 2] are compared with biological variation data (CVE*), a curved relationship may be observed (analog to the curved line in Figure 2). With the Cotlove etal. [16] model (analog to the straight line in Figure 2), too stringent requirements are postulated, which can hardly be reached by the present technology for measurands with relatively small CVE* values. For measurands with larger biological varia-tion, the requirements may be too permissive. The curved relationship for the present technology can be simply described by Eq. (3):

0.5A EpCV (CV 0.25 ) .

= (3)

This equation approximately describes the empirical rela-tion between CVE* and the major part of the presently used permissible imprecision values. Eq. (3) describes a curved relation between permissible analytical CV (pCVA) and CVE*. It can be easily adapted to future technical improve-ments by modifying its parameters. The exponent in Eq. (3) may be reduced from 0.5 to, e.g., 0.45 if one wishes more stringent permissible limits.

An algorithm to estimate pCVA values for a single target or measured values has been recently proposed [2].

Permissible limits for combined and expanded measurement uncertaintyThree types of measurement uncertainty supported by many international organizations (BIPM, IEC, IFCC, ISO, IUPAC, IUPAP, and OIML) are described in the Guide to the Expression of Uncertainty in Measurement [17]: standard uncertainty uS (imprecision, standard deviation), combined uncertainty uC (u12+u22+u32)0.5, and expanded uncertainty U=kuC (if coverage factor k=1.96, the level of confidence is 95%). In laboratory medicine, at least two components have to be combined, impreci-sion and bias.

The permissible uncertainty (of measurement) can be given by Eq. (3), e.g., u1 in uC. For permissible bias (pB) (e.g., u2 in uC), 0.5pCVA was previously suggested [2]. According to the GUM concept [17], an uncertainty com-ponent should be added for which also 0.5pCVA was pro-posed [2]:

pU%

, RM

SD

0

5

25

20

10

15

0 20 40CV

60E*

80 100

Testosterone

Figure 2:Comparison of empirical biological variation (CVE*) with permissible analytical variation derived of the present state of the art (technical feasibility). RMSD (circles), root-mean-square measurement deviation (column 3 of Table 2a of the German Guideline 2008); straight line, 0.5CVE* line; diamonds, present proposal.

UnauthenticatedDownload Date | 5/19/15 5:06 AM

890Haeckel etal.: Optimizing the state-of-the-art performance criteria

and the 95% interval may be

EQAS ApU % 1.96 pU% 4.68 pCV .= = (7)

Examples for the various permissible uncertainties were recently published [2]. Mean limits for 64 plasma quantities were similar to those required by the new RiliBK [8].

DiscussionThe empirical (biological) variation (CVE*) derived from the reference range is suggested as a surrogate for the bio-logical variation. RLs are available to all measurands, and most probably, the laboratories have more experience with these data because they have to validate them before their introduction in the diagnostic service and then to verify them periodically according to good laboratory practice [5]. CVE* values can be used to derive permissible uncer-tainty by algorithms that may reconcile the presently com-peting biological variation model and the state-of-the-art model.

Although CVE values correlate quite well with biologi-cal variation data (Figure 1), they combine intra- and inter-individual variations (CVC). The relation between both is not constant. The better choice would be to use only CVI data. Presently, however, these data vary considerably in the literature (see example presented in Table 1). There-fore, several authors refused to use CVI to derive permis-sible performance criteria [1, 2, 19]. The application of CVE data may be an acceptable compromise until more reliable data on intra-individual variations are available. Further-more, CVE data vary between laboratories.

The working group Guide limits of the DGKL devel-oped easily to handle Excel tools for the estimation of RIs from intra-laboratory data pools and the estimation of the permissible uncertainty. These tools are distributed gra-tuitously, e.g., via website of the DGKL [20] or from the authors and could be implemented by software compa-nies in their information systems.

AppendixEstimation of CVE*1. Assuming a logarithmic normal distribution for the

measurand, the biological standard deviation derived of the RLs is on the logarithmic scale

E ,ln 2 1s (lnRL lnRL ) / 3.92.=

2. The mean on the logarithmic scale is

ln 2 1mean (lnRL lnRL ) / 2.= +

3. The corresponding mean on the linear scale is [21]2

lin ln E ,lnmean exp(mean 0.5 s ).= +

However, this mean is not needed to calculate CVE*, as is shown below.

4. sE,ln can be transformed to the linear scale according to Aitchison and Brown [21]:

2 0.5E ,lin lin E,lns mean (exp(s ) 1) .=

5. CVE*=sE,lin100/meanlin.6. CVE*=meanlin(exp sE,ln21)0.5100/meanlin.7. CVE*=(exp(sln2)1)0.5100.(2)

Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.Financial support: None declared.Employment or leadership: None declared.Honorarium: None declared.Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.

References1. Haeckel R, Wosniok W. A new concept to determine permissible

imprecision and bias on the basis of biological variation and state-of-the-art. Clin Chem Lab Med 2011;49:62335.

2. Haeckel R, Wosniok W, Gurr E, Peil B. Permissible limits for uncertainty of measurement in laboratory medicine. Clin Chem Lab Med 2015. doi: 10.1515/cclm-2014-0874 [Epub ahead of print 23 Jan 2015].

3. Smith SJ, Cooper GR, Myers GL, Sampson EJ. Biological variability in concentrations of serum-lipids: sources of variation among results from published studies and composite predicted values. Clin Chem 1993;39:101222.

4. Simundic AM, Kackov S, Miler M, Fraser CG, Hylthoft Petersen P. Terms and symbols used in studies on biological variation: the need for harmonization. Clin Chem 2015;61:12.

5. International Standard Medical laboratories particular require-ments for quality and competence, ISO 15189-2012.

6. Haeckel R, Wosniok W. Observed, unknown distributions of clinical chemical quantities should be considered to be log-normal: a proposal. Clin Chem Lab Med 2010;48:13936.

7. Ricos C, etal. Available from: www.westgard.com. Accessed 2014.8. Richtlinie der Bundesaerztekammer zur Qualittssicherung

laboratoriumsmedizinischer Untersuchungen. Dt Aerzteb-latt 2008;105:C30113. Available from: www.aerzteblatt.de/plus1308.

UnauthenticatedDownload Date | 5/19/15 5:06 AM

Haeckel etal.: Optimizing the state-of-the-art performance criteria891

9. Erden G, Barazi AO, Tezcan G, Yildirimkaya MM. Biological variation and reference change values of CA 19-9, CEA and AFP in serum of healthy individuals. Scand J Clin Lab Invest 2008;68:2128.

10. Sletormos G, Semjonow A, Sibley PE, Lamerz R, HylthoftPetersen P, Albrecht W, etal. Biological variation oftotal prostate-specific antigen. A survey of published estimates and consequences for clinical practice. Clin Chem 2005;51:134251.

11. Fraser CG. Biological variation: from principles to practice. Washington, DC: AACC Press, 2001:1151.

12. Deijter SW, Martin JS, McPherson RA, Lynch JH. Daily variability in human serum prostate-specific antigen and prostatic acidphosphatase: a comparative evaluation. Urology 1988;32:28892.

13. Erden G, zden T, Soydas A, Yildirimkaya MM. Biological vari-ation and reference change value (RCV) of prostate specific antigen (PSA) levels in the serum of healthy young individuals. Gazi Med J 2009;20:1526.

14. Schifman RB, Ahmann FR, Elvick A, Ahmann M, Coulis K, BrawerMK. Analytical and physiological characteristics of prostate specific antigen and prostatic phosphatase in serum compared. Clin Chem 1987;33:20868.

15. Gurr E, Krnig F, Golbeck A, Arzideh F. Limits for the deter-mination of guiding values from intralaboratory data basis, demonstrated by prostate specific antigen (PSA). J Lab Med 2009;33:6770.

16. Cotlove E, Harris EK, Williams GZ. Biological and analytic com-ponents of variation in long-term studies of serum constituents in normal subjects. Clin Chem 1970;16:102832.

17. ISO (International Organisation for Standardisation). ISO/IEC Guide 98-3:2008 Uncertainty of measurement. Part 3: guide to the expression of uncertainty in measurement (GUM:1995). Genf: ISO, ISBN 92-67-10188-9.

18. Haeckel R, Wosniok W, Kratochvila J, Carobene A. A prag-matic approach for permissible limits in external assessment schemes with a compromise between biological variation and the state of the art. Clin Chem Lab Med 2012;50:8339.

19. Fuentes-Arderiu X. Variability of the biological variation. Scand J Clin Lab Invest 2002;62:5614.

20. Permissible imprecision (pCVA) and combined uncertainty (pU%) for a particular measurand (xi). Available from: www.dgkl.de. Accessed on 10 January, 2014.

21. Aitchison J, Brown JA. The lognormal distribution. Cambridge: Cambridge University Press 1969:1176.

UnauthenticatedDownload Date | 5/19/15 5:06 AM