clinica chimica acta · commutability of the reference material used to assign values to the...

1

2

3

4Q1

5

6789101112131415161718

19

20212223242526272829303132

57

58

Clinica Chimica Acta xxx (2013) xxx–xxx

CCA-13278; No of Pages 8

Contents lists available at ScienceDirect

Clinica Chimica Acta

j ourna l homepage: www.e lsev ie r .com/ locate /c l inch im

External quality assurance programs as a tool for verifyingstandardization of measurement procedures: Pilot collaborationin Europe

D P

RO

OF

C. Perich a,b, C. Ricós a,⁎, V. Alvarez a,c, C. Biosca a,d, B. Boned a,e, F. Cava a,f, M.V. Doménech a,g,P. Fernández-Calle a,h, P. Fernández-Fernández a,i, J.V. García-Lario a,j, J. Minchinela a,k, M. Simón a,l, R. Jansen m

a SEQC-Analytical Quality Commission, Barcelona, Spainb Laboratori Clínic Bon Pastor, Barcelona, Spainc Laboratori Clínic L'Hospitalet, Hospitalet de Llobregat, Spaind Hospital Germans Trias i Pujol, Badalona, Spaine Hospital Royo Villanova, Zaragoza, Spainf Hospital Universitario Fundación Alcorcón, Madrid, Spaing Laboratori Clínic Manso, Barcelona, Spainh Hospital Universitario La Paz, Madrid, Spaini Laboratoris Clínics Hospital Vall d'Hebron, Barcelona, Spainj Hospital Universitario Virgen de las Nieves, Granada, Spaink Laboratori Clínic Barcelonès Nord i Vallès Oriental, Badalona, Spainl Consorci del Laboratori Intercomarcal de l'Alt Penedès, l'Anoia i el Garraf, Barcelona, Spainm SKML Dutch Foundation for Quality Assessment in Medical Laboratories, Nijmegen, The Netherlands

NCO

Abbreviations: AMP, 2-amino-2-methyl-1-propanol; Cglomerular filtration rate; ERM, standard referencemateritopic dilution with coupled plasma mass spectrometry;Measurements (previously BCR, Bureau Comunitaire deTPTZ, 2,4,6-tri-(2-pyridyl)-5-triazine.⁎ Corresponding author at: Plaza Gala Placídia 2, atico,

E-mail address: [email protected] (C. Ricós).

0009-8981/$ – see front matter © 2013 Elsevier B.V. All rihttp://dx.doi.org/10.1016/j.cca.2013.11.005

Please cite this article as: Perich C, et al, ExtePilot collaboration in Europe, Clin Chim Acta

TE

a b s t r a c t
a r t i c l e i n f o
33 Q5
Article history:
34

35

36

37

38

39

40

41

RECReceived 13 August 2013

Received in revised form 22 October 2013Accepted 8 November 2013Available online xxxx

Keywords:StandardizationHarmonizationExternal quality assurance programs

Introduction: Current external quality assurance schemes have been classified into six categories, according totheir ability to verify the degree of standardization of the participating measurement procedures. SKML(Netherlands) is a Category 1 EQA scheme (commutable EQA materials with values assigned by referencemethods), whereas SEQC (Spain) is a Category 5 scheme (replicate analyses of non-commutable materialswith no values assigned by reference methods).Aim: The results obtained by a group of Spanish laboratories participating in a pilot study organized by SKML areexamined, with the aim of pointing out the improvements over our current scheme that a Category 1 programcould provide.Method: Imprecision and bias are calculated for each analyte and laboratory, and compared with quality specifi-

42

43

44

45

46

47

48

49

50

51

52

53

Rcations derived from biological variation.Results:Of the 26 analytes studied, 9 had results comparablewith those from referencemethods, and 10 analytesdid not have comparable results. The remaining 7 analytes measured did not have available reference methodvalues, and in these cases, comparisonwith the peer group showed comparable results. The reasons for disagree-ment in the second group can be summarized as: use of non-standard methods (IFCC without exogenous pyri-doxal phosphate for AST and ALT, Jaffé kinetic at low-normal creatinine concentrations and with eGFR); non-commutability of the reference material used to assign values to the routine calibrator (calcium, magnesiumand sodium); use of reference materials without established commutability instead of reference methods forAST and GGT, and lack of a systematic effort by manufacturers to harmonize results.Conclusions: Results obtained in this work demonstrate the important role of external quality assurance pro-grams using commutable materials with values assigned by reference methods to correctly monitor the stan-dardization of laboratory tests with consequent minimization of risk to patients.

54
U © 2013 Elsevier B.V. All rights reserved.
5556

DC, Centers for Disease Control; DPD, 3,5-dichlorophenyldiazonium tetrafluoroborate; DTE, dithioerythritol; eGFR, estimatedal; GC–IDMS, gas chromatography–isotopic dilution with mass spectrometry; GLDH, glutamate dehydrogenase; ID-CP-MS, iso-IFCC, International Federation of Clinical Chemistry and Laboratory Medicine; IRMM, Institute for Reference Materials andRéference); NAC, n-acetylcysteine; NIST, National Institute for Standards and Technology; SRM, standard reference material;

08006 Barcelona, Spain. Tel.: +34 93 218 14 20.

ghts reserved.

rnal quality assurance programs as a tool for verifying standardization of measurement procedures:(2013), http://dx.doi.org/10.1016/j.cca.2013.11.005

http://dx.doi.org/10.1016/j.cca.2013.11.005

mailto:[email protected]


http://www.sciencedirect.com/science/journal/00098981


T

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158159160

161

162

163

164

165

166

167

168

169

170

171

172173

2 C. Perich et al. / Clinica Chimica Acta xxx (2013) xxx–xxx

UNCO

RREC

1. Introduction

It is well known that 70% of medical decisions are based on laborato-ry reports [1] and, consequently, it is of the upmost importance toachieve standardization among measurement procedures. This wouldalso ensure that all laboratories produce harmonized results, with theaim of eliminating redundant requests and to facilitating interpretationof reports. With this in mind, laboratories should attain the followingobjectives: analytical accuracy, delivering reports in due time, efficacyand with a focus on patient safety [2].

Standardization can be defined as the property of obtaining compa-rable results, independent from analytical fundamentals, measurementmethod, measurement procedure (the combination of instrument,reagent, calibrator and operative mode) [3]. Therefore, measurementresults should be traceable to the International Units (SI) System andshould satisfy the following conditions [4,5]:

• Routine procedures have to quantify exactly the same property as inthe reference measurement and, with the same specificity.

• A primary reference material or a recognized reference methodshould exist.

• Secondary calibrators, including routine calibrators should be trace-able to the primary referencematerial ormethod, and should be com-mutable with patient samples.

In the case of non-compliance of one or more of these conditions,measurement procedures are not standardized but could be harmo-nized (or made comparable) if the results obtained were traceable to areference or a consensus method when testing patient samples [2].

The tool that allows us to continuously verify compliancewith theseconditions is participation in interlaboratory comparison exercises.These exercises are formalized as External Quality Assurance (EQA) Pro-grams and aim to evaluate not only analytical performance, but methodperformance, monitoring of in vitro diagnostics, continuous education,training and support [6].

Recently, EQA programs have been classified into six categories ac-cording to their ability to verify the degree of standardization or harmo-nization of the participating measurement procedures [7]:

1. Use of commutable EQA-samples, with values assigned by a refer-ence method (RM), and with replicate analysis of the EQA-samplesdistributed.

2. Commutable EQA-samples with values assigned by a RM and with-out replicate analysis of EQA-samples.

3. Commutable EQA-samples, no RM values and with replicate analysisof EQA-samples.

4. Commutable EQA-samples, no RM values, no replicate analysis.5. Non-commutable EQA-samples, no RM values and with replicate

analysis.6. Non-commutable EQA samples, no RM values and no replicate

analysis.

EQA programs organized by the Spanish Society of Clinical Biochem-istry and Molecular Pathology (SEQC) and running since 1976 use rep-licate analysis of the same non-commutable EQA-samples during eachcycle (lasting one year), so they are in the 5th category. This impliesthat they can verify the concordance of each individual participant lab-oratorywith other routinemethods [8]. However, they cannot verify ac-curacy of the individual laboratory in a strict sense, because they do notdistribute commutable materials (i.e. without matrix effects) and thematerials are not traceable to reference standards. The Calibration Pro-ject 2000 in The Netherlands [9–11] and its external quality assurancescheme organizer SKML (Stichting Kwaliteitsbewaking MedischeLaboratorium Diagnostiek) has achieved the most stringent conditionsfor several common chemistry analytes and is denoted as a Category 1EQA scheme [7]. In particular, the materials for general chemistry areproven to be commutable andmethod target values are assessed by ref-erence methods [12–14]. SKML provided SEQC with a set of its EQA

Please cite this article as: Perich C, et al, External quality assurance programPilot collaboration in Europe, Clin Chim Acta (2013), http://dx.doi.org/10.

ED P

RO

OF

materials, which were also distributed to usual SKML participants inthe 2011 cycle as well as to some laboratories in the UK. This pilotstudy showed that the degree of standardization was better for Dutchlaboratories than for the others [15].

The results obtained by the Spanish representatives are further ex-amined here with the prospect of underlining the need for a category1 EQA program to verify standardization in our setting.When non-stan-dardization was found, the analytical procedures used were checkedthoroughly to have an idea of the reasons for discrepancies.

2. Materials and methods

2.1. EQA materials

Six EQA samples from SKML were tested for 6 days (one sample perday in duplicate). These samples were human serum pools, frozen at−80 °C, distributed and maintained at this temperature until their analy-sis. Over the same time period (Jan to Jun 2013) SEQC-EQA sampleswere also analyzed.

2.2. Analytes tested

A total of 26 analytes were tested: alanine-aminotransferase (ALT),albumin, alkaline phosphatase, α-amylase (total), aspartate-aminotransferase (AST), bilirubin, calcium, chloride, cholesterol, crea-tine kinase (CK), creatinine, γ-glutamyltransferase (GGT), estimatedglomerular filtration rate (GFR), glucose, HDL-cholesterol, inorganicphosphate, iron, lactate dehydrogenase (LDH), lipase, magnesium, po-tassium, protein, sodium, triglyceride, urate, and urea.

2.3. Laboratory platforms

Ten laboratories from different parts of Spain participated; two ofthem used two measurement procedures, giving a total of 12 proce-dures. Exceptionally, creatinine was measured by 13 procedures be-cause one laboratory used the enzymatic method as a third procedure.The instruments usedwere: Abbott Architect ci 16200, Siemens Dimen-sion Vista, Beckman Coulter Olympus AU 5400, Roche Cobas 711, RocheCobas 6000 and Siemens Advia 2400.

2.4. Data analysis of imprecision and bias

2.4.1. Within-batch imprecisionFor each analyte and EQA-sample from SKML, imprecision was cal-

culated from the duplicated analysis, applying the formula:

SD ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi∑d2=2n� �q

where d is the difference in duplicates (r1, r2) and n the number ofpairs, 6 in this case. Generally CV = SD / (Σr / 2n).

2.4.2. Between-laboratory imprecisionFor each analyte and SKML-EQA-sample, the first value of each pair

of results provided by each laboratory was taken to calculate the coeffi-cient of variation among the 12 laboratory procedures and, then, the av-erage among the six CV obtained for the six EQA-samples werecalculated. The same procedure was applied to results of six SEQC-EQA samples analyzed during the pilot study period.

2.4.3. Bias (systematic error)For each analyte and EQA sample, the first result (r1) of each labora-

tory procedure was taken to calculate the percentage deviation (PD)against the target value (TV), according to the formula:

PD ¼ 100 � r1–TVð Þ=TV:

s as a tool for verifying standardization of measurement procedures:1016/j.cca.2013.11.005


174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

t1:1

t1:2

t1:3

t1:4

t1:5

t1:6

t1:7

t1:8

t1:9

t1:10

t1:11

t1:12

t1:13

t1:14

t1:15

t1:16

t1:17

t1:18

t1:19

t1:20

t1:21

t1:22

t1:23

t1:24

t1:25

t1:26

t1:27

t1:28

t1:29

t1:30

t1:31

t1:32

t1:33

t1:34

t1:35

Table 2a t2:1

t2:2Analytes testedwith commutable EQAmaterials and values assigned by referencemethodt2:3obtaining comparable results.

t2:4Analyte Reference methodand laboratory

Routine method(number of labs)

Routinecalibratortraceability

t2:5Bilirubin IFCCDGKL, Hanover

DPD (9)Jendrassik-Grof (2)Vanadate oxidation (2)

NIST-SRM916⁎

NIST-909b⁎⁎

GC/MSDoumas

t2:6Chloride CoulometryINSTAND

Indirect potentiometry(12)

NIST-SRM919⁎

NIST-SRM909b⁎⁎

3C. Perich et al. / Clinica Chimica Acta xxx (2013) xxx–xxx

In this way, six PD values were obtained for each analyte and mea-surement procedure.

The target value was assigned by a reference method when avail-able; if this kind of method did not exist, target value was the peergroup mean (laboratories using the same measurement method) ob-tained by the 200 participants in the SKML pilot study (170 labs fromThe Netherlands, 10 from Spain, 10 from Portugal and 10 from the UK).

Results obtained were considered to be comparable when the per-centage deviation fell within the acceptability limit derived frombiolog-ical variation for total error [16]. These limits were desirable, minimumor optimum [17] according to the criterion applied in the SEQC-EQA re-ports for each analyte. Because the six values throughout the concentra-tion range are considered together, the overall view for each analytedesignates the systematic error.

RO

OF NIST-SRM

956⁎⁎⁎

“Masterlot”t2:7Cholesterol CDC, Abell Kendall

Erasmus Medical Center,Rotterdam

Cholesterol-oxidase,esterase, peroxidase(12)

NIST-SRM917b⁎

NIST-909b⁎⁎

CDCIDMS

t2:8Creatinekinase

IFCC (NAC activator, 37 °C)Haga Hospital, La Haya

UV, NAC activator (11)UV, DTE activator (1)

ERM/IFCC-AD455$

IFCC

3. Results and discussion

3.1. Imprecision

Within and between-laboratory imprecision are shown in Table 1.Within-run laboratory imprecision falls within the acceptability limitderived from biological variation in all cases, except for sodium. Formost of the analytes studied, imprecision between laboratories is great-er when using SEQC-samples compared with SKML-samples.

ED P Absorptivity

t2:9HDL-Cholesterol

CDC designatedcomparison methodErasmus Medical Center

Direct (10) NIST-909b⁎⁎

CDCt2:10Glucose GC–IDMS

INSTAND, DüsseldorfHexokinase (12) NIST-SRM

917b⁎

NIST-SRM965a⁎⁎⁎

3.2. EQA value assignment

Reference methods and reference laboratories assigning values toEQA materials, as well as routine methods and material or methodused for traceability of routine calibrators are shown in Tables 2a–2c.

UNCO

RRECT

199

200

201

202

203

204

205

206

207

208

209

210

211

Table 1Within and between laboratory imprecision for pilot study EQA samples.

Analyte CVw-l (%) CVb-l (%)

SKML SEQC

Albumin 0.6 3.3 4.2Alkaline phosphatase 0.8 6.4 18.1ALT 0.8 8.7 11.2α-Amylase (total) 0.3 11.6 12.4AST 1.2 6.0 5.1Bilirubin 0.9 9.6 19.1Calcium 0.5 3.5 3.7Chloride 0.4 1.4 2.4Cholesterol 1.0 4.1 3.5CK 0.9 4.9 5.2Creatinine 1.3 7.0 8.0GGT 0.8 12.0 10.0Glucose 0.7 5.9 3.1eGFR 0.8 5.2 8.4HDL-cholesterol 1.1 5.2 19.5Inorganic phosphate 0.6 2.9 3.0Iron 0.7 2.3 3.4LDH 0.7 6.1 10.0Lipase 0.6 4.1Magnesium 0.9 4.5 4.3Potassium 0.5 1.5 2.0Protein 1.2 3.2 5.3Sodium 0.9 2.1 1.8Triglycerides 0.6 3.5 4.2Urate 0.6 5.2 5.4Urea 0.9 3.4 4.1

nd, not measured.CVw-l: average within laboratory CV from the pilot study.CVb-l from SKML: between laboratory CV from the pilot study.CVb-l from SEQC: between laboratory CVof same laboratories for 6 SEQC–EQA samples dis-tributed during the pilot study period.

IDMSAbsorptivity

t2:11Potassium ID-CP-MSFlame spectrometryINSTAND, Düsseldorf

Indirectpotentiometry (12)

NIST-SRM909b⁎⁎

NIST-SRM956⁎⁎⁎

“Masterlot”t2:12Protein Biuret

INSTAND, DüsseldorfBiuret (12) NIST-SRM

927c#

t2:13Urate HPLCErasmus Medical Center

Uricase-colorimetric(3)Uricase-peroxidase(9)

NIST-SRM913⁎

NIST-SRM909b⁎⁎

IDMS

t2:14DPD: 3,5-dichlorophenyldiazonium tetrafluoroborate; NAC: n-acetylcysteine; DTE:t2:15dithioerythritol; ERM: standard reference material.

⁎ Aqueous solution, pure substance. t2:16⁎⁎ Lyophilized human serum with unproven commutability. t2:17⁎⁎⁎ Frozen human serum with unproven commutability. t2:18

$ Lyophilized human serum with unproven commutability. t2:19# Non-commutable bovine serum. t2:20


Tables 2a–2c are organized as follows with the analytes involvedpresented in alphabetical order:

a) Analytes tested with commutable EQA materials and valuesassigned by reference method obtaining comparable results.

b) Analytes tested with commutable EQA materials and valuesassigned by reference method not obtaining comparable results.

c) Analytes tested with commutable EQA materials without valuesassigned by reference methods but by peer groups.

3.3. Bias assessment

3.3.1. Analytes tested with commutable EQA materials and values assignedby reference method obtaining comparable results

The analytes in this group are: bilirubin, chloride, cholesterol, HDL-cholesterol, creatine-kinase, glucose, potassium, protein and urate.



D P

RO

OF

212

213

214

215

216

217

Table 2bt3:1

t3:2 Analytes tested with commutable EQA materials and values assigned by reference method not obtaining comparable results.

t3:3 Analyte Reference method and laboratory Routine method(number of labs)

Routine calibrator traceability

t3:4 ALTAST

IFCC (TRIS buffer with pyridoxal phosphate, 37 °C)Haga Hospital, The Hague

IFCC, with pyridoxal phosphate (4)IFCC without pyridoxal phosphate (8)

IFCCAbsorptivity

t3:5 α-Amylase IFCC (Maltoheptaoside with p-nitrophenol and ethylidene, 37 °C)Haga Hospital, The Hague

Maltoheptaoside-p-nitro phenol and ethylidene (5)Maltotrioside with 2 chloro-p-nitrophenol (3)

IRMM/IFCC 456@@

“Masterlot”IFCCAbsorptivity

t3:6 Calcium Atomic absorption spectrometryINSTAND, Düsseldorf

Arsenazo (7)O-cresolphthalein (5)

NIST-SRM 915⁎

NIST-SRM 909b⁎⁎

NIST-SRM 956⁎⁎⁎

t3:7 CreatinineeGFR

GC–IDMSDGKL, Hanover

Enzymatic (1)Jaffé kinetic (5)Jaffé kinetic compensated (7)

NIST-SRM 914⁎

NIST-SRM 909b⁎⁎

NIST-SRM 967⁎⁎⁎⁎

IDMS&Q3t3:8 GGT IFCC (gamma-glutamyl-3-carboxy-4-nitroanilide N 4 mmol/L, 37 °C)

Haga Hospital, The HagueIFCC (12) ERM/IFCC 452&&

t3:9 LDH IFCC (lactate to pyruvate, 37 °C)Haga Hospital, The Hague

Lactate to pyruvate (7)Pyruvate to lactate (5)

IRMM 453@@

t3:10 Magnesium Atomic absorption spectrometryINSTAND, Düsseldorf

Arsenazo (1)Xylidyl blue (3)Chlorophosphonazo (1)

NIST-SRM 929⁎

NIST-SRM 909b⁎⁎

t3:11 Sodium Flame emission spectrometryINSTAND, Düsseldorf

Indirect potentiometry (12) NIST-SRM 909b⁎⁎

NIST-SRM 956⁎⁎⁎

t3:12 eGFR, estimated glomerular filtration rate.⁎ Aqueous solution, pure substance.t3:13⁎⁎ Lyophilized human serum with unproven commutability.t3:14⁎⁎⁎ Frozen human serum with unproven commutability.t3:15⁎⁎⁎⁎ Frozen human serum with proven commutability.t3:16

& Animal tissue, commutable.t3:17&& Animal tissue l, not commutable.t3:18@@ Human tissue, commutability not proved.t3:19

t4:1

t4:2

t4:3

t4:4

t4:5

t4:6

t4:7

t4:8

t4:9

t4:10Q4

t4:11

t4:12

t4:13

t4:14

t4:15

t4:16

t4:17


For each analyte, percentage deviation of results obtained versus thecorresponding reference method value is compared with the qualityspecification for total error derived from biological variation, being the

UNCO

RRECT 218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

Table 2cAnalytes testing commutable EQA materials without values assigned by referencemethods but by peer groups.

Analyte Comparisonmethod

Routine method(number of labs)

Routinecalibratortraceability

Albumin Peer group mean Bromocresol green(10)Bromocresol purple(2)

ERM-DA470k⁎⁎

NIST-SRM 927c#

Alkalinephosphatase

Overall mean 4-nitrophenyl-phosphate,AMP buffer (12)

IFCCInternal calibratorAbsorptivity

Inorganicphosphate

Overall mean Phosphomolybdate,340 nm (12)

NIST-SRM 21861Internal calibrator

Iron Overall mean Ferrozine (3)TPTZ (4)Ferene (1)Colorimetric (2)

NIST-SRM 937⁎

Lipase Overall mean 1,2 diglyceride withglycerol-kinase andglycerol-3-phosphate-peroxidase (5)

Internal calibrator

Triglyceride Overall mean Lipase/glycerol kinase (11) IDMS⁎⁎⁎

NIST-SRM 909b⁎⁎

AbsorptivityUrea Overall mean Urease-GLDH (12) IDMS

NIST-SRM 912⁎

NIST-SRM 909b⁎⁎

AMP: 2-amino-2-methyl-1-propanol; TPTZ: 2,4,6-tri-(2-pyridyl)-5-triazine; GLDH:Gluta-mate dehydrogenase.⁎ Aqueous solution, pure substance.⁎⁎ Lyophilized human serum with unproven commutability.⁎⁎⁎ Frozen human serum with unproven commutability.

# Bovine serum, not commutable.


Edesirable limit for 5 analytes (cholesterol, HDL-cholesterol, glucose, po-tassium and urate), the minimum limit for two analytes (chloride andprotein) and the optimum limit for two analytes (bilirubin andcreatine-kinase). More than 95% of deviations obtained are within thelimits, so we can consider that results provided by participant laborato-ries are comparable and, consequently, method procedures used arewell standardized. This happens both when the same measurementprocedure is used in all laboratories (6 analytes) and when differentmeasurement procedure are used (3 analytes), even with differenttraceability of routine calibrators (Table 2a).

In the case of glucose, only two results are not comparable, corre-sponding to the two measurement procedures using a calibrator physi-cally traceable to the NIST-SRM 917 reference material, which has anaqueous matrix and, consequently, is expected to be non-commutablewith human serum.

3.3.2. Analytes tested with commutable EQA materials and values assignedby reference method not obtaining comparable results

This group includes: alanine aminotransferase (ALT), aspartate ami-notransferase (AST), α-amylase (total), calcium, creatinine, eGFR, γ-glutamyltransferase (GGT), lactate dehydrogenase (LDH), magnesiumand sodium.

The percentage deviation of results obtained versus the correspond-ing referencemethod value is depicted in Figs. 1 and 2 for analytes withclear reasons for non-comparability; Supplemental data file Fig. 1 showsthe remaining analytes of this group. Percentage deviations are com-pared with quality specifications for total error derived from biologicalvariation, with the minimum limit for two analytes (calcium and sodi-um), the optimum limit for two analytes (ALT and GGT) and the desir-able limit for the remaining six analytes.

3.3.2.1. ALT and AST. 4 out of the 12 participants use a routine methodbased on the IFCC reference method (TRIS buffer with pyridoxal-5-phosphate (P5P), at 37 °C), and 8 labs use a similar method butwithoutP5P. Percentage deviation of results versus the reference method value



CTED P

RO

OF

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

-40-30-20-10

0102030

30 60 90 120 150 180

ALT

UI/L

100 150 200 250 300 350 400

Amylase

U/L100 200 300 400

G7 PNP-ethilidene

Amylase-Method effect

U/L

1,9 2,1 2,3 2,5 2,7 2,9 3,1 3,3

Calcium

mmol/L

-8

-6

-4

-2

0

2

4

6

2 2,4 2,8 3,2

NIST SRM 909-

NIST SRM 915-

NIST SRM 956-

Calcium - Traceability effect

mmol/L

-40

-30

-20

-10

0

10

20

-30

-20

-10

0

10

20

30

-15

-10

-5

0

5

10

-30

-20

-10

0

10

20

30

L1 L7

L12

L11 L3 L4 L5 L6 L8 L9

L10

L11

TRIS with P5P

TRIS without P5P

ALT-148 U/L - Method effect

G3-2Cl-PNP

Fig. 1. Analytes tested with commutable EQA materials and values assigned by reference method not obtaining comparable results. Y-axis: Percentage deviation compared with the ref-erence method value. X-axis in all figures except ALT at 148 U/L: reference method value for the six EQA-samples (ALT: 38, 83, 104, 120, 148, 170 U/L; amylase: 130, 172, 199, 256, 298,382 U/L; calcium: 2.10, 2.27, 2.29, 2.60, 2.78, 3.10 mmol/L); Laboratory procedures 1 to 12; dashed lines represent acceptability limits derived from biological variation (optimum for ALT,desirable for amylase, minimum for calcium).


UNCO

RREfall outside the optimum limits for both analytes in a number of cases, as

was shown before by Jansen et al. [18]. For example ALT at 148 U/Lshows lower results for measurement procedures without P5P (Fig. 3),which may be due to a possible instability of ALT in frozen samples, asdescribed by Infusino et al. [19]. However, studies on the SKML samplesshowed stability at−80 °C for ALT and AST with recoveries against thereference target value falling around 100% for all instrument groups,both in samples measured in January and in samples tested in Decem-ber, year after year.When considering the traceability of the routine cal-ibrator used, no discrepancies are seen among the 11 measurementprocedures traceable to the IFCC referencemethod and the single proce-dure using molar absorptivity.

3.3.2.2. α-Amylase (total). Only 8 out of the 12 participants test this ana-lyte, 5 of them using the G7-PNP ethylidenemethod (IFCC recommend-ed) and 3 the G3-2chloride-PNP. There are several results below theacceptability limit for the whole concentration interval studied(Fig. 1). When stratifying the results according to the measurementmethod,we can see that substrate heptaosidemethod achieves satisfac-tory results,whereas substrate trioside obtains lower results, alsowith ahigh interlaboratory imprecision as shown in Table 1. The results are inagreementwith those previously reported [18]. No effect due to the cal-ibrator traceability is observed for this analyte.

3.3.2.3. Calcium. The majority of results obtained fall within the mini-mum limits derived from biological variation for total error, indepen-dently of the measurement procedure used. However, there are someunacceptably low values provided by laboratories using a routine


calibrator traceable to the NIST-SRM 915 standard (calcium carbonateaqueous dissolution, amatrix so different to human serum that it is con-sidered to be non-commutable) (Fig. 1). Consequently, it seems thatthere is a direct relationship between the accuracy of results and thematrix of the standard used to assign the values of the routinecalibrators.

3.3.2.4. Creatinine. Results fall, in general, within the acceptable interval.However, at the level of clinical relevance (79 μmol/L) several high re-sults can be seen. Fig. 2 shows the deviation of participantmeasurementprocedures classified according to recommendations of the SEQC-RenalFunction Commission [20]. Correct results are provided by theenzymatic procedure and, also by 5 of the 7 compensated Jaffe kineticmeasurement procedures; the other 5 procedures based on the non-compensated Jaffé kinetic method give unacceptably high results.Similar findings were published by Delanghe et al. [21]. When stratify-ing results according to the metrological traceability of routine calibra-tors used, we can see that all procedures traceable to IDMS give correctresults but not those traceable toHPLC.Moreover, when the routine cal-ibrator is traceable to a referencematerial, no clear association with theaccuracy of results was observed.

3.3.2.5. Estimated glomerular filtration rate (eGFR). eGFR is calculatedfrom the creatinine result assuming that samples come from a 55-year-old white female. The formulae used are MDRD or MDRD–IDMSand the reference material or methods used to offer traceability are de-scribed in the previous paragraph. Results obtained are depicted inFig. 2, where we can see deviations up to 15%, in both senses at the



TED P

RO

OF

300

301

302

303

304

305

306

307

308

309

310

311

312

313

-20

-10

0

10

20

30

-20

-10

0

10

20

30

-20

-10

0

10

20

30

-20

-10

0

10

20

30

-20

-10

0

10

20

-20

-30

-10

0

10

20

70 120 170 220

Creatinine

L7 L8 L2 L6 L11 L7 L3 L4 L5 L6 L9 L10 L1

Enzymatic

Jaffé-kinetic

Jaffé-Kinetic compensated

Creatinine 79 mol/L-Method effect

10 30 50 70

eGFR

mL/min/1.73m2

-30

-20

-10

0

10

20

L8 L11 L7 L2 L6 L5 L3 L4 L10 L9 L1

Jaffé-Kinetic

Jaffé-Kinetic compensated

66 mL/min /1.73m2-Method effect

60 80 100 120 140 160UIL

-glutamyl transferase

L2 L7 L11 L6 L1 L8 L5 L3 L4 L10 L9

MDRD

MDRD-IDMS

66mL/min/1.73m2-Formula effect

L1 L3 L5 L7 L9 L11

GGT, 78 U/ L-Traceability effect

IRMM 452-

Absorbance

IFCC

internal calibrator

m

Fig. 2. Analytes tested with commutable EQA materials and values assigned by reference method not obtaining comparable results. Y-axis: Percentage deviation compared with the ref-erencemethod value. X-axis in all figures except creatinine at 79 μmol/L, eGFR at 66 mL/min/1.72 m2 and GGT at 78 U/L: referencemethod value for the six EQA-samples (creatinine:79,126, 149, 162, 196, 219 μmol/L; eGFR: 21, 23, 30, 32, 39, 66 mL/min/1.72 m2; GGT: 62, 78, 88, 110, 127, 159 U/L); Laboratory-procedures 1 to 12; dashed lines represent acceptability limitsderived from biological variation (desirable for creatinine, optimum for GGT).

Q2


RECconcentration levels of 40 and 66 mL/min/1.73 m2. When stratify-

ing results obtained at the level of maximum clinical interest(66 mL/min/1.73 m2) according to the creatinine method used, itis evident that all results from the Jaffé kinetic method have a nega-tive deviation, with falsely low results under the cut-off value of60 mL/min/1.73 m2. Such results could lead clinicians to incorrectlyassess the severity of risk to health in patients. When stratifying

UNCO

R

Fig. 3


results according to the formula employed we can see that neitherof the two equations compensate for the low values located in thearea of risk to health (negative deviation).

Concerning traceability, we can see that procedures traceable tothe NIST 967 overstate patient risk when using the Jaffé kineticmethod because results have a negative bias, whereas results aremore satisfactory when using the compensated Jaffé procedure.

.



T

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431Q7

432


UNCO

RREC

3.3.2.6. GGT. Results obtained are quite different from each other andsome of them exceed the acceptable limit, as was observed before[18]; also, a high interlaboratory imprecision can be seen in Table 1.This happens despite the fact that all laboratories use the same metro-logical method.

Stratifying results according to the metrological traceability of thecalibrator values prove that the higher values correspond to one proce-dure traceable to the IRMM452 standard material and two laboratoriesusing an internal calibrator (masterpool from themanufacturer). On theother hand, calibrator values traceable to a reference method give re-sults well within the acceptability limits.

3.3.2.7. LDH. The participant laboratories use two different methods:

• Lactate to pyruvate (L–P), using the same substrate as for the refer-ence method (7 laboratories).

• Pyruvate to lactate (P–L), (5 laboratories).

Because results from the second method are approximately doublethose of the first, bias of the two groups is studied separately. The sub-strate pyruvate procedure, as with the IFCC reference method, obtainsdeviations within the acceptable limits, except for two results corre-sponding to one laboratory using a molar absorptivity. The lack of stan-dardization seen here is in agreement with previous reports [15,18].

3.3.2.8. Magnesium. Participating laboratories use three differentmethods (Table 2b). Percentage deviation of results versus the refer-ence method falls generally within the acceptable limits; however, var-ious exceptions are distinguished that cannot be associated with themetrological procedure used. All procedures are traceable to non-commutable reference materials (Table 2b) and maybe this fact couldexplain the poor results observed.

3.3.2.9. Sodium. Only a limited percentage of results fall quite within theacceptable limits. There are several values under the lower limit, comingfrom themetrological procedure traceable to theNIST-SRM909b,whichis not a proven commutable material.

3.3.3. Analytes testing commutable EQA materials without values assignedby reference methods but by peer groups

This section includes: albumin, alkaline phosphatase, inorganicphosphate, iron, lipase, triglyceride and urea.

Percentage deviation of results versus the peer group mean is com-pared with quality specifications for total error derived from biologicalvariation. The limits are minimum for albumin, desirable for alkalinephosphatase, inorganic phosphate, urea, and optimum for iron, lipaseand triglyceride (Supplemental data file, Fig. 2).

3.3.3.1. Albumin. Two different metrological procedures, bromocresolgreen (BCG) and bromocresol purple (BCP), are used to test albuminwith 2.5% low biased results from BCP compared with BCG, comparingeach laboratory with its specific peer group. Practically all proceduresusing BCG are traceable to the ERM-DA 470 reference material, and allusing BCP are traceable to the NIST-SRM 927c. The percentage devia-tions compared with the corresponding peer group mean fall withinthe acceptable limit in the majority of cases. One laboratory using BCGand a routine calibrator traceable to ERM-DA 470 obtained low resultsover the whole concentration range studied.

3.3.3.2. Other analytes. When testing alkaline phosphatase, inorganicphosphate, lipase, triglyceride and urea, all participantmetrological pro-cedures are based on the same method and, therefore, are comparedwith the overall mean for each analyte.

Although traceability of routine calibrators is different, percentagedeviation of results versus the overall mean fall within the acceptablelimits, a good harmonization among users of the same method whentesting these analytes.


Lastly, in the case of iron procedures used in this study, all have thesame metrological basis (reduction of Fe3+ to Fe2+ and colorimetricreading of the complex formed) and, for this reason, the target valueis the overall mean. Percentage deviations obtained reach the optimumacceptability limit, so a good harmonization for iron procedures isdemonstrated.

ED P

RO

OF

4. General discussion

The results show that one of the most likely explanations for unac-ceptable performance is the lack of commutability of the reference ma-terial used to assign the value of a routine calibrator. Detailedinformation about the procedure used to assign routine calibratorvalues is often not provided by manufacturers, and laboratories cannotknow if the traceability chain has been entirelymaintained. For this rea-son, several authors recommend using reference methods instead ofreference materials to assign the value of the routine calibrators, be-cause commutability of reference materials with human serum for thedifferent routine procedures cannot always be ensured [2,30]. Also itwas shown in this work that lack of commutability of the routine cali-brators may be a reason for non-standardization [2,31–35], becausethe physical traceability chain is broken. An example is with calciumtesting,with the possibility of obtaining false low resultswhen themea-surement procedure is traceable to a non-tested commutable referencematerial. This constitutes a serious problem, aswithout this informationand despite operating according to the manufacturer's specifications,the laboratory could produce false results. As manufacturers are obligedby the European In Vitro Diagnostics Directive [36] to provide the cus-tomer with this information, any error of this kind is themanufacturer'sresponsibility.

Failure of standardization may have an important repercussion onpatient healthcare. A clear example is the non-compensated Jaffé kinet-ic procedures and those not traceable to IDMS, which produce high re-sults at concentration close to the clinical decision level and a falsenumber of GFR results below the cut-off point. In this way, erroneousdiagnosis of chronic renal disease could happen, with the consequentharm for patients and increase of healthcare costs. The use of the differ-ent formulae for estimation of GFR, based on routine calibrator trace-ability might not be enough to harmonize results at the clinicaldecision level [37].

This lack of harmonization has been recently studied by Panteghini,who considers the need to urgently revise clinical decision limits in ashared task with laboratory professionals, IVD manufacturers, EQA or-ganizers and clinicians [38]. Whereas in this pilot study use of commut-able EQAs has never given different results among instruments usingthe same method, the same cannot be said for the SEQC-EQA withnon-commutable EQAs where important discrepancies among instru-ments using the same method can be seen [39]. This is another reasonthat obliges us to move towards a higher category of EQA.

In summary, results obtained in this work demonstrate the impor-tant role of external quality assurance programs using commutableEQAs with values assigned by reference methods to correctly monitorperformance in laboratory medicine and, consequently, to correctlyidentify patient risk. The main inconvenience for its implementation isthe economic cost. This difficulty could be minimized by integratingthis type of EQA material with limited frequency in one of the cyclesof the current program. Our organization is firmly committed to pro-moting this activity within our country.

Supplementary data to this article can be found online at http://dx.doi.org/10.1016/j.cca.2013.11.005.

5. Uncited references

[22,23,24,25,26,27,28,29]





T

433

434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473Q8474475476477478479480481482483484485486487

488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543

544


REC

References

[1] Report of the review of NHS pathology services in England. An independent reviewfor the Department of Health. London: Department of Health; 2006 6.

[2] Miller GW, Myers GL, Gantzer ML, et al. Roadmap for harmonization of clinical lab-oratory measurement procedures. Clin Chem 2012;57:1108–17.

[3] Siekmann L. Metrological traceability — a concept for standardization in laboratorymedicine. Clin Chem Lab Med 2013;51:953–7.

[4] Panteghini M. Traceability as a unique tool to improve standardization in laboratorymedicine. Clin Biochem 2009;42:236–40.

[5] Asociación Española de Normalización y Certificación. In vitro diagnosis medical de-vices — measurement of quantities in biological samples— metrological traceabilityof values assigned to calibrators and control materials UNE-EN ISO 17511; 2004.

[6] Maziotta D, Harel D, Schumann G, Libeer JC. Guidelines for the requirements for thecompetence of EQAP organizers in medical laboratories. International Federation ofClinical Chemistry (IFCC)/Education and Management Division (EMD)/Committeeof Analytical Quality (C-AQ); 2002.

[7] Miller GW, Jones GRD, Horowitz GL,Weycamp C. Proficiency testing/external qualityassessment current challenges and future directions. Clin Chem 2011;57:1670–80.

[8] Ramón F, Alsina MJ, Álvarez V, et al. XXXII Programa de Garantía Externa de laCalidad de Bioquímica (suero) de la Sociedad Española de Bioquímica Clínica yPatología Molecular. In: http://www.contcal.org; 2011 . [Accessed 3-11-12].

[9] Jansen RTP. The quest for comparability: Calibration 2000. Accred Qual Assur2000;5:363–6.

[10] Jansen RTP, Kuypers AWHM, Baadenhuijsen H, et al. Kalibratie 2000. Ned TijdschrKlin Chem 2000;25:153–8.

[11] Baadenhuijsen H, Scholten R, Willems HL, Weykamp CW, Jansen RTP. A model forharmonization of routine clinical chemistry results between clinical laboratories.Ann Clin Biochem 2000;37:330–7.

[12] Baadenhuijsen H, Kuypers A, Weykamp C, Cobbaert C, Jansen R. External quality as-sessment in The Netherlands: time to introduce commutable survey specimens. Les-sons from the Dutch “Calibration 2000” project. Clin Chem Lab Med 2005;43:304–7.

[13] Baadenhuijsen H, Steigstra H, Cobbaert C, Kuypers A, Weykamp C, Jansen R.Commutability assessment of potential reference materials using a multicentersplit-patient-sample between-field-methods (twin-study) design: study withinthe framework of the Dutch project “Calibration 2000”. Clin Chem 2002;48:1520–5.

[14] Cobbaert C, Weykamp C, Franck P, et al. Systematic monitoring of standardizationand harmonization status with commutable EQA-samples — five year experiencefrom The Netherlands. Clin Chim Acta 2012;414:234–40.

[15] Jansen R, Jassam N, Thomas A, et al. A category 1 EQA scheme for comparison of lab-oratory performance and method performance: an international pilot study in theFramework of theCalibration 2000project. Clin ChimActa 2013 [in this issue of CCA].

[16] Minchinela J, Ricós C, Perich C, et al. Biological variation database, and quality specifica-tions for imprecision, bias and total error (desirable, optimum and minimum). The2012 update. http://www.westgard.com/biodatabase1/htm . [accessed 26-11-2012].

[17] Fraser CG, Hyltoft Petersen P, Libeer JC, Ricós C. Proposals for setting generally appli-cable quality goals solely based on biology. Ann Clin Biochem 1997;34:8–12.

[18] Jansen R, Schumann G, Baadenhuijsen H, et al. Trueness verification and traceabilityassessment of results from commercial systems for measurement of six enzyme ac-tivities in serum. An international study in the EC4 framework of the Calibration2000 project. Clin Chim Acta 2006;368:160–7.

[19] Infusino I, Schumann G, Ceriotti F, Panteghini M. Standardization in clinical enzy-mology: a challenge for the theory of metrological traceability. Clin Chem Lab Med2010;48:301–7.

[20] Comisión de Función Renal de la SEQC. Ecuaciones de estimación del filtradoglomerular a utilizar según combinación procedimiento de medida-analizador.

UNCO

R


ED P

RO

OF

http://www.seqc.es/es/Comisiones/18/22/279/Noticias_y_Anunciosde_la_Comision_de_Funcion_renal_%7C_Comite_Cientifico/ . [accessed 26-11-2012].

[21] Delanghe JR, Cobbaert C, Harmoinen A, Jansen R, Laitinen P, Panteghini M. Focusingon the clinical impact of standardization of creatinine measurements: a report bythe EFCC Working Group on Creatinine Standardization. Clin Chem Lab Med2011;49:977–82.

[22] Kenny D, Fraser CG, Hyltoft Petersen P, Kallner A. Consensus agreement. In: HyltoftPetersen P, Fraser CG, Kallner A, Kenny D, editors. Strategies to set global analyticalquality specifications in laboratory medicineScan J Clin Lab Invest 1999;59(7):585.

[23] Ricós C, Baadenhuijsen H, Libeer JC, et al. Currently used criteria for evaluating per-formance in EQA in European countries and a proposal for harmonization. Eur J ClinChem Clin Biochem 1996;34:159–65.

[24] IFCC/EMD/C-AQ:2002. Guidelines for the requirements for the competence of EQAP or-ganizers in medical laboratories. http://www.ifcc.org/ifccfiles/docs/EQAP_version_3-2002.pdf . [accessed 26-11-2012].

[25] Cobbaert C, Weycamp C, Franck P, et al. Systematic monitoring of standardizationand harmonization status with commutable EQA samples: five year experiencefrom The Netherlands. Clin Chim Acta 2012;414:234–40.

[26] Sikaris K. Analytical quality — what should we be aiming for? Clin Biochem Rev2008;29:S5–S10.

[27] CLSI. Verification of comparability of patient results within one health care system;proposed guideline. CLSI document C54-P. Wayne, (PA): CLSI; 2007.

[28] SEQC, Comité de Garantía de la Calidad, Comisión de Calidad Analítica.Recomendación para el uso de las especificaciones de la calidad analítica notificadasen el informe mensual de un programa de garantía externa de la calidad de la SEQC.http://www.seqc.es/es/Comisiones/7/51/102.

[29] Infusino I, Panteghini M. Serum albumin: accuracy and clinical use. Clin Chim Acta2013;419:15–8.

[30] Cattozzo G, Guerra E, Ceriotti F, Franzini C. Commutable calibrator with valueassigned by the IFCC reference procedure to harmonize serum lactate dehydroge-nase activity results measured by 2 different methods. Clin Chem 2008;54:1349–55.

[31] Canalías F, García E, Sánchez M. Metrological traceability of values for α-amylasecatalytic concentration assigned to a commutable calibrator materials. Clin ChimActa 2010;411:7–12.

[32] Miller WG, Myers GL, Rej R. Why commutability matters. Clin Chem 2006;52:553–4.[33] Vesper HW, Miller WG, Myers GR. Reference materials and commutability. Clin

Biochem Rev 2007;28:139–47.[34] CLSI. Characterization and qualification of commutable reference materials for labo-

ratory medicine. Approved guideline. CLSI document C53-A. Wayne, (PA): CLSI;2010.

[35] Franzini C, Ceriotti F. Impact of referencematerials on accuracy in, clinical chemistry.Clin Biochem 1998;31:449–57.

[36] Directive 98/79/EC of the European Parliament and of the Council of 27 October1998 on in vitro diagnostic medical devices. Off J Eur Communities 7 December1998:L331.

[37] Alvarez V, Ricós C, Fernández-Calle P, et al. Impacto de la trazabilidad de los métodosde creatinina en la estimación del filtrado glomerular. Abstract al VI CongresoNacional de Laboratorio Clínico; Octubre 2012 [Barcelona, https://intranet.pacifico-eetings.com/amsysweb/publicacionOnline.jsf?id=95 (accessed 26-11-2012)].

[38] Panteghini M. Implementation of standardization in clinical practice: not always aneasy task. Clin Chem Lab Med 2012;50:1237–41.

[39] SEQC, Comité de Garantía de la Calidad. XXXIII programa de Garantía externade la Calidad de Bioquímica (suero) de la Sociedad Española de BioquímicaClínica y Patología Molecular. http://www.contcal.org/qcweb/Documents/40%20Avaluacio%20anual/50%20Programes%202012/40.%20Suero.pdf; 2012 .[accessed 01-06-2013].


http://www.contcal.org

http://www.westgard.com/biodatabase1/htm

http://www.seqc.es/es/Comisiones/18/22/279/Noticias_y_Anunciosde_la_Comision_de_Funcion_renal_%7C_Comite_Cientifico/

http://www.seqc.es/es/Comisiones/18/22/279/Noticias_y_Anunciosde_la_Comision_de_Funcion_renal_%7C_Comite_Cientifico/

http://www.ifcc.org/ifccfiles/docs/EQAP_version_3-2002.pdf

http://www.ifcc.org/ifccfiles/docs/EQAP_version_3-2002.pdf

http://www.seqc.es/es/Comisiones/7/51/102

https://intranet.pacifico-eetings.com/amsysweb/publicacionOnline.jsf?id=95

https://intranet.pacifico-eetings.com/amsysweb/publicacionOnline.jsf?id=95

http://www.contcal.org/qcweb/Documents/40%20Avaluacio%20anual/50%20Programes%202012/40.%20Suero.pdf

http://www.contcal.org/qcweb/Documents/40%20Avaluacio%20anual/50%20Programes%202012/40.%20Suero.pdf
