diagnostics 1 methods comparison studies for quantitative nucleic acid assays jacqueline law, art...
TRANSCRIPT
Dia
gn
osti
cs
1
Methods Comparison Studies for Quantitative
Nucleic Acid Assays
Jacqueline Law, Art DeVaultRoche Molecular Systems
Sept 19, 2003
Dia
gn
osti
cs
2
OutlineOutline
IntroductionPCR based quantitative nucleic acid
assaysLiterature referencesAcceptance criteriaExamplesReferences
IntroductionPCR based quantitative nucleic acid
assaysLiterature referencesAcceptance criteriaExamplesReferences
Dia
gn
osti
cs
3
Methods Comparison Studies: to validate a new assayMethods Comparison Studies: to validate a new assay
Purposes:To show that the new assay has good
agreement with the reference assaysTo show that the assay performs similarly with
different types of specimen
Premises of methods comparison studies:A linear relationship between the two assays LOD, dynamic range have to be already
established Appropriate transformation to normalize the
data
Analysis: To detect constant bias and proportional bias
Purposes:To show that the new assay has good
agreement with the reference assaysTo show that the assay performs similarly with
different types of specimen
Premises of methods comparison studies:A linear relationship between the two assays LOD, dynamic range have to be already
established Appropriate transformation to normalize the
data
Analysis: To detect constant bias and proportional bias
Dia
gn
osti
cs
4
Constant Bias: the difference between the two methods is constant across the data range
Constant Bias: the difference between the two methods is constant across the data range
Method X
Me
tho
d Y
2 3 4 5 6 7 8
34
56
78
Method Y vs. Method X
Average
Diff
ere
nce
3 4 5 6 7 8
-2-1
01
2
Difference vs. Average
Dia
gn
osti
cs
5
Proportional Bias: the difference between the two methods is linear across the data range
Proportional Bias: the difference between the two methods is linear across the data range
Method X)
Me
tho
d Y
3 4 5 6 7
34
56
78
Method Y vs. Method X
Average
Diff
ere
nce
3 4 5 6 7 8
-0.5
0.0
0.5
1.0
Difference vs. Average
Dia
gn
osti
cs
6
PCR based nucleic acid assaysPCR based nucleic acid assays
To quantify the viral load by PCR method
Characteristics:A wide dynamic range (e.g. 10cp/mL to
1E7 cp/mL)Skewed distribution (non-normal):
typically log10 transformation for the dataHeteroscedasticity: variance is higher at
higher titer levels log10 transformation may not achieve
homogeneity in variance (variance at lower end may increase)
Other transformation:
To quantify the viral load by PCR method
Characteristics:A wide dynamic range (e.g. 10cp/mL to
1E7 cp/mL)Skewed distribution (non-normal):
typically log10 transformation for the dataHeteroscedasticity: variance is higher at
higher titer levels log10 transformation may not achieve
homogeneity in variance (variance at lower end may increase)
Other transformation: 2log 2 xx
Dia
gn
osti
cs
7
PCR based assays: a wide dynamic range - data are log10 transformed
PCR based assays: a wide dynamic range - data are log10 transformed
Nominal Titer (cp/mL)
Observ
ed T
iter
(cp/m
L)
0 2*10^7 6*10^7 10^8 1.4*10^8
05*1
0^7
10^8
1.5
*10^8
2*1
0^8
2.5
*10^8
3*1
0^8
Untransformed Data
Nominal Titer (Log cp/mL)
Observ
ed T
iter
(Log c
p/m
L)
2 4 6 8
-4-2
02
46
8
Log10 Titer
Dia
gn
osti
cs
8
PCR based assays: log10 transformation may remove some skewness
PCR based assays: log10 transformation may remove some skewness
Untransformed
log10 transformed
0 2 4 6 8 10
0.0
0.1
0.2
0.3
0.4
Titer
7 cp/mL
20 40 60 80 100 120 140
0.0
0.01
00.
020
0.03
0
Titer
7.2E1 cp/mL
6*10^6 10^7 1.4*10^7
05*
10^-
81.
5*10
^-7
Titer
1.4E7 cp/mL
-1.5 -0.5 0.0 0.5 1.0
0.0
0.2
0.4
0.6
0.8
Titer
7 cp/mL
1.4 1.6 1.8 2.0 2.2
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Titer
7.2E1 cp/mL
6.8 6.9 7.0 7.1 7.2
01
23
45
Titer
1.4E7 cp/mL
Dia
gn
osti
cs
9
Literature references on Methods Comparison StudiesLiterature references on Methods Comparison Studies
Correlation coefficientOther coefficientsT-testBland-Altman plotOrdinary least squares regressionPassing-Bablok regressionDeming regression
Correlation coefficientOther coefficientsT-testBland-Altman plotOrdinary least squares regressionPassing-Bablok regressionDeming regression
Dia
gn
osti
cs
10
Correlation coefficient R or R2Correlation coefficient R or R2
Measures the strength of linear relationship between two assays
Does not measure agreement: cannot detect constant or proportional bias
Correlation coefficient can be artificially high for assays that cover a wide range: how high is high? 0.95? 0.99? 0.995?
Measures the strength of linear relationship between two assays
Does not measure agreement: cannot detect constant or proportional bias
Correlation coefficient can be artificially high for assays that cover a wide range: how high is high? 0.95? 0.99? 0.995?
Dia
gn
osti
cs
11
Other coefficientsOther coefficients
Concordance coefficient (Lin, 1989):Measures the strength of relationship
between two assays that fall on the 45o line through the origin
Gold-standard correlation coefficient (St.Laurent 1998):Measures the agreement between a new
assay and a gold standard
Concordance coefficient (Lin, 1989):Measures the strength of relationship
between two assays that fall on the 45o line through the origin
Gold-standard correlation coefficient (St.Laurent 1998):Measures the agreement between a new
assay and a gold standard
22122
21
212
C
GGDD
GGG SS
S
Dia
gn
osti
cs
12
T-testT-test
Paired t-test on the difference in the measurements by two assays
Can only detect constant biasCannot detect proportional bias
Paired t-test on the difference in the measurements by two assays
Can only detect constant biasCannot detect proportional bias
Dia
gn
osti
cs
13
Bland-Altman graphical analysis(Bland and Altman, 1986)Bland-Altman graphical analysis(Bland and Altman, 1986)
Methods:Plot the Difference of the two assays (D = X-Y)
vs. the Average of the two assays (A = (X+Y)/2)Visually inspect the plot and see if there are
any trends in the plot proportional biasSummarize the bias between the two assays by
the mean, SD, 95% CI constant biasModification: regress D with A, test if slope = 0
(Hawkins, 2002)
A useful visual tool: transformation, heteroscedasticity, outliers,
curvature
Methods:Plot the Difference of the two assays (D = X-Y)
vs. the Average of the two assays (A = (X+Y)/2)Visually inspect the plot and see if there are
any trends in the plot proportional biasSummarize the bias between the two assays by
the mean, SD, 95% CI constant biasModification: regress D with A, test if slope = 0
(Hawkins, 2002)
A useful visual tool: transformation, heteroscedasticity, outliers,
curvature
Dia
gn
osti
cs
14
Bland Altman plot (continued)Bland Altman plot (continued)
Method X (log Titer)
Me
tho
d Y
(lo
g T
iter)
2 4 6 8
24
68
Method Y vs. Method X
Average (log Titer)
Diff
ere
nce
(lo
g T
iter)
3 4 5 6 7 8
-0.5
0.0
0.5
1.0
Difference vs. Average
Dia
gn
osti
cs
15
Ordinary least-squares regressionOrdinary least-squares regression
Methods: Regress the observed data of the new assay
(Y) with those of the reference assay (X)Minimize the squared deviations from the
identity line in the vertical directionModifications: weighted least squares
Assumptions:The reference assay (X) is error free, or the
error is relatively small compared to the range of the measurements
e.g. in clinical chemistry studies, the measurement errors are minimal
Methods: Regress the observed data of the new assay
(Y) with those of the reference assay (X)Minimize the squared deviations from the
identity line in the vertical directionModifications: weighted least squares
Assumptions:The reference assay (X) is error free, or the
error is relatively small compared to the range of the measurements
e.g. in clinical chemistry studies, the measurement errors are minimal
Dia
gn
osti
cs
16
Ordinary least-squares regression (continued)
Ordinary least-squares regression (continued)
If measurement errors exist in both assays, the estimates are biasedslope tends to be smallerintercept tends to be larger
If measurement errors exist in both assays, the estimates are biasedslope tends to be smallerintercept tends to be larger
Dia
gn
osti
cs
17
Passing-Bablok regression (Passing and Bablok, 1983)Passing-Bablok regression (Passing and Bablok, 1983)
A nonparametric approach - robust to outliersMethods:
Estimate the slope by the shifted median of the slopes between all possible sets of two points (Theil estimate)
Confidence intervals by the rank techniques
Assumptions:The measurement errors in both assays follow the
same type of distribution (not necessarily normal)The ratio of the variance is a constant (variance not
necessarily constant across the range of data)The sampling distributions of the samples are
arbitrary
A nonparametric approach - robust to outliersMethods:
Estimate the slope by the shifted median of the slopes between all possible sets of two points (Theil estimate)
Confidence intervals by the rank techniques
Assumptions:The measurement errors in both assays follow the
same type of distribution (not necessarily normal)The ratio of the variance is a constant (variance not
necessarily constant across the range of data)The sampling distributions of the samples are
arbitrary
Dia
gn
osti
cs
18
Deming regression(Linnet, 1990)Deming regression(Linnet, 1990)
Methods:Orthogonal least squares estimates: minimize the
squared deviation of the observed data from the regression line
Standard errors for the estimates obtained by Jackknife method
Weighted Deming regression when heteroscedastic
Assumptions:Measurement errors for both assays follow
independent normal distributions with mean 0Error variances are assumed to be proportional
(variance not necessarily constant across the range of data)
Methods:Orthogonal least squares estimates: minimize the
squared deviation of the observed data from the regression line
Standard errors for the estimates obtained by Jackknife method
Weighted Deming regression when heteroscedastic
Assumptions:Measurement errors for both assays follow
independent normal distributions with mean 0Error variances are assumed to be proportional
(variance not necessarily constant across the range of data)
Dia
gn
osti
cs
19
Comparison of the 3 regression methods(Linnet, 1993)
Comparison of the 3 regression methods(Linnet, 1993)
Electrolyte study (homogeneous variance):OLS, Passing-Bablok: biased slope, large Type I
error, larger RMSE than DemingDeming: unbiased slope, correct Type I error
Metabolite study (heterogeneous variance):All have unbiased slope estimatesWeighted LS and weighted Deming are most
efficientType I error is large for OLS, weighted LS,
Deming and Passing-Bablok
Presence of outliers:Passing-Bablok is robust to outliersDeming regression requires detection of outliers
Electrolyte study (homogeneous variance):OLS, Passing-Bablok: biased slope, large Type I
error, larger RMSE than DemingDeming: unbiased slope, correct Type I error
Metabolite study (heterogeneous variance):All have unbiased slope estimatesWeighted LS and weighted Deming are most
efficientType I error is large for OLS, weighted LS,
Deming and Passing-Bablok
Presence of outliers:Passing-Bablok is robust to outliersDeming regression requires detection of outliers
Dia
gn
osti
cs
20
SoftwareSoftware
Statistical packages: SAS, Splus
Other packages (for Bland-Altman plot, OLS regression, Passing-Bablok regression, Deming regression):Analyse-it (Excel add-on): does not
support weighted Deming regressionMethod Validator (a freeware)CBStat (Linnet K.)
Statistical packages: SAS, Splus
Other packages (for Bland-Altman plot, OLS regression, Passing-Bablok regression, Deming regression):Analyse-it (Excel add-on): does not
support weighted Deming regressionMethod Validator (a freeware)CBStat (Linnet K.)
Dia
gn
osti
cs
21
Acceptance criteria for regression type analysisAcceptance criteria for regression type analysis
Independent acceptance criteria for slope and intercept estimates:e.g. slope estimate within (0.9, 1.1),
intercept estimate within (-0.2, 0.2)
Drawback: asymmetrical acceptance region across the data range
Independent acceptance criteria for slope and intercept estimates:e.g. slope estimate within (0.9, 1.1),
intercept estimate within (-0.2, 0.2)
Drawback: asymmetrical acceptance region across the data range
Dia
gn
osti
cs
22
Asymmetrical acceptance regionAsymmetrical acceptance region
Method X (Log Titer)
Me
tho
d Y
(Lo
g T
iter)
2 4 6 8
24
68
Y = 0.2 + 1.1 * X
Slope=(0.9, 1.1), Int=(-0.2,0.2)
Y = -0.2 + 0.9 * X
Method X (Log Titer)
Bia
s =
Me
tho
d Y
- M
eth
od
X (
Lo
g T
iter)
2 4 6 8
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
Asymmetrical Acceptance Region
Dia
gn
osti
cs
23
Proposed acceptance criteriaProposed acceptance criteria
Goals: to show that the new assay is ‘equivalent’
to the reference assayto demonstrate that the bias between the
two assays is within some acceptable threshold across the clinical range
Acceptance Criteria:
Choice of tolerance level A: accuracy specification for the new assay
Goals: to show that the new assay is ‘equivalent’
to the reference assayto demonstrate that the bias between the
two assays is within some acceptable threshold across the clinical range
Acceptance Criteria:
Choice of tolerance level A: accuracy specification for the new assay
AXYEBiasE
Dia
gn
osti
cs
24
Mathematical modelsMathematical models
Reference Assay:
New Assay:
where is the true concentration,
and are the independent random measurement errors
Bias:
i i i
i i i
i
i i
i i
X
Y
Y X
1
Acceptance Criteria: 1
i i i
i i iE Y X A
Dia
gn
osti
cs
25
Comparison of the acceptance criteria:{Int (-0.2,0.2), Slope (0.9,1.1) } vs. { A= 0.5, L=2, U=7}
Comparison of the acceptance criteria:{Int (-0.2,0.2), Slope (0.9,1.1) } vs. { A= 0.5, L=2, U=7}
Method X (Log Titer)
Me
tho
d Y
(L
og
Tite
r)
2 3 4 5 6 7
23
45
67
Acceptance Region for the Data
Method X (Log Titer)
Bia
s: M
eth
od
Y -
Me
tho
d X
(L
og
Tite
r)
2 3 4 5 6 7
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
Symmetrical Region
Dia
gn
osti
cs
26
Acceptance region for the parameters:criteria for the intercept and slope are dependent
Acceptance region for the parameters:criteria for the intercept and slope are dependent
Intercept (Alpha)
Slo
pe
(B
eta
)
-0.5 0.0 0.5
0.8
0.9
1.0
1.1
1.2
Acceptance Region for the Parameters
Dia
gn
osti
cs
27
Equivalence testEquivalence test
Methods:If the 90% two-sided confidence interval of
the Bias lies entirely within the acceptance region (- A, A), then the two assays are equivalent
Deming-Jackknife is used to do the estimation
Methods:If the 90% two-sided confidence interval of
the Bias lies entirely within the acceptance region (- A, A), then the two assays are equivalent
Deming-Jackknife is used to do the estimation
0 : vs. :aH Bias A H Bias A
where A is the accuracy specification of the new assay
Dia
gn
osti
cs
28
Deming regression:(a.k.a. errors-in-variables regression, a structural or functional relationship model)
Deming regression:(a.k.a. errors-in-variables regression, a structural or functional relationship model)
Minimize the sum of squares:Minimize the sum of squares:
The solutions are given by:The solutions are given by:
2 2
1
n
i i i ii
S x y
2 21ˆ 42
ˆˆ
yy xx xx yy xyxy
S S S S SS
y x
where = Var()/Var() (assumed known or to be estimated)
Weighted Deming regression:Weighted Deming regression:
22 ˆˆ
11
iiii
YXSDw
Dia
gn
osti
cs
29
Estimation of in Deming regressionEstimation of in Deming regression
Duplicate measurements:
>2 replicates: residual errors by ANOVA
Mis-specification of (Linnet 1998):biased slope estimatelarge Type I error
Duplicate measurements:
>2 replicates: residual errors by ANOVA
Mis-specification of (Linnet 1998):biased slope estimatelarge Type I error
2
2
221
2221
2
ˆ
2
1 ,
2
1
Y
X
iiYiiX
SD
SD
yyN
SDxxN
SD
Dia
gn
osti
cs
30
Jackknife estimation: to obtain the final parameter estimates and the SEs
Jackknife estimation: to obtain the final parameter estimates and the SEs
Omit one pair of data at a time, obtain the Deming-regression estimates:
Omit one pair of data at a time, obtain the Deming-regression estimates:
The ith pseudo-values of the intercept and slope are:The ith pseudo-values of the intercept and slope are:
ˆ ˆ1
ˆ ˆ1
i i
i i
n n
n n
Final estimates and SEs for and are the mean and standard error of i and i
Final estimates and SEs for and are the mean and standard error of i and i
ii ˆ ,ˆ
Dia
gn
osti
cs
31
Bias estimation by JackknifeBias estimation by Jackknife
( ) ( )1 1 1ii iBias n n
The bias estimate and the SE at each nominal level are the mean and SE of Biasi
The 90% CI of the bias at each nominal level are compared to the acceptance region (-A, A)
The two assays are concluded to be equivalent if all the CI lie entirely within (-A, A)
The bias estimate and the SE at each nominal level are the mean and SE of Biasi
The 90% CI of the bias at each nominal level are compared to the acceptance region (-A, A)
The two assays are concluded to be equivalent if all the CI lie entirely within (-A, A)
At each nominal level , the ith pseudo-value of the Bias is:
At each nominal level , the ith pseudo-value of the Bias is:
Dia
gn
osti
cs
32
Example 1: methods comparison for two HIV-1 assays
Example 1: methods comparison for two HIV-1 assays
Reference Method (log Titer)
New
Met
hod
(log
Tite
r)
3 4 5 6
34
56
Methods Comparison Study
Dia
gn
osti
cs
33
Bland-Altman plot: potential outliers in the dataBland-Altman plot: potential outliers in the data
Average of Reference and New (log Titer)
Diff
ere
nce
= N
ew
- R
efe
ren
ce (
log
Tite
r)
3 4 5 6
-0.5
0.0
0.5
1.0
Bland-Altman Plot: Full Data Set
Dia
gn
osti
cs
34
Identify outliers: fitting a linear regression line to the Bland Altman plotIdentify outliers: fitting a linear regression line to the Bland Altman plot
Fitted Difference
Stu
dent
ized
Res
idua
l
0.0 0.01 0.02 0.03 0.04
-3-2
-10
12
3
Residual Plot for Diff vs. Avg
34860
34794
Samples
Leve
rage
0 10 20 30 40 50
0.02
0.04
0.06
0.08
0.10
34944
34851
Leverage
Dia
gn
osti
cs
35
Remove outliers: Bland-Altman plot shows no trend in Difference vs. AverageRemove outliers: Bland-Altman plot shows no trend in Difference vs. Average
Average of Reference and New (log Titer)
Diff
eren
ce =
New
- R
efer
ence
(lo
g T
iter)
3 4 5 6
-0.6
-0.4
-0.2
0.0
0.2
0.4
Bland-Altman Plot
slope = 0.033 (p-value = 0.5)
mean difference = 0.02
(95% CI: -0.06, 0.10)
Dia
gn
osti
cs
36
Regression analysis: results from the 3 methods are very similar
Regression analysis: results from the 3 methods are very similar
Reference Method (log Titer)
New
Met
hod
(log
Tite
r)
3 4 5 6
34
56
Regression Analysis
OLS: Y=0.084+0.986X Passing-Bablok: Y=-0.137+1.044XDeming-Jackknife: Y=-0.127+1.033X
Dia
gn
osti
cs
37
Bias estimation: almost all 90% CI lie within the tolerance bounds (-0.2, +0.2)Bias estimation: almost all 90% CI lie within the tolerance bounds (-0.2, +0.2)
Reference Method (log Titer)
Diff
eren
ce =
New
- R
efer
ence
(lo
g T
iter)
3 4 5 6
-0.6
-0.4
-0.2
0.0
0.2
0.4
Estimated Bias (90% CI) by Deming-Jackknife Regression
Dia
gn
osti
cs
38
Example 2: to show matrix equivalency between EDTA Plasma and SerumExample 2: to show matrix equivalency between EDTA Plasma and Serum
EDTA (log Titer)
Ser
um (
log
Tite
r)
2 3 4 5 6 7 8
23
45
67
8
Matrix Equivalency Study
Dia
gn
osti
cs
39
Bland-Altman plot on average titer:most titers higher than 1E5 IU/mL, heteroscedasticity?
Bland-Altman plot on average titer:most titers higher than 1E5 IU/mL, heteroscedasticity?
Average of EDTA and Serum (log Titer)
Diff
ere
nce
= S
eru
m -
ED
TA (
log
Tite
r)
3 4 5 6 7
-1.0
-0.5
0.0
0.5
Bland-Altman Plot
slope = 0.03 (p-value = 0.6)
mean difference = -0.06 (95% CI: -0.16, 0.04)
Dia
gn
osti
cs
40
Checking for heteroscedasticity:residual errors from random effects models
Checking for heteroscedasticity:residual errors from random effects models
Mean EDTA (log Titer)
SD
ED
TA
(lo
g T
ite
r)
3 4 5 6 7
0.0
20
.04
0.0
60
.08
0.1
00
.12
0.1
4
EDTA
Mean Serum (log Titer)
SD
Se
rum
(lo
g T
ite
r)
3 4 5 6 7
0.0
20
.04
0.0
60
.08
0.1
00
.12
0.1
4
Serum
Dia
gn
osti
cs
41
1: Pooled within-sample SD for EDTA = 0.0706Pooled within-sample SD for Serum = 0.0715
1: Pooled within-sample SD for EDTA = 0.0706Pooled within-sample SD for Serum = 0.0715
Average of EDTA and Serum (log Titer)
Va
r(E
DTA
Err
ors
)/V
ar(
Se
rum
Err
ors
)
3 4 5 6 7
02
46
8 Median Lambda = 0.945
Lambda Estimation
Dia
gn
osti
cs
42
Bias estimation: large variability at low titers due to sparse data - fail to demonstrate equivalency at low end
Bias estimation: large variability at low titers due to sparse data - fail to demonstrate equivalency at low end
EDTA (log Titer)
Diff
eren
ce =
Ser
um -
ED
TA (
log
Tite
r)
3 4 5 6 7
-1.0
-0.5
0.0
0.5
Estimated Bias (90% CI) by Deming-Jackknife Regression
Dia
gn
osti
cs
43
ReferencesReferences
Bland M., Altman D. (1986). ‘Statistical methods for assessing agreement between two methods of clinical measurement’. Lancet 347: 307-310.
Hawkins D. (2002). ‘Diagnostics for conformity of paired quantitative measurements’. Stat in Med 21: 1913-1935.
Lin L.K. (1989). ‘A concordance correlation coefficient to evaluate reproducibility’. Biometrics 45: 255-268.
Linnet K. (1990). ‘Estimation of the linear relationship between the measurements of two methods with proportional bias’. Stat in Med 9: 1463-1473.
Linnet K. (1993). ‘Evaluation of regression procedures for methods comparison studies’. Clin Chem 39: 424-432.
Linnet K. (1998). ‘Performance of Deming regression analysis in case of misspecified analytical error ratio in method comparisons studies’. Clin Chem 44: 1024-1031.
Linnet K. (1999). ‘Necessary sample size for method comparison studies based on regression analysis’. Clin Chem 45: 882-894.
Bland M., Altman D. (1986). ‘Statistical methods for assessing agreement between two methods of clinical measurement’. Lancet 347: 307-310.
Hawkins D. (2002). ‘Diagnostics for conformity of paired quantitative measurements’. Stat in Med 21: 1913-1935.
Lin L.K. (1989). ‘A concordance correlation coefficient to evaluate reproducibility’. Biometrics 45: 255-268.
Linnet K. (1990). ‘Estimation of the linear relationship between the measurements of two methods with proportional bias’. Stat in Med 9: 1463-1473.
Linnet K. (1993). ‘Evaluation of regression procedures for methods comparison studies’. Clin Chem 39: 424-432.
Linnet K. (1998). ‘Performance of Deming regression analysis in case of misspecified analytical error ratio in method comparisons studies’. Clin Chem 44: 1024-1031.
Linnet K. (1999). ‘Necessary sample size for method comparison studies based on regression analysis’. Clin Chem 45: 882-894.
Dia
gn
osti
cs
44
References (continued)References (continued)
Passing H., Bablok W. (1983). ‘A new biometrical procedure for testing the equality of measurements from two different analytical methods’. J Clin Chem Clin Biochem 21: 709-720.
Passing H., Bablok W. (1984). ‘Comparison of several regression procedures for method comparison studies and determination of sample sizes’. J Clin Chem Clin Biochem 22: 431-445.
St. Laurent R.T. (1998). ‘Evaluating Agreement with a Gold Standard in Method Comparison Studies’. Biometrics 54: 537-545.
Passing H., Bablok W. (1983). ‘A new biometrical procedure for testing the equality of measurements from two different analytical methods’. J Clin Chem Clin Biochem 21: 709-720.
Passing H., Bablok W. (1984). ‘Comparison of several regression procedures for method comparison studies and determination of sample sizes’. J Clin Chem Clin Biochem 22: 431-445.
St. Laurent R.T. (1998). ‘Evaluating Agreement with a Gold Standard in Method Comparison Studies’. Biometrics 54: 537-545.