so how good are your results? (an introduction to quantitative qc) graham jones chemical pathology...
TRANSCRIPT
![Page 1: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/1.jpg)
So how good are your results?
(An introduction to quantitative QC)
Graham JonesChemical Pathology
St Vincent’s Hospital, Sydney
![Page 2: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/2.jpg)
Overview
• Standard QC• Quantitative QC
– How good are we?– How good do we need to be?
• Bias
• A short course in what we will need to know
![Page 3: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/3.jpg)
Quality System
• Staff– Choosing– Training
• Instruments– Choosing– Optimising– Maintaining
• Sample management
• QC– Planning– Performing– Responding
• Quality Assurance– Performance– Interpretation– Action
• Result management
![Page 4: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/4.jpg)
Quality System
• Staff– Choosing– Training
• Instruments– Choosing– Optimising– Maintaining
• Sample management
• QC– Planning– Performing– Responding
• Quality Assurance– Performance– Interpretation– Action
• Result management
![Page 5: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/5.jpg)
Quality Terminology• QA - Quality Assurance
– Planned, systematic actions providing confidence that a quality output will be produced
– Laboratory Procedures• QC - Quality Control (QC)
– Procedures use to assess validity of results in real time, controls release of results
– QC material run with patient samples• EQA - External Quality Assessment (PT)
– Procedures operated by an external agency which allow retrospective review of performance
– RCPA-AACB
![Page 6: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/6.jpg)
Running a QC Program
• Selection of material (matrix)• Selection of levels (decision points)• Setting of targets and ranges• Decision on frequency (batch vs RA)• Decision on number of QC samples (n)• Decision on rules and interpretation• Response to out-of-range values
• Quality planning
![Page 7: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/7.jpg)
QC Quiz
Your new trainee scientist asks you the following question:
“In our lab, how far can the results of an assay vary from the actual concentration in the sample?”
What is the answer:
+/- 1SD; 2 SD; 3 SD; 4 SD; 5 SD ?
![Page 8: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/8.jpg)
Assay Characteristics
• Stable assays:
• Performance defined by mean and SD
• QC never fails
• Results “always” within +/- 2SD
![Page 9: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/9.jpg)
Stable Assay
14
16
18
20
22
24
26
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49
QC
-3SD
-2SD
Mean
+2SD
+3SD
Mean = 20, SD = 195% of QC results between 18 and 22.Interpretation: A result of 20 has a 95% confidence interval of 18 to 22.
![Page 10: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/10.jpg)
Unstable Assays
• Mean drifts over time (fluctuating bias)
• QC process used to detect drifts
• Variation in results due to scatter plus drift
![Page 11: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/11.jpg)
Unstable Assay
14
16
18
20
22
24
26
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37
-3SD
-2SD
Mean
+2SD
+3SD
QC
Mean = 20, SD = 1, Plus fluctuating mean.Interpretation: Result of 20 has 95% confidence limit of18 – 22, PLUS bias at time of assay.
![Page 12: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/12.jpg)
Unstable assays
• How bad can this be?
• How can we measure this?
![Page 13: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/13.jpg)
Original Westgard Multi-rules
![Page 14: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/14.jpg)
Original Westgard Multi-rules
![Page 15: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/15.jpg)
Original Westgard Multi-rules
What does “In Control” mean?
![Page 16: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/16.jpg)
www.westgard.com/
![Page 17: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/17.jpg)
Power Function Chart
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0 1 2 3 4
Pro
bab
ility
of R
ule
s F
irin
g
Shift in Mean (multiples of SD)
N=213s/22s/R4s
![Page 18: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/18.jpg)
Power Function Charts
16
18
20
22
24
26
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73
-3SD
-2SD
Mean
+2SD
+3SD
![Page 19: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/19.jpg)
Power Function Chart
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0 1 2 3 4
Pro
bab
ility
of R
ule
s F
irin
g
Shift in Mean (multiples of SD)
N=213s/22s/R4s
90% error detection at 3.2 x SD
![Page 20: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/20.jpg)
Shifts and Results (unstable assay)
• Imprecision: up to 2 SD.• Undetected shifts in mean: 3 SD• Total spread: up to 5 SD
+3 SD
With the assay still “in control”!
+2 SD
+5 SD
![Page 21: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/21.jpg)
QC Quiz
• The result may differ from the correct result by addition of:– random variation of the assay (up to 2 x SD) – the undetected bias at the time of the assay (up
to 3 x SD).
• Example– CV cholesterol assay: 2.0%– At 6 mmol/L, 5 x SD = 0.6 mmol/L
• Accumulation of errors all in the same direction is rare, but can happen.
![Page 22: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/22.jpg)
Understanding our assays
• For any assay, with the QC protocol in place, we should be able to say how much analytical error may occur.
“These rules have the power to cause a STOP 90% of the occasions when there is a shift in the assay of 2.8 x LSD and cause a PAUSE 90% of the occasions when there is a shift in the assay of 2.6 x LSD.”
- SydPath Quality Control SOP
![Page 23: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/23.jpg)
Westgard - Quantifying QC
• With the rules I have in place, what shifts in assay performance can I detect.
or• How can I be sure that I can detect
important changes– Capability– Setting QC protocols– What are “important changes”
![Page 24: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/24.jpg)
Capability
• Capable assays are easily able to “do their job”
• Capable assays (almost) never produce results outside important limits.
• Poorly capable assays will produce results outside the set limits.
Capable assay Incapable assay
![Page 25: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/25.jpg)
Capability
• Good assays (capable) have an analytical performance (SD) which is much less than the clinically important change.
• This can be quantified as the Capability Index: Cp=ALP/SD(ALP=Allowable limit of Performance)
>6 great; 4-6 OK; <4 poor
![Page 26: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/26.jpg)
Capability
ImportantChange
Cp = 6, good
1 SD Shift
2 SD spreadCp = 4, OK
Cp = 3, poor
2 SD Shift
4 SD Shift
Required shiftdetection
![Page 27: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/27.jpg)
Capability
• Capability is the concept we use to discuss quality of assay performance.
• Relates assay precision to required precision.
![Page 28: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/28.jpg)
Reverse Engineering QC
• If we have limits to our assay performance we want QC protocols which allow us to detect assay problems before “wrong” results may be issued.
• Choose QC protocols which allow appropriate error detection.
• Use Power Function Charts….
![Page 29: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/29.jpg)
Power Function Graph (n=2)
12.5s
13.5s
MR
12s
![Page 30: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/30.jpg)
Setting QC Protocols
• Capable assays – simple rules• Poorly capable assays – Need:
– More rules– More QC
• Incapable assays – will not achieve target performance– Have to accept less chance of finding shifts– Or choose better assay
![Page 31: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/31.jpg)
Quality Specifications Hierarchy
How good do we need to be:
1. Proven analyte-specific data on clinical decision making
2. General-clinical decision making– Based on biological variation– Based on medical opinions
3. Professional recommendations
4. Regulations or EQA targets
5. Published state-of-the-art data
![Page 32: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/32.jpg)
Within-subject biological variation
• Variability in patient results due to changes in the patient.
• www.westgard.com– Archives– Reference Material (near bottom on right)– Biological Variation Database
• www.westgard.com/biodatabase1.htm• eg Sodium: 0.7%; ALT 24.3%
![Page 33: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/33.jpg)
Analytical v Biological CV
CVa/CVb
2 1 0.5 0.25 0.125
Relative total CV
2.2 1.4 1.08 1.01 1.0
![Page 34: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/34.jpg)
CVa / CVb
P
erce
nt in
crea
se in
CV
![Page 35: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/35.jpg)
Precision Targets
• Within-person biological variation provides limits to benefits of improved assay precision
• Many assays clearly capable (optimal)• Some assays not able to reach targets
Test ALT Trig Glucose Albumin Calcium Sodium
CVa/CVb 0.14 0.13 0.23 0.57 0.86 1.86
![Page 36: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/36.jpg)
Bias
• Most discussion thus far has related to precision
• Attention to bias will be the next major issue
![Page 37: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/37.jpg)
Summary (1)
• Some aspects of QC can be quantified• Power function charts are a tool• This process allows us to use
appropriate QC• can either
– Know how good we are or– Aim to reach certain targets
![Page 38: So how good are your results? (An introduction to quantitative QC) Graham Jones Chemical Pathology St Vincents Hospital, Sydney](https://reader034.vdocument.in/reader034/viewer/2022051412/5513cd6d55034679748b4b27/html5/thumbnails/38.jpg)
Summary (2)
• Targets can come from various sources
• Within-person biological variation provides a useful reference point.
• Bias will need to be addressed
• Control of bias will provide many advantages