internal quality control. shewhart or levey- jennings control charts in a stable test environment...
TRANSCRIPT
INTERNAL QUALITY CONTROL
Shewhart or Levey- Jennings control charts
In a stable test environment the distribution of the results of the same sample analyzed a number of times is a Gaussian distribution
Deviation indicates a systematic error The levey-Jennings control chart is derived from the Gaussian
distribution indicating the mean and the one, two, and three standard deviation ranges on both sides of the mean
The chart illustrates the relationship between the levey-Jennings control chart and the Gaussian distribution from which it is derived
This figure also shows the expected percentage of results that should fall within each of the standard deviation ranges
Shewhart or Levey- Jennings control charts
Run Number/ Date
Shewhart or Levey- Jennings control charts
On the control chart used to evaluate the result of the control runso the dates or number of analyses are listed along the X-axis o and the values of the control along the Y- axis
The mean and the 1,2 and 3 standard deviation (s) limits of the control analysis to date are marked
As data is obtained it is plotted one point at a time along the chart
Shewhart or Levey- Jennings control charts
oIn a random distribution and in a correctly operating test system approximately: o 65% of the values will be between the
± 1s ranges and will be evenly distributed on either side of the mean
o 95% of the values should fall between the ± 2s ranges
o and 99% between the ± 3s limits
This means that one data point in 20 should be placed between either of the 2s and 3s limits
One data point placed outside of the 3s limits will occur once in 100 analyses
Shewhart or Levey- Jennings control charts
The ± 2s limits are considered as warning limits A value between the 2s and 3s limit indicates the analysis
should be repeated The ± 3s limits are rejection limits. Analysis should stop,
patient results held and the test system investigated
Run Number/ Date
Inspecting the pattern of plotted points can lead to the detection of increased random error and shifts and/or trends in calibration or in the analysis process
Example of a shift in calibration can be failure in recalibration when lot numbers of reagents changed during analytical run
A trend can start on one side of the mean and move across it or it can occur entirely on one side of the mean
Trends can be caused by the deterioration of reagents, pump tubing or light sources in instruments
Occurrence of shifts and trends is the result of either proportional or constant systematic error
9
Westgard Multi-rules Analysis
Two levels of control at different concentrations will be more efficient in monitoring the method when evaluated statistically
By running and evaluating the results of two controls together, trends and shifts can be detected much earlier
Westgard and associates have formulated a series of so called “multi-rules” to help evaluate results from Gaussian distribution both within a level of control and between control levels
Violation of any of these rules individually or in any combination indicates an error
Modified shewhart control charts for use with multi-rules
10
11
Westgard Multi-rules
•12S rule
•13S rule
•22S rule
•R4S rule
•41S rule
•10X rule
12
Westgard Multi-rules, 12S Rule
One of two control results falls outside ±2s Warning Rule – does not cause rejection of a run Alerts technologist to possible method or instrument
malfunction
13
Westgard Multi-rules, 13s Rule When the control limits are set as the mean ± 3s A run is rejected when a single control measurement exceeds
the mean plus 3s or the mean minus 3s control limit
14
Westgard Multi-rules, 22S Rule
22s - reject when 2 consecutive control measurements exceed the same mean plus 2s or the same mean minus 2s control limit
Level 1
Level 2
15
Westgard Multi-rules, R4S Rule
R4s- reject when 1 control measurement in a group exceeds the mean plus 2s and another exceeds the mean minus 2s
Level 1
Level 2
16
Westgard Multi-rules, 41S Rule
41s- reject when 4 consecutive control measurements exceed the same mean plus 1s or the same mean minus 1s control limit
Level 1
Level 2
17
Westgard Multi-rules, 10x Rule
10x- reject when 10 consecutive control measurements fall on one side of the mean
Level 1
Level 2
18
Random or Systematic
13S and R4S usually associated with random error
22S, 41S, and 10X most often associated with systematic errors
M
1SD
2SD
3SD
M
1SD
2SD
3SD
-1SD
-2SD
-3SD
-1SD
-2SD
-3SD
Level 2 Glucose
Level 1 Glucose
21
Use of Patient Samples as Controls
Using of patient samples for quality control is
acceptable because:
1. The physical and chemical differences between control
material and patient samples are absent
2. The data being generated by the analysis of the patient
samples require no extra effort to obtain for use as
quality control data
3. Inexpensive method of quality control
22
Absurd Values Values that are beyond reference intervals or are not
consistent with the survival of life. Results should be checked by repeat analysis on a fresh
aliquot of the same sample or by an alternate method. The values are usually the result of a random error or
mistake such as short sampling, clot formation. Comparison of absurd values to the clinical picture of the
patient provides validity of the result.
EXTERNAL QUALITY CONTROL TESTING:
PROFICIENCY SURVEYS
24
1- Proficiency Surveys
In a proficiency survey, a laboratory is compared to other laboratories with the same or similar methodology by the analysis of identical samples sent to each of the participants.
Upon receiving the survey specimen, the participant prepares it according to the provided instructions and analyzes it as a routine sample
The results and information concerning the method of analysis are recorded on a provided report form and mailed back to the supplier of the survey.
25
1- Proficiency Surveys For the evaluation, results for each sample are grouped by
analyte and methodology The performance of the group is determined either by
calculating a mean result and standard deviation for each specimen or by determining the consensus or majority opinion
The performance of the in dividual laboratories that comprise the group is evaluated by comparison to the group performance
The mean or consensus result for a survey specimen is considered the accurate and true value for that sample
The closer the in dividual laboratory result is to the survey mean or consensus, the more ac curate it is
26
1- Proficiency Surveys
CAP and similar surveys are designed to accomplish the following goals:
1. To assess the current state of the art of laboratory medicine
2. To provide information to help laboratory directors select the best meth ods and reagents
3. To satisfy the accreditation requirements
4. To provide a voluntary educational peer comparison program that allows a laboratory to compare its performance with its peers
27
A- Survey Samples The proficiency survey samples sent to participating
laboratories are very different from patient samples The rigors of shipping materials great dis tances and the need
for a large batch of sample to send to all of the partici pants does not permit unstabilized or unfixed specimens to be used
Profi ciency samples include: o lyophilized serum, plasma, urine, spinal fluid, stabilized blood
cells, ampules of whole blood for blood gas analysis, lyophi lized cultures, photomicrograph transparencies, and stained blood films for identification
Therefore, survey specimens may be different physically and chemically from patient samples, and are not subjected to collection and processing procedures
28
A- Survey Samples These differences prevent proficiency surveys from accurately
reflecting the overall performance of the laboratory. The preparation of survey material by the laboratory is critical The speci mens must be prepared according to instructions to
reduce interlaboratory variation Exact details for the reconstitution of lyophilized samples are
given, and sometimes the diluent is provided Once the sample is ready for analysis it should be treated as a
routine sample The questionnaire or report form must also be filled out correctly Mistakes made in filling out these forms would be interpreted as
analytical errors and may lead to time wasted in trying to investigate a bad result
29
B- Evaluation of Data from Quantitative Procedures Quantitative procedures are evaluated statistically by
comparing the individual laboratory's results to its peer group
A peer group is defined as a minimum of 20 laboratories having the same or similar methodology or instrument in common
When fewer than 20 laboratories are in a peer group, the group is evaluated against a chosen comparative method
A comparative method is not always a reference method but one that has historically performed well and has good agreement with most of the common methods.
30
B- Evaluation of Data from Quantitative Procedures The mean and standard deviation(s) of the peer group is
calculated Outliers beyond three standard deviations are excluded and the
mean and standard deviation is calculated a second time The second mean and standard devia tion is used to calculate
the Standard Deviation Index Mistakes such as reporting the wrong values for the wrong
samples and the misplacing of decimal points are common transcription errors that can exert undue influence on the calculation of the mean and standard deviation statistics
Their elimination will result in a more accurate picture of the peer group.
31
B- Evaluation of Data from Quantitative Procedures
The Standard Deviation Index (SDI) describes how far from the mean an individual laboratory's result falls in a peer group survey
The closer to the mean or the smaller an individual laboratory's SDI, the more accurate the result
The mean is accepted as the true value of the specimen All of the results for a survey are grouped by sample,
analyte and methodology The mean and standard deviation is calculated for each
peer group.
32
B- Evaluation of Data from Quantitative Procedures The SDI is calculated as follows:
A chart showing the SDI of successive survey specimens will graphically display the laboratory's performance over several separate shipments or sur vey samples
A cumulative SDI chart similar to the one illustrated in Figures 8-1 and 8-2 demonstrates random and systematic analytical error by the pat tern of the SDI values for successive surveys
SDI = Individual Lab Result — Group mean
Group Standard Deviation
33
B- Evaluation of Data from Quantitative Procedures
If a laboratory is consistently below the mean, the SDI will be a series of negative numbers
If the labora tory is consistently above the mean, the SDI will be a series of positive values
All SDIs beyond the ±2 SDI are unacceptable and need to be investigated
Values falling in the ± 1.5 to ±2.0 SDI range, though acceptable, are flagged as a warning of the beginning of a possible trend or shift.
34
35
36
C- The Youden Plot In 1964 Dr. W. J. Youden proposed a method of evaluating precision
as well as accuracy in chemistry and hematology analyzers and manual analytical methods using paired survey samples
The technique involves sending two samples to participating laboratories
One sample would have a low concentration, the other a high concentration
The results are grouped by analyte and methodology and a mean and standard deviation is calculated for each sample
A control chart is prepared so that one sample is on the x-axis and the other on y-axis
Each axis is scaled so that 1 standard deviation is the same size on each coordinate and the mean value for each sample is at the midpoint
37
C- The Youden Plot The ± 3s (from the mean) limit for each sample is marked A line perpendicular to the axis is drawn from the mean for
each sample A circle with a radius = 3s is drawn where the two
perpendicular lines meet A 45o line is drawn from the X-Y intercept, through the center
of the circle, and beyond (Figure 8.3) After the Youden Plot is constructed, the individual
laboratories are plotted, using the results of the paired samples This shows the distribution of the participants of that survey
38
39
C- The Youden Plot The closer an individual lies to the junction of the two
sample means, the more accurate its performance Interpretation of the plot is as follows:o Laboratories falling near the intersection of the means and
within the circle are considered accurate. The closer to the center, the more accurate the result
o Laboratories that fall either high or low along the 45o line or between lines parallel to it and tangent to both sides of the circle are indicative of systematic error, that is, the method is precise but not necessarily accurate
o Laboratories outside of the 45o parallel tangent lines are of random error
40
C- The Youden Plot An easier to construct, modified form of the Youden Plot
can be made by using uniform scales on both axes and drawing rectangle that encompasses the area within the 3s limits of both samples (Figure 8.4 and 8.5)
The 45o line is drawn in but not the parallel tangent lines. Interpretation is similar to the traditional form
42
43
44
D- Qualitative Procedures For qualitative procedures a "consensus method" or referees are
used for evaluating individual results. A consensus method is one that has been chosen by either the
majority of the participants or the survey committee as the best standardized and easily performed method for the analysis.
The use of referees is most common in the identification of blood cells, urine sediment, and microorganisms.
These individuals or laboratories receive samples, as do the other participants, and submit their results.
The consensus of a group of referees is designated as the acceptable identification.
Accurate and acceptable performance is in agreement with the referees or the consensus method.
45
E- Survey Evaluation by the Participants
The survey summaries are returned to the participants for evaluation of their performance.
The laboratory director or his designee, preferably the section head, should review the survey report and initiate an investigation of all unacceptable results.
The findings of the investigation are to be documented, signed by the laboratory director, and retained with the survey report as part of a permanent record of the laboratory's participation
46
Possible causes of unacceptable proficiency survey performance1. Transcription errors made when filling out the report form.
2. Improper or incomplete reconstitution of lyophilized survey samples.
3. Inaccurate identification of cells, microorganisms, or formed urine sediment or transparen cies or stained slides.
4. Inaccurately calibrated analyzer.
5. Analytical test procedure error, i.e., short sampling, timing error, loss of precision due to sample clot, or poorly maintained instrument.
6. Failure to provide a suitable environment for the growth of microorganism.
7. The use of reagents and solutions that have lost reactivity and sensitivity.
8. Mixing up of paired or multiple samples during analysis.
9. Damage to survey material during shipping, i.e.. exposure to excessive heat, or rough han dling resulting in physical damage
47
F- Regional Quality Control Programs
The manufacturers of commercial control material offer a form of external quality control through the use of regional quality control programs.
A labora tory buys sufficient control product to last for a year or longer.
Each month the laboratory submits to the manufacturer's regional computer center either the results of all of its control runs grouped by control lot number and analyte or a summary of the control runs consisting of the monthly mean, standard deviation and the number of control runs for each analyte.
The computer center calculates a monthly mean, standard deviation, and coefficient of vari ation (CV) for the peer group that consists of all laboratories with the same method and lot number of control for each analyte.
48
Regional Quality Control Programs
Comparison among the members of the peer group may be either by SDI or Youden Plots.
The SDI or the Youden Plots indicate where the laboratory stands with respect to the other members of the group and the group mean.
Assuming that the mean value of each group is the accurate value for that control material the accu racy of each individual laboratory can be inferred by its position with respect to the mean.
For laboratories who submit all of their data to the computer center a monthly mean s, and CV of each analyte is calculated.
All participating laboratories receive the lot-to-date cumulative mean, s, and CV for each analyte in that lot of control. This summary data is returned to the participat ing laboratory for evaluation of its precision by the laboratory director.
49
Regional Quality Control Programs
Regional quality control programs do not have the number of participants that the larger CAP and other survey programs are able to attract
The vendors of commercial controls offer regional programs and statistical evaluation as an enhancement to purchase their product.
Because of fewer participants each group may or may not be classified by methodology or instrumentation.
An advantage of the regional programs is that the data is evaluated more frequently, on a monthly basis, and not quarterly, as with most surveys.
50
Fallacies Of External Quality Control
Good performance on proficiency surveys and in regional quality control programs does not necessarily indicate a proficient laboratory
If proficiency survey samples receive special treatment, a true picture of the laboratory's accuracy and analytical error is not shown
Nor do these samples reflect any problems with collection, sample processing, instrument maintenance programs, transcription errors, or the rapport the laboratory has with the users of its services.
The use of the results of the external quality control samples should be placed in proper perspective as an important but partial picture of the laboratory's performance.