control charts and control materials in · pdf filecontrol charts and control materials page:...

NMKL PROCEDURE

No. 3 (2016)

Control charts and control materials

Page: 1 of 28

Version: 2

Date: 8 March 2016

Approved: Franklin Georgsson

CONTROL CHARTS AND CONTROL

MATERIALS IN INTERNAL QUALITY

CONTROL IN FOOD CHEMICAL

LABORATORIES

TABLE OF CONTENTS

FOREWORD ................................................................................................................................... 2

1. INTRODUCTION ................................................................................................................... 3

2. CONSTRUCTION OF CONTROL CHARTS .................................................................... 5

2.1 Control chart for the trueness and within-laboratory precision of analyses ................. 6

2.2 ±R-chart for the checking of precision as repeatability .............................................. 12

2.3 R-charts for the checking of precision ....................................................................... 15

2.4 Control charts using target control limits ................................................................... 16

2.5 Setting control limits for the multimethods/multielement analyses ........................... 17

2.6 Control charts based on other parameters .................................................................. 17

3. CONTROL MATERIALS ................................................................................................. 17

3.1 Certified reference materials ...................................................................................... 18

3.2 Other control materials ............................................................................................... 19

3.3 Recovery tests ............................................................................................................ 21

4. EXCLUSION OF OUTLYING RESULTS ....................................................................... 22

4.1 Examples of the application of statistical methods to exclude outlying results ........ 23

5. USE OF CONTROL CHARTS ......................................................................................... 27

6. REFERENCES .................................................................................................................. 28

NMKL PROCEDURE

No. 3 (2016)

Control Charts and control materials

Page: 2 of 28

Version: 2

Date: 8 March 2016


FOREWORD

This NMKL Procedure was first published in 1997. The Procedure was elaborated by a project

group under the Nordic Council of Ministers. The following persons participated in the project:

Denmark: Mogens Bergstrøm-Nielsen (project leader) and Kristian Hansen

Finland: Harriet Wallin

Iceland: Arngrímur Thorlacius

Norway: Kåre Julshamn

Sweden: Kjell Larsson

The procedure is revised in a project group consisting of:

Norway: Dag Grønningen (project leader), Norwegian Veterinary Institute

Denmark: Ole Danielsen, Danak, the Danish Accreditation Fund

Finland: Kati Hakala, Evira, Finnish Food Safety Authority

Iceland: Heida Palmadottir, Matis

Sweden: Joakim Engman, National Food Agency

The main changes from the previous version are that the procedure now refers to the requirements

of the accreditation standard ISO / IEC 17025 and harmonisation of the interpretations of control

charts with the Nordtest’s "Internal Quality Control" (Trollboken).

NMKL invites all readers of this Procedure to send comments concerning its contents. Comments

should be sent to General Secretariat, see www.nmkl.org.

www.nmkl.org

NMKL PROCEDURE

No. 3 (2016)


Page: 3 of 28

Version: 2

Date: 8 March 2016


1. INTRODUCTION

The use of control charts is a suitable way of documenting internal quality control in food

chemical laboratories. It is also one way of fulfilling the demands in ISO/IEC 17025 5.9.1 [1],

which deals with procedures for detection of trends. Data from control charts can also be used in

estimations of the measurement uncertainty, see NMKL Proc. No. 5 [2].

The most commonly used control chart is obtained when a measurand of a control material is

determined several times, and the result is plotted on the vertical axis against the code of the

analysis (possible a date or a number) on the horizontal axis. The control material may be a

certified reference material, another reference material, or an internally prepared control material,

see NMKL Proc. No. 9 [3]. The term “control sample” is used in this Procedure for the test

portion, which is taken from the control material prior to analysis. The normal control chart

contains horizontal lines, i.e. an average line and warning as well as action limit lines, which

originate from the normal distribution N (µ,σ2) (µ = true or accepted value; σ

2 = variance) which

is presumed to be applicable to the random distribution of the values. The true values of the

average (µ) and variance (σ2) of chemical determinations will always be unknown. For this

reason, statistical estimates, mean ( x ) for µ and standard deviation (s) for2 , are used in

practice. These estimates may be calculated from results of series of replicated determinations

made on one control material. Estimates are more reliable, the larger the number of

determinations carried out: the number should not be less than 20, assuming that the results are

normally distributed. In order to include variation over time, and variation resulting from different

analysts, determinations should be carried out on different days, and the work be distributed

between relevant analysts. The precision of such a series of replicate determinations is designated

within-laboratory reproducibility. The precision of a series of determinations carried out as

replicate determinations in one and the same analytical run (same analyst, same day) is designated

repeatability, see NMKL Proc. No. 4 [4]. The repeatability standard deviation is generally smaller

than the reproducibility standard deviation, since the latter is associated with more sources of

error.

A determination which continuously gives correct results with acceptable precision is said to be

under statistical control. Normal random errors cause only one result in twenty (5%) to fall

outside the lines x ± 2s. This means that in the case of longer series one or a few results should

NMKL PROCEDURE

No. 3 (2016)


Page: 4 of 28

Version: 2

Date: 8 March 2016


be outside ± 2s. These lines are designated warning limits. About three results in 1000 (0.3%) fall

outside the lines x ± 3s, which are called action limits. Strictly speaking, these probabilities hold

for intervals of type µ ± 2σ and µ ± 3σ, respectively, but may, as estimates, be applied for x ±2s

and x ±3s. Horizontal lines corresponding to x , x ± 2s and x ± 3s should be introduced into the

control chart.

It is recommended that wherever possible, control charts are constructed and used continuously in

all routine analyses. The charts should be kept available to all persons involved in the analytical

work in question, and results should be recorded into the charts immediately after the

determinations has been carried out. A control chart demonstrating that the activities of the

laboratory continues to be on the predetermined quality level, is a document which increases

confidence in work among all categories of staff. Control charts are tools, which enable

laboratories to detect faults at an early stage, and to take corrective actions in a timely manner.

It is important that control charts are constructed on the basis of measurements carried out in the

laboratory, not on the published performance characteristics such as trueness and precision, which

are possibly stated in the method.

If it is known or suspected that the dry matter content of a control material changes with time, the

control chart should be based on the measurand concentration calculated on the dry weight basis.

However, the laboratory should investigate what the supplier recommends to do with the control

material before analysis.

A control chart demonstrating that an analysis is under statistical control supports the correctness

of the results obtained from unknown samples. It is therefore important that control charts are

saved for at least the period of time that the corresponding analytical results are kept on file. If an

analytical result is questioned, the laboratory should be able to track the result to a specific day

A control chart is a diagram intended to examine whether a process is under statistical

control. In the chart, the values of one or more characteristics (average, standard deviation,

variation width, etc.) are plotted, and compared to givens limits.

Upper/lower action limits are limits/lines in a chart, which are used as criteria of the need

for corrective actions of a process.

Upper/lower warning limits are lines in a chart, which are used as criteria for the need of

stricter monitoring of a process. The warning limits are between the action limits.

NMKL PROCEDURE

No. 3 (2016)


Page: 5 of 28

Version: 2

Date: 8 March 2016


and a specific analytical run. It would be a strong indication (but not absolute proof) that the

unknown sample was correctly analysed, if the laboratory can demonstrate that, in the same run, a

control material was analysed with an acceptable result.

It is important that it is made clear to all members of the staff that control charts are used to

ensure that any abnormalities are detected at an early stage, not in order to put pressure on the

staff.

2. CONSTRUCTION OF CONTROL CHARTS

The most commonly used control charts are charts for checking of the trueness of analyses (see

2.1, fig 1) and for checking the precision (see 2.2, fig 4). Many laboratories monitor both trueness

and precision and use combination charts. Trueness charts are often called x charts (x bar),

precision charts often called R charts (from range), and combination charts x - R charts. The

advantage of a combination chart is that errors or trends having an effect on both trueness and

precision are detected at an early stage. The charts may contain a protocol part for the raw data,

dates, and the analyst, as well as possibly also the code of the sample. If the charts do not contain

a protocol part, raw data must be documented elsewhere in such a way that a value in a control

chart can be traced to day, run and analyst.

When checking the trueness, the control materials used should be either certified reference

materials (see 3.1) or other materials, which, if possible, have been traceably calibrated against

certified reference materials (see 3.2). Otherwise the x -chart can only show trends over time and

the within-laboratory precision, not trueness expressed as bias. Two or more determinations

should be carried out on the samples if R-charts or x - R-charts are to be constructed.

When taking a new control material into use, it takes some time to reach the required number of

20 results under within-laboratory reproducibility conditions. In that case, the required number of

replicate determinations may be carried out within a short interval of time (however, not less than

two days). Then x and sr are calculated from the results, and the obtained values are used to

construct a preliminary control chart, which thus is based more on repeatability than on within-

laboratory reproducibility conditions. This chart is used in the daily work for the first 20

analytical runs. Thereafter the results of the first 20 determinations are utilised to construct the

final control chart, which is used in future routine work.

NMKL PROCEDURE

No. 3 (2016)


Page: 6 of 28

Version: 2

Date: 8 March 2016


It is recommended that new control limits are calculated annually using all new results obtained

during the year. Then preform a Fischer’s test for examining whether the standard deviations

from two series of observations are statistically different.

2.1 Control chart for the trueness and within-laboratory precision of analyses

Figure 1 shows a x -chart constructed after a number of determinations had been carried out on

one and the same control material, and after calculation of the average and the standard deviation.

The central line of the chart shows the obtained average. The uppermost and lowest (red) lines

represent the action limits which are set at the average ± 3 times the standard deviations. The

lines in between (blue) are the warning limits placed at the average ± 2 times the standard

deviations. The chart has been used for the analysis of a control material 20 times in connection

with corresponding determinations of unknown samples. The results obtained have been recorded

on the chart. Determination is under statistical control.

Figure 1: Control x -chart with warning and action limits in which results from 20

determinations have been recorded.

If the laboratory has a customer that requires smaller measurement uncertainty, the laboratory

must implement measures to accommodate this. The first measure one can implement is to

perform duplicate determinations on samples and control material. An average value of two

4,5

4,6

4,7

4,8

4,9

5

5,1

5,2

5,3

5,4

5,5

1 3 5 7 9 11 13 15 17 19

Control x-chart

Result

Average

Upper Warning limit

Lower Warning limit

Upper Action limit

Lower Action Limit

NMKL PROCEDURE

No. 3 (2016)


Page: 7 of 28

Version: 2

Date: 8 March 2016


(preferably more) separate measurements made on the same day will reduce the spread due to the

repeatability component. The reduction can be calculated according to following formula:

22

br

x sn

ss

Where

xs is the standard deviation of the mean values

sr is the repeatability standard deviation

sb is the between day standard deviation

n is the number of replicate measurements

These mean values are less sensitive against errors and are more likely to be normally distributed.

There are also a number of other actions that the laboratory can introduce.

It is an advantage if analytical data of routine work are recorded on control charts immediately

after each completed determination. In many cases this is handled in the Laboratory Information

Management System (LIMS) or in Excel. (A Google or YouTube search will give you many

results on how to create a control chart.) Laboratory made IT programs for control charts shall be

validated and locked when it is used. Previous editions of programs shall be stored.

When results deviating from the normal (see section 5) are obtained, a note should always be

made on the control chart demonstrating that action has been taken, and it should be specified

what action has in fact been taken. If there is insufficient space in the control chart for notes, the

data should be equipped with reference indicating how this action is documented.

The placement of the warning and action limits on the control chart is determined by the

estimated standard deviation.

Upper action limit = average + 3 x standard deviation

Upper warning limit = average + 2 x standard deviation

Lower warning limit = average - 2 x standard deviation

Lower action limit = average - 3 x standard deviation

NB! If the limits have been calculated on the basis of relevant raw data, it should be expected

that 1 in of 20 results will be outside the warning limits and 3 in 1000 outside the action

limits. Thus, due to random distribution an individual result may fall outside the upper or the

lower action limit even if nothing abnormal has occurred.

NMKL PROCEDURE

No. 3 (2016)


Page: 8 of 28

Version: 2

Date: 8 March 2016


It is advisable to annually check that the precision of the analysis, i.e. its standard deviation, has

not been changed. This holds true especially for determinations which were new to the laboratory

at the time when the control chart was constructed, and where the laboratory, with time, has

gained more experience of the determination. In such cases it is possible that the precision of the

determination has improved, i.e. that the random errors have decreased, resulting in a lower

standard deviation. New (and stricter) warning and action limits may then be called for.

Correspondingly, the precision may in some cases become poorer, and in such cases there may be

reasons to establish new, wider warning and action limits. Statistical methods, for example,

Fischer’s test, should be employed when examining whether a new estimated standard deviation

is, in fact statistically significantly different from the previous one.

2.1.1 Fisher’s test for the comparison of variances

Fischer’s test is a statistical test suitable for examining whether the standard deviations from two

series of observations are statistically different. The test can be applied in order to determine

whether the standard deviation of an analytical method changes with time so that new warning

and action limits should be applied.

Standard deviations cannot be compared as such, but must be converted to variances, which are

the square of standard deviations. Provided that both series of observations are normally

distributed, the relationship between the variances will follow Fischer’s F-distribution. The

Fischer test may be carried out on different significance levels, and as one- or two-sided tests.

By applying the one-sided Fischer test an answer can be obtained to the question:

“Is method A less accurate than method B”? This may be expressed as “Is sA ˃ sB”?

Fischer’s two-sided test can give an answer to the question “Do methods A and B have different

repeatabilities? This may also be expressed as: “Is sA = sB”?

In control chart conjunction, it is recommended that variances are examined in one-sided tests on

a 5% significance level.

In Fischer’s test, the variances of the two series of observations are first calculated, i.e. the

standard deviations are squared. Then the value F = s12/s2

2 is calculated, after which the F value is

compared to a value in a statistical table. Such tables of critical values for the F distribution may

be found in most elementary textbooks on statistics. In order to demonstrate how the tables are

used, an excerpt from a table covering a relevant, but narrow interval (one-sided test, 95%

probability, more than ten observations) is presented below.

NMKL PROCEDURE

No. 3 (2016)


Page: 9 of 28

Version: 2

Date: 8 March 2016


Note that the higher standard deviation should always be designed s1, i.e. s1 ˃ s2. An obtained F

value which is higher than the table value indicates that the standard deviation s1 is statistically

significantly higher than the standard deviation s2. As the differences between variances decrease,

the F value will come closer to unity.

Example:

An analytical method has been in use for two years, during which time a control material has

been analysed in every analytical run, and the results obtained plotted in a control chart. The chart

was originally constructed based on a series of 20 measurements with a standard deviation of s =

2.3. The variance of the analysis was thus s2 = 5.29.

In order to find out whether the repeatability of the determination de facto had improved over

time, it was decided to examine whether a new variance, calculated on results from the most

recent 20 duplicate determinations was statistically significantly lower than the original. A new

standard deviation, s, was calculated and a value of 1.4 was obtained. The variance was calculated

as s2, giving the value 1.96.

The laboratory wanted to find out: “is the variance s22 = 1.96 significantly lower than the variance

s12 = 5.29”?

The F value is calculated: F = s12/s2

2 = 5.29/1.96 = 2.70

The critical value for F, 95% probability, one-sided test and 20 degrees of freedom is read from

the relevant statistical table.

The degrees freedom = the number of observations - 1; in this example the number is actually 19,

but since this number is not included in the table below, the value given for 20 degrees of

freedom is selected. This value is 2.12. The obtained value for F, 2.7 is thus higher than the table

value indicating that the new standard deviation of 1.4 is in fact significantly lower than original

standard deviation of 2.3. The laboratory should consider adjusting the warning and action limits

of the control chart.

NMKL PROCEDURE

No. 3 (2016)


Page: 10 of 28

Version: 2

Date: 8 March 2016


Table 1: Critical values of Fischer’s test, one-sided test, 95% probability

No. of degr.

of freed/lower

variance

Number of degrees of freedom, higher variance

10 12 15 20 60 Infinite

10 2.98 2.91 2.85 2.77 2.62 2.54

12 2.75 2.69 2.62 2.54 2.38 2.30

15 2.54 2.48 2.40 2.33 2.16 2.07

20 2.35 2.28 2.20 2.12 1.95 1.84

60 1.99 1.92 1.84 1.75 1.53 1.39

Infinite 1.83 1.75 1.67 1.57 1.32 1.00

New control chart should be constructed when a new control material is taken into use.

It is not recommended to continuously correct warning and action limits, which does occur if the

laboratory adds the result to the raw data of the control chart in connection with each

determination and estimates a new standard deviation. Such “continuous correction” is included

in some computer software, and should be avoided.

2.1.2 Example of the construction of an x -chart for trueness and within-laboratory precision

An example is given below of the construction of an x -chart, using results of individual

determinations of an arbitrary measurand. The example may also be applied to analyses in which

duplicate determinations have been carried out. In such cases the average of the duplicates should

be noted in the result column.

A total of twenty single determinations were performed on different days. The first three columns

of Table 2 present the code, date and the result of the analysis. The average x of the twenty

determinations is calculated by adding all results, and dividing the sum (100.33) by the number of

measurements (20). The average is 5.02 and the average line can be introduced into the control

chart (Figure 2).

The standard deviation, s is calculated by using the formula below:

s =1

)( 2

n

xx

Where x is an individual result (measurement result), x is the average and n the number of

determinations.

NMKL PROCEDURE

No. 3 (2016)


Page: 11 of 28

Version: 2

Date: 8 March 2016


Table 2. Example of raw data for the construction of a control chart

Excel

Column

and

Row

A B C D E

1 Number of

Analysis

Date Result, g/kg

(x) (x - x ) (x - x )

2

2

1

04.06.96

4.94

-0.08

0.00585

3 2 05.06.96 4.94 -0.08 0.00585

4 3 06.06.96 5.21 +0.19 0.03744

5 4 07.06.96 4.82 -0.02 0.03861

6 5 10.06.96 5.03 +0.01 0.000182

7 6 11.06.96 5.06 +0.04 0.00189

8 7 13.06.96 5.10 +0.08 0.00697

9 8 14.06.96 5.14 +0.12 0.01525

10 9 17.06.96 4.99 -0.03 0.00070

11 10 18.06.96 5.02 ±0 0.00001

12 11 19.06.96 4.96 -0.06 0.00319

13 12 20.06.96 5.02 ±0 0.00001

14 13 24.06.96 4.86 -0.06 0.02449

15 14 26.06.96 5.03 +0.01 0.000182

16 15 27.06.96 5.06 +0.04 0.00189

17 16 28.06.96 5.00 -0.02 0.000272

18 17 01.07.96 5.02 ±0 0.00001

19 18 02.07.96 5.14 +0.012 0.01525

20 19 03.07.96 4.96 -0.06 0.00319

21 20 08.07.96 5.03 +0.01 0.000182

Number of

Analyses

(n) = 20

Sum = 100.33

Average

( x ) = 5.0165

g/kg

2xx =

0.1614

For manual calculation of the standard deviation the last two columns in Table 2 contain the

difference of each individual result from the average (x - x ), the square of this difference

(x - x )2, and finally the sum of the squares,

2)( xx . Obtained values are placed into the

formula for the standard deviation:

19

1614,0s = 0.09216 = 0.092

When the standard deviation has been calculated, warning and actions limits may be computed.

Warning limits are calculated as x ± 2s. These will be 4.84 and 5.20. Action limits are calculated

as x ± 3s. These will be 4.74 and 5.29. Warning and action limits are introduced into the chart,

see Figure 2.

NMKL PROCEDURE

No. 3 (2016)


Page: 12 of 28

Version: 2

Date: 8 March 2016


Figure 2. x -chart, ready to be taken into use

2.2 ±R-chart for the checking of precision as repeatability

In addition to checking the trueness and within-laboratory precision of analyses, it is often of

interest also to check the repeatability, i.e. the agreement of two duplicate determinations is

monitored. In such cases charts for the difference between duplicate determinations are used. For

constructing such a chart for duplicate determinations, two samples of the same control material

are determined under repeatability conditions (within a short period of time, using the same

method, same analyst, on separate test portions). This could be repeated e.g. 15 times.

Alternatively, results of authentic samples may be used, collected over a certain period of time, so

that the required 20 duplicate results are available.

Different analytical methods have errors of a different nature. Some methods, for example

gravimetric and titrimetric methods have constant errors, whereas other methods, for example

atomic absorption methods have constant, relative errors, i.e. the standard deviation of duplicated

determinations increase in proportion to the concentration. When deciding on the parameter to be

monitored on a control chart, attention should be given to this fact. A control chart with numerical

(absolute) limits is more suitable for analysis having constant errors. If analysis have relative

errors, a chart with relative limits, i.e. the relative standard deviation (=

) instead of s, is more

4,5

4,6

4,7

4,8

4,9

5

5,1

5,2

5,3

5,4

5,5

1 2 3 4 5 6 7 8 9 1011121314151617181920

Control x-chart

Result

Upper Warning limit

Lower Warning limit

Upper Action limit

Lower Action Limit

NMKL PROCEDURE

No. 3 (2016)


Page: 13 of 28

Version: 2

Date: 8 March 2016


suitable. If the method has relative errors, results from control samples having varying

concentrations cannot be introduced in the same control chart when using absolute limits.

Prerequisites of a relevant chart for duplicate determinations are that:

the two test portions are labelled every time in the same way, for example a and b,

the same difference (a-b) is always used, maintaining the sign of the difference, and

test portion a is always analysed before test portion b

The absolute value of the difference, a-b, is used for calculating limits for an R chart (see 2.3).

2.2.1 Example of construction of a chart based on differences between duplicate determinations

Below is given an example of the construction of control chart, using results from duplicate

determinations of a control material. A total of ten duplicate determinations were carried out on

different days. In the first four columns of Table 3 are given the number of the determination, the

date and the results, a and b. The differences (a-b) are in the fifth column, with signs retained, and

in the last column, the squares of the differences, (a-b)2.

The mean difference a-b between duplicate determinations is set to the value 0 (the values stem

from identical homogeneous sample materials) with a random variation around the average (= 0),

which may be estimated to 2rs , where rs is the standard deviation of a single determination;

the factor 2 is due to two determinations being made. It should be noted that if the

measurements are prone to a drift in the same direction most of the times the mean difference will

not be zero.

Table 3. Raw data for the construction of a control chart for 10 duplicate determinations

Number of the

Analysis Date

Result (mg/kg)

a b (a-b) (a-b)2

1 04.06.96 100.0 99.1 +0.9 0.81

2 05.06.96 103.9 103.0 +0.9 0.81

3 06.06.96 104.8 104.7 +0.1 0.01

4 07.06.96 104.0 104.1 -0.1 0.01

5 10.06.96 101.9 103.0 -1.1 1.21

6 11.06.96 103.0 102.9 +0.1 0.01

7 13.06.96 103.8 103.7 +0.1 0.01

8 14.06.96 99.5 99.4 +0.1 0.01

9 17.06.96 100.2 100.7 -0.5 0.25

10 18.06.96 102.9 102.4 +0.5 0.25

Number of analyses = 10

Sum

3.38

NMKL PROCEDURE

No. 3 (2016)


Page: 14 of 28

Version: 2

Date: 8 March 2016


First, the standard deviation, sr of a single determination is calculated from the formula:

k

basr

2

)( 2 =

102

38.3

= 0.4111

Where k is the number of duplicate determinations. The standard deviation will in this example

be 0.4111.

The variation around the average 0 of differences a-b from duplicate determinations will therefore

be 2rs = 0.41 ∙ 1.414 = 0.58.

Warning and action limits of the control chart for duplicate determinations are calculated

according to the same principles as those for a x-chart. The central line is given the value 0.

Upper/lower action limits will be 0 ± 3 ∙ 0.58 (+1.8 and – 1.8, respectively), and upper/lower

warning limits 0 ± 2 ∙ 0.58 (+1.2 and -1.2, respectively). The chart is presented in Figure 3, and it

is now ready to be taken into use in routine work.

Figure 3. Prepared control ±R-chart for the monitoring of precision

-2

-1,5

-1

-0,5

0

0,5

1

1,5

2

1 2 3 4 5 6 7 8 9 10

Control ±R-chart

(a-b)

Upper Action limit

Upper Warning limit

Lower Warning Limit

Lower Action limit

NMKL PROCEDURE

No. 3 (2016)


Page: 15 of 28

Version: 2

Date: 8 March 2016


Each time the measurand in question is determined, two test portions (a and b) of the control

material are included in the run, and the difference between the results obtained are plotted in the

control chart. The two test portions should preferably be inserted randomly into the run. If

required, test portion a, may always be placed in the beginning of the run, and test portion b, at

the end. The control chart would then indicate if, within a run there is a tendency towards changes

in the analytical results. It should be noted, however, that when a randomized order has not been

used, this may lead to an underestimation of various contributions to the total variation.

2.3 R-charts for the checking of precision

The description above concerns the monitoring and documentation of the precision of

determinations by analysing separate control materials and registering the results on control

charts. Many laboratories check the precision of determinations by monitoring, with the aid of

control charts, the results of duplicate determinations made on one actual sample in an analytical

run. The obvious advantage of such a routine is that resources in such cases need not be used for

including a separate control material in each analytical run, but that the results from actual

samples can be utilized for control purposes. A prerequisite, however, is that replicate

determinations are carried out routinely on a sufficient number (minimum of 20) of representative

samples to establish the average of the differences.

For the construction of R charts, the average of the differences (a-b) from a series of duplicate

determinations is first calculated. This average will be the centre of line of the chart.

kbaR /

In the case of two test portions, the upper action limit will be = 3.267∙ R , and the upper warning

limit = 2.518 ∙ R . In addition the relation R = 1.128 s, where s is the standard deviation. The

lower limit = 0 (the same result for test portions a and b). When calculating the control limits for

the first time, all differences larger than the upper action limit should be discarded, and the

warning and action limits should be re-calculated. This adjustment should be carried out only

once.

An example of an R-chart is given in Figure 4.

NMKL PROCEDURE

No. 3 (2016)


Page: 16 of 28

Version: 2

Date: 8 March 2016


Figure 4. R-chart with warning and action limits in which results from ten duplicate

determinations have been plotted.

If in routine work involving duplicate determinations, an obtained difference, |a-b|, exceed the

action limit, this result should lead to corrective action (see section 5). Note that R and s may be

calculated as both relative and absolute values. Usually relative values are calculated, since

absolute values are only applicable to materials containing the measurand at the same

concentration level as the material used for constructing the chart.

When this model is applied to sample materials, R-charts should be developed for

matrices/measurand levels which have approximately the same relative standard deviation. All

samples analysed in the laboratory are thus utilized in the monitoring.

2.4 Control charts using target control limits

In the previous examples it has been shown how control limits are set based on the spread of the

measurements. An alternative approach that can be useful is to use target control limits. If the

requirements of the measurements are defined e.g. as a maximum measurement uncertainty, it is

possible to estimate the permissible within laboratory standard deviation that could be used in an

x-chart. One can say, as a rule of thumb, that your expanded measurement uncertainty is 4 times

the standard deviation. Based on this, a control card with warning (2∙sr) and action (3∙sr) limits

can be constructed.

0

0,2

0,4

0,6

0,8

1

1,2

1,4

1,6

0 5 10 15

Control R-chart

Absolutt(a-b)

Upper Action limit

Upper Warning limit

NMKL PROCEDURE

No. 3 (2016)


Page: 17 of 28

Version: 2

Date: 8 March 2016


2.5 Setting control limits for the multimethods/multielement analyses

Modern methods can cover up to several hundreds of measurands. When statistical control limits

are used we know that on average 1 of 20 results will by pure random error be outside the

warning limits. If we use e.g. a pesticide residue method that cover 200 measurands we would in

1 out of 5 runs have a result outside the action limit by pure random error. This makes the

interpretation of the control charts a bit problematic. The cost of re-analysis can be high and may

be unnecessary. The method may have worked perfectly fine with its normal spread. If the

customer permit, wider statistical control limits or target control limits would be preferable to use.

2.6 Control charts based on other parameters

In this procedure control charts are based on measurands concentration or recovery. Anyway, in

some cases also other parameter e.g. ion ratio (mass spectrometric detection), absorbance or S/N

might be valuable and recommended for the checking trueness. The principle of construction is

the same as long as normal distribution of results can be assumed.

3. CONTROL MATERIALS

The basis for the use of control charts in internal quality control is that the laboratory analyses

homogeneous, known control samples together with actual, unknown samples in exactly the same

manner. The control samples may be e.g. certified reference materials, other reference materials

or internally produced control materials (see NMKL Proc. No. 9 [3]).

By a “control sample” is meant the test portion taken for analysis from the control material. The

type of the control materials (measurand level, matrix etc.) should be similar to the actual

samples. To be fully representative, the control material should have exactly the same matrix as

regards the composition, including components present at low concentrations, if those are

expected to have an effect on the accuracy. It is very important that a control material

contains the measurand on a level comparably to that of the actual samples to be analysed.

The control material should also have a physical form similar to that of the sample, for example

the same degree of comminution. Control materials should also fulfil other important criteria:

they should be sufficiently stable over the period of use, and it must be possible to take out

identical test portions from the material.

NMKL PROCEDURE

No. 3 (2016)


Page: 18 of 28

Version: 2

Date: 8 March 2016


Often a control material must be available in large quantities, so that it can be used over a longer

period of time. It is also important that routinely used control materials have a reasonable price.

Attention should be paid to the fact that the dry matter concentration of materials stored for a long

time may vary. It is therefore necessary periodically to monitor the dry matter of control

materials. If variations in dry matter significantly contribute to the total variation (if dry matter

variations are seen in a control chart), the control chart should be based on the concentration of

the dried control material. Such problems may be avoided if it is possible to store the control

material in a desiccator.

3.1 Certified reference materials

Certified reference materials (CRM), if available, may be used to demonstrate traceability of the

analytical results. The following definition of a certified reference material (CRM) is from

ISO/IEC Guide 99:2007 [5]: “reference material, accompanied by documentation issued by an

authoritative body and providing one or more specified property values with associated

uncertainties and traceabilities, using valid procedures”.

Previously it was thought that the certified reference materials may be used only for reference and

calibration purposes, not routinely. The perception is now about to change; certified reference

material can be used more widely for control of the analytical work, for example in conjunction

with analysis performed rarely where it is therefore impractical to have an internally produced

control material.

Certified reference materials are most often used for monitoring of the trueness of results. Other

reference or control materials may also be used for the checking of trueness, e.g. materials used in

proficiency tests. These other materials are usually used in monitoring the precision of

determinations. Other reference materials should, if possible, be calibrated against certified

reference materials.

It is important that the certified reference materials used contain the measurand at suitable

concentration levels, since trueness and traceability can be referred to only at the level examined.

The analysis of certified reference materials is of little value if the reference material has the

wrong level. For example, it is impossible to estimate the trueness of a determination at a level

0.1 µg/kg–100 µg/kg using a reference material containing the measurand at 1000 µg/kg.

The use of certified reference materials is limited by the following constraints:

although the availability of certified reference materials has greatly increased recently,

there is still a lack of suitable materials for many food analyses.

NMKL PROCEDURE

No. 3 (2016)


Page: 19 of 28

Version: 2

Date: 8 March 2016


for natural reasons, reference materials will never be available for analyses where the

measurand and/or the matrix are unstable

even if the price of certified reference materials is not prohibitive in relation to the total

costs of the analytical work, it may be difficult for laboratories working in several

analytical fields to keep a stock of each relevant kind of reference material

certified reference materials can often not be purchased in quantities required for routine

quality control over longer periods of time

it should be realized that all certified reference materials are not of high quality. Caution is

recommended in cases where the information in the accompanying certificates is

incomplete.

3.2 Other control materials

If, for one or more of the reasons mentioned under 3.1, it is not possible, suitable, or

economically justifiable to use certified reference materials as control materials, it is necessary

that the laboratory, possibly in cooperation with other laboratories, prepares its own control

materials. Below are given some suggestions as to how internal control materials may be

prepared. All methods are not suitable for all analytical tasks.

3.2.1 Establishment of a “true” value for a control material by analysis

In principle it is possible to determine a “true” value of a property of a stable material simply by

careful analysis. It is important that great care is taken in order to avoid association of the value

with a systematic error. Such actions require some sort of impartial checking, for example that the

sample is analysed in several laboratories and, if possible, using methods based on different

physical chemistry principles. Carelessness in impartial assessment of a control material may lead

to serious weaknesses in the internal quality control.

A traceable “true” value of an analytical property of a control material may be obtained by

analysing the material and a selection of similar certified reference materials. Replicate

determinations should be carried out, and the test portions should appear in a random order in the

analytical run. The certified reference materials should be similar to the samples as regards

composition and measurand level. In these situations certified reference materials are used in

order to calibrate the analytical method for the determination of the property in question of the

control material. The value received for the certified reference material must naturally lie within

the certified limits, when the measurement uncertainty of the measurement is taken into account.

This means that the “true” value for your control material will have an uncertainty that is the

NMKL PROCEDURE

No. 3 (2016)


Page: 20 of 28

Version: 2

Date: 8 March 2016


combined of your result’s measurement uncertainty and the uncertainty of the “true” value of the

certified material.

If the results obtained for the certified reference material are systematically outside the certified

limits, the results on the unknown samples should not be corrected for this, since it is not certain

that the same systematical error applies to the samples. The laboratory should instead carefully

examine the reason for the error, and in such work, if possible, use alternative methods. In such

cases the laboratory should also consider (and consult) the manufacturer of the certified reference

material, in order to find out whether other laboratories have had analytical problems with the

material in question.

3.2.2 Materials which have been studied in proficiency tests

Materials which have been included in proficiency tests may be valuable sources of control

materials. Such materials have been analysed by many laboratories, often using a number of

different methods. If there is no evidence to the contrary (evident systematic errors, abnormal

statistical distribution of the results, etc.), the average obtained by the participating laboratories

may be considered a validated value for the material in question. The measurement uncertainty of

this value should be evaluated whether it is fit for purpose or not. There is a theoretical problem

with demonstrating the traceability of such a consensus value, but this does not undermine the

usefulness of the procedure. Such materials are available, to a limited extent only, from

organizations conducting proficiency tests, provided that the organization produced the material

in amounts greater than needed for the proficiency test. It must be possible to demonstrate that the

material is stable. The WEB site www.eptis.bam.de helps you to find a suitable proficiency test

(PT) and the address of the organization conducting the PT.

3.2.3 True value from formulation

In some cases a control material may be prepared simply by mixing components of known purity

in specified proportions. This procedure could, for example, be acceptable in cases where the

routine samples are solutions. Problems are often encountered in the preparation of solid control

materials, and in cases where it must be ensured that the chemical form and physical distribution

of the measurand in the matrix is realistic. Furthermore, it must be ensured that the components

have been mixed sufficiently well, i.e. that the material is homogeneous.

http://www.eptis.bam.de/

NMKL PROCEDURE

No. 3 (2016)


Page: 21 of 28

Version: 2

Date: 8 March 2016


3.2.4 Spiked control materials

Spiking, i.e. addition of the analyte, is a method to prepare control materials, in which preparation

by formulation is combined with assessment of true value by analysis. This method is suitable

when a material is available which does not contain the analyte to be determined, but which

otherwise is similar to the sample to be analysed. After careful analytical control to ensure that

the background level is sufficiently low, a known amount of the analyte is added to the material.

A control material prepared in this way has the same matrix as the unknown sample and contains

a known amount of the analyte. The uncertainty of the concentration (or other character) of the

control material is limited to possible mistakes in the analysis of the unspiked sample. However,

it may be difficult to ensure that the added analyte has, for example, the same chemical and

physical form as that in the unknown sample, and that the spiked sample is homogenous (see

NMKL Proc. No. 25 [6]).

3.3 Recovery tests

In cases where it is impossible or impractical to use control materials, a limited control of the

systematic error may be carried out by checking the recovery of the analysis (see NMKL Proc.

No. 25 [6]). This technique is especially useful in cases where the measurand and/or the matrix

is/are unstable, or when a single (infrequent) analysis is to be carried out. A known amount of the

measurand is added to a test portion of the sample, and this spiked sample is then analysed at the

same time as the original sample. It is important that the amount added is of the same order of

magnitude as the amount present in the weighed unknown sample. The recovery (%) of the added

measurand is the difference in results between the two measurements, divided by the added

amount times 100:

Found amount – Original amount of the sample ∙ 100

Added amount

However, a good recovery is no proof of the correctness of the result. In metal analysis, for

example, contamination or uncorrected matrix effects may have the effect that the added

measurand is “pushed” in front of an elevated level. Example: if the “true” concentration of the

measurand is 0.050 mg/kg, and matrix effects or contamination result in a reading of 0.50 mg/kg,

measurement of the spiked sample and then calculation of the recovery, which in both cases could

be 100%, would not reveal this.

The apparent advantage of the use of recovery tests is that the matrix is representative. The

technique is in frequent use since most sample types may be spiked, but has the same limitations

NMKL PROCEDURE

No. 3 (2016)


Page: 22 of 28

Version: 2

Date: 8 March 2016


(species, chemical bonding, physical form) as the spiking technique (see under 3.2.4). In addition,

it may in some cases be questioned whether the same recovery is obtained from an added

measurand as from the naturally occurring measurand of the sample. However, it may be stated

that poor recovery of an added measurand is a strong indication of similar or even poorer

recovery of the naturally occurring measurand.

The use of spiked control materials and recovery tests in analytical work must be separated from

the standard addition method, which is a measurement technique. Same spiked sample cannot at

the same time serve as a calibration point of the measurement and a quality control. For both

purposes individual sample portions should be prepared and measured.

4. EXCLUSION OF OUTLYING RESULTS

An extreme result, or an outlier, is a value of an original population which deviates so much from

other values of the population that it may statistically be regarded as disagreeing with other

values. Experience shows that outlying results cannot always be avoided. When collecting data

for the construction of a control chart, the collection of data may thus contain outliers, which need

to be excluded. Possible outlying results should be excluded on statistical grounds only. It should

therefore be made clear to all staff that it is not permissible to discard the result of a measurement

on subjective grounds merely because a value seems to be outlying. Laboratories should also take

great care in excluding numerous extreme values uncritically: for example, a ratio of extreme

values in a population exceeding 2/9 (ca. 22%) should be regarded as an indication of lack of

robustness of the used method. Laboratories should be aware that a generous attitude towards the

exclusion of outliers will always lead to tighter warning and action limits, and thus to a higher

frequency of deviations.

The literature contains several approaches to the calculation of whether or not an obtained value

is statistically outlying. In food analysis it is recommended that extreme differences in results

from replicate determinations (abnormally poor replicate results) are identified using the Cochran

test, and that individual results (either single results or averages of replicate results) are tested for

the presence of outliers using Grubbs’ tests. It is recommended that the Grubbs’ tests are

performed as two-sided tests (the deviating result is either abnormally high or abnormally low),

and that both tests are performed on a probability level of 2.5%.

NMKL PROCEDURE

No. 3 (2016)


Page: 23 of 28

Version: 2

Date: 8 March 2016


4.1 Examples of the application of statistical methods to exclude outlying results

Two practical examples given below are intended to illustrates the use of the Cochran test to

determine whether a difference between duplicate results deviates statistically from other

differences obtained, and the use of the Grubbs’ test to determine whether averages (or individual

results) obtained are outlying results. Excel can be used to calculate both Cochran og Grubbs’

test. A Google or YouTube search will give you many results on how to do it.

Example 1:

Application of the Cochran tests for identifying extreme differences between duplicate results.

A total of eleven duplicate determinations were carried out to form the basis of a control chart for

monitoring the precision of a determination (table 4). In one case a difference in duplicate results

was obtained which appeared too high. The laboratory decided to investigate whether the

difference of 0.32 was significantly higher than the other obtained differences.

Table 4. Example with data for Cochran outlier test.

No.

Replicate

a

Replicate

b (a-b) (a-b)2

1 5.23 4.66 0.57 0.32*

2 5.24 5.05 0.19 0.04

3 5.00 4.82 0.19 0.03

4 5.03 4.87 0.16 0.03

5 5.04 5.05 -0.01 0.00

6 5.08 4.99 0.09 0.01

7 5.04 4.98 0.06 0.00

8 4.86 4.99 -0.13 0.02

9 5.10 5.00 0.10 0.01

10 4.82 5.03 -0.21 0.04

11 4.87 4.96 -0.09 0.01

Sum 0.5151

Test

value 63.1

Critical 62.2

The laboratory used the Cochran test, which may be carried out as follows:

1. Calculate the square of each difference [(a-b)2. Add together all squares [Ʃ(a-b)

2. The

value 0.5151 is obtained.

2. Identify the highest difference, (a-b)max. This value is 0.57. Calculate the square of the

value (0.32).

NMKL PROCEDURE

No. 3 (2016)


Page: 24 of 28

Version: 2

Date: 8 March 2016


3. Calculate [(a-b)2 max

/ Ʃ(a-b)

2 ∙ 100. The value 63.1 is obtained.

NB ! Step 1-3 may be expressed simply as:

Calculate [[(a-b) max 2/Ʃ(a-b)

2 ∙ 100

4. Compare the obtained value with the critical Cochran value given in the table 6 below.

This value is for 11 observations, and on a probability level 0.025 (2.5% probability) =

62.2. As the obtained value is higher than the critical value, the Cochran test does

indicate an outlying result.

It should be noted that the Cochran test only addresses the repeatability, i.e. the agreement of

duplicate results. The test thus does not give any information on how well the average of two

duplicate results agrees with other averages in the population.

Example 2:

Application of the Grubbs’ test for identifying extreme values in a normally distributed

population.

A total of eleven duplicate determinations were carried out to form the basis of a control chart for

monitoring the trueness of a determination. After sorting the data a result was obtained which

appeared too high. The laboratory decided to investigate whether the value of 2.01 was

significantly higher than the other obtained results (table 5).

Table 5. Example with data for Grubb’s outlier test.

No. Replicate a Replicate b Mean Grubb high

4 1.45 1.32 1.38 1.38

9 1.39 1.41 1.40 1.40

5 1.62 1.42 1.52 1.52

2 1.45 1.60 1.52 1.52

8 1.56 1.49 1.52 1.52

11 1.54 1.52 1.53 1.53

1 1.39 1.72 1.55 1.55

3 1.66 1.45 1.56 1.56

7 1.59 1.62 1.61 1.61

6 1.55 1.67 1.61 1.61

10 1.96 2.05 2.01

s 0.162 0.075

Grubb test

value

53.9

Critical

39.3

NMKL PROCEDURE

No. 3 (2016)


Page: 25 of 28

Version: 2

Date: 8 March 2016


The laboratory used the Grubbs’ test, which may be carried out as follows:

1. Calculate the standard deviation, s of all the values. The value 0.162 is obtained.

2. Exclude the highest value of 2.01, and calculate the standard deviation of the remaining

values, sH. The value 0.075 is obtained.

3. Calculate the Grubbs value GH = 100 ∙

[1 – (sH/s). The value 53.9 is obtained. This value

expresses, in %, the improvement of the standard deviations caused by eliminating the

suspected high value.

4. Compare the value 53.9 with the critical value given in the Table 7 below for 11

observations, this value is 39.3. The obtained Grubbs value is higher than the critical

value in the table, and indicates an outlying result, which should be excluded from further

calculations.

The Grubbs’ test described above is a so-called single Grubbs’ test (it is tested whether one value

(either a high or a low one) is an extreme value. In a corresponding manner, the double Grubbs’

test may be applied to investigate whether the result series contains pair-wise extreme values: one

high and one low value in the same series; two high values; two low values. Values suspected to

be extreme are excluded as a pair, one after the other, and new standard deviations are calculated

using the remaining values, whereafter the lowest new standard deviation is identified, and the

double Grubbs’ value calculated according to step 3 above. If the value obtained is higher than

the relevant value in the table 7 below, it is an indication that the population contains a double

Grubbs’ outlying result, which should be eliminated from further calculations. In Excel this can

be done by adding more columns and calculate Grubb’s test values for all cases. An example is

given below (Table 6).

Table 6. Example on how Excel can be used for testing of single as well as double Grubb outliers.

Excel

Column

and

Row

A B C D E F G

1 No. Results Grubb

high

Grubb low 2 High 2 Low High-

Low

2 4 1.38 1.38 1.38

3 9 1.40 1.40 1.40 1.40 1.40

4 5 1.52 1.52 1.52 1.52 1.52 1.52

5 2 1.52 1.52 1.52 1.52 1.52 1.52

6 8 1.52 1.52 1.52 1.52 1.52 1.52

7 11 1.53 1.53 1.53 1.53 1.53 1.53

NMKL PROCEDURE

No. 3 (2016)


Page: 26 of 28

Version: 2

Date: 8 March 2016


8 1 1.55 1.55 1.55 1.55 1.55 1.55

9 3 1.56 1.56 1.56 1.56 1.56 1.56

10 7 1.61 1.61 1.61 1.61 1.61 1.61

11 6 1.61 1.61 1.61 1.61 1.61

12 10 2.01 2.01 2.01

13 s 0.162 0.075 0.16 0.07 0.15 0.06

14 Grubb test

value

53.9 2.0 55.8 4.5 62.4

Table 7. Critical values of the Cochran and Grubbs’ tests, 2.5% probability

Number of

results

Cochran

values

Single-Grubbs’

values, highest

or lowest

Double Grubbs’

values, 2 highest

or 2 lowest

Double Grubbs’

values, one highest

and one lowest

4

94.2

86.1

98.9

99.1

5 88.6 73.5 90.9 92.7

6 83.2 64.0 81.3 84.0

7 78.2 57.0 73.1 76.2

8 73.6 51.4 66.5 69.6

9 69.3 46.8 61.0 64.1

10 65.5 42.8 56.4 59.5

11 62.2 39.3 52.5 55.5

12 59.2 36.3 49.1 52.1

13 56.4 33.8 46.1 49.1

14 53.8 31.7 43.5 46.5

15 51.5 29.9 41.2 44.1

16 49.5 28.3 39.2 42.0

17 47.8 26.9 37.4 40.1

18 46.0 25.7 35.9 38.4

19 44.3 24.6 34.5 36.9

20 42.8 23.6 33.2 35.4

21 41.5 22.7 31.9 34.0

22 40.3 21.9 30.7 32.8

23 39.1 21.2 29.7 31.8

24 37.9 20.5 28.8 30.8

25 36.7 19.8 28.0 29.8

26 35.5 19.1 27.1 28.9

27 34.5 18.4 26.2 28.1

28 33.7 17.8 25.4 27.3

29 33.1 17.4 24.7 26.6

30 32.5 17.1 24.1 26.0

40 26.0 13.3 19.1 20.5

50

21.6

11.1

16.2

17.3

NMKL PROCEDURE

No. 3 (2016)


Page: 27 of 28

Version: 2

Date: 8 March 2016


5. USE OF CONTROL CHARTS

Control charts are constructed in a way which means that if only “normal” random errors occur,

then the probability is approximately 95% that an obtained value will be within the warning

limits, and 99.7% that it will be within the action limits. This reasoning is based on standard

deviation estimated from 30 or more observations. In practice, it is unimportant if rather fewer

samples were used to construct that control chart.

Results obtained in the analysis of control materials may be expected to be above and below the

centre with the same frequency.

There are simple rules for the interpretation of control charts.

1. in control

the control values is within the warning limits

the control values is between warning and action limits and two previous control

values were within limits

2. in control but long-term it is out of statistical control

7 control values in consecutive order gradually increase or decrease

10 out of 11 consecutive control values are lying on the same side of the central

line

3. out of control

The control value is outside the action limits

The control value is between the warning and the action limit and at least one of

the two previous control values is also between warning and action limit

These rules are harmonised against Nordtest Report TR 569 [7].

It is the responsibility of the laboratory to establish suitable acceptance limits for the analytical

work, i.e. for example to define the maximum acceptable difference between duplicate results.

Any detected abnormalities should be noted in the control chart.

Analytical work should be discontinued and the reason for the abnormality investigated before

work is resumed. Results obtained on unknown samples must be discarded. Results may be

reported only after the reason for the abnormality has been established, through the analysis of

control materials that the analysis is again under statistical control. All actions and results must be

documented.

It is important that control charts, including the protocol part (or corresponding raw data

documented elsewhere) are kept on file at least for the same period of time that raw data (and

NMKL PROCEDURE

No. 3 (2016)


Page: 28 of 28

Version: 2

Date: 8 March 2016


other data on methods, staff, instruments etc.) of the analysis of the unknown samples analysed at

the same time as the control material are kept.

No system for internal quality control is fool proof, but experience shows that well established

routines for the construction and use of control charts based on control materials analysed

together with unknown samples supplemented with accurate documentation and archiving are

useful and simple methods for monitoring the quality of analytical work.

Control charts may also be a source of inspiration to those involved in the analytical work: for

example a control chart on the notice board of the laboratory, demonstrating that analytical work

fulfils established quality requirements demonstrates to the staff and to others that the laboratory

is working up to standards on a continuous basis.

6. REFERENCES

1) General requirements for the competence of calibration and testing laboratories. ISO/IEC

17025:2005.

2) Estimation and expression of measurement uncertainty in chemical analysis. NMKL Procedure

No. 5, 2. Ed. 2003

3) Evaluation of method bias using certified reference materials. NMKL Procedure No. 9, 2. Ed.

2007

4) Validation of chemical analytical methods. NMKL Procedure No. 4, 3. Ed. 2009

5) International Vocabulary of Metrology -- Basic and General Concepts and Associated Terms

(VIM) ISO/IEC Guide 99:2007

6) Recovery information in analytical measurement. NMKL Procedure No. 25, 2014

7) Nordtest report TR 569 Internal quality control, ISBN 82-577-5054-9

control charts and control materials in · pdf filecontrol charts and control materials page:...

Documents