REFERENCE MATERIALS IN QUALITY ASSURANCE

OF TESTING LABORATORIES

Jukka HyrynenHeikki Kallio

Anna-Maija Kosonen

NORDTEST

TasksThe tasks of Nordtest are to promote the safety of life, health, environment and materialvalues and to encourage a free exchange to trade. The approach adopted by Nordtest toachieve its objectives is:

to develop, adopt and recommend test methods and to promote the use of theseby industry and the authorities and also in the standardisation work

to obtain international recognition of test results and also the competence of theNordic countries, for instance by quality assurance and verification of testingactivity

- to endeavour that tests and approval of test results are made in a resource andcost effective manner

to promote the technical testing infrastructure in the Nordic countries by meansof research, development of competence and collaboratian, and

to participate in the European and international development of testing and topromote Nordic interests,

OrganisationThe organisation consists of a board, a secretariat and nine technical groups. These groupsare Acoustics and Noise, Building, Electronics, Environment, Fire, Mechanics, Polymers, VVS(Mechanical Building Services) and Quality Assurance.

The work is directed by the board which comprises representatives of all the Nordiccountries, The members are appointed by the government or appropriate department of thecountry concerned.

The technical groups initiate and evaluate projects. The projects are often structured in sucha way that they can be used as catalysist for development of the combined technicalcompetence in the Nordic countries. At present, about 250 Nordic projects are being carriedout in some 40 firms and institutions.

The board as well as the technicai groups are assisted by the secretariat which is responsiblefor day to day activity. The secretariat is located at Esbo, Finland.

Financial frameworkThe cost of the Nordtest secretariat and a Iarge proportion of project activity is financedfrom the budget of the Nordic Council of Ministers. The grant for 1992 is approx. 2 millionsECU, The work of the board and the technical groups is financed by the participatingorganisations.

PublicationsRegister of 1300 test methods and technical reports

- Test methodsTechnical reports-

-

-

-

-

-

NT TECHN REPORT 177

Approved 1992-04

Authors:Jukka HyrynenHeikki KallioAnna-Maija Kosonen

NORDTEST project number: 956-90-l

Institution: VTT

Title (English): -

Title (Original): REFERENCE MATERIALS IN QUALITY ASSURANCE OF TESTINGLABORATORIES

Abstract:

This paper is the final report of NORDTEST Project 956-90-l Referensmaterial vid

kvalitetsskring av provningslaboratorier (Reference materials in quality assurance of testing

laboratories). The scope of this project was to collect information for the users of reference

materials in the Nordic countries and to propose recommendations for the use of reference

Technical Group: Quality Assurance

ISSN: 0283-7234 Language: English Pages: 35

Class (UDC): 620.1 Key words: reference materials, quality assurance, testinglaboratories

Distributed by:NORDTESTTekniikantie 12FIN-02150 EspooFinland

Publication code:

materials

Nordtest Nordtest

Nordtest Nordtest

CONTENTS

1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2 Certification of reference materials . . . . . . . . . . . . . . . . . . . . . . 71.3 Organizations within the CRM schemes . . . . . . . . . . . . . . . . . . 7

2 REFERENCES IN QUALITY AND TESTING STANDARDS . . . . . . . . . . . . . . 82.1 EN45001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.2 ISO/IEC Guide 25 : 1990(E) . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.3 IS0 9001 : 1987 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.4 NAMAS accreditation standard Ml0 . . . . . . . . . . . . . . . . . . . . . 10

2.4.1 NAMAS information sheets . . . . . . . . . . . . . . . . 102.5 Testing standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.5.1 Hardness test standards . . . . . . . . . . . . . . . . . . . 112.5.2 Impact test standards . . . . . . . . . . . . . . . . . . . . . 12

3 CHOOSING AND MAINTAINING REFERENCE MATERIALS . . . . . . . . . . . . 133.1 Choosing a reference material . . . . . . . . . . . . . . . . . . . . . . . . . 133.2 Working standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.3 Household practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4 REFERENCE MATERIALS IN QUALITY ASSURANCE OF TESTINGLABORATORIES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.1 Indirect verification of test instruments . . . . . . . . . . . . . . . . . . . 16

4.1.1 Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174.2 Controlling laboratory performance . . . . . . . . . . . . . . . . . . . . . . 17

4.2.1 X-R charts . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.2.2 Value-time charts . . . . . . . . . . . . . . . . . . . . . . . 19

4.3 Using the control charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.4 Interlaboratory tests in estimating laboratory performance

and the uncertainty of testing . . . . . . . . . . . . . . . . . . . 21

5 SOME SOURCES OF CERTIFIED MATERIALS . . . . . . . . . . . . . . . . . . . . . 225.1 BCR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225.2 NIST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

6 COMAR DATABASE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

7 NORDIC QUESTIONNAIRE ON REFERENCE MATERIALS . . . . . . . . . . . 24

8 CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

APPENDIX 1 Excerpts from BCR REFERENCE MATERIALS CATALOG 2p.APPENDIX 2 Example of a certificate of measurement 2p.APPENDIX 3 Producers of RMs and CRMs 1p.

-3-

1 INTRODUCTION

High-quality quantitative information on the properties of materials is needed for

the safe, efficient and reliable design and use of modern industrial products.

Confidence in the materials and the material properties is being enhanced through

standardized test methods and verified performance of laboratories; formal

accreditation is chosen as an accepted path in Europe towards harmonized

practices and results. Accredited laboratories are expected to perform tests

consistently from day to day; furthermore a test carried out at different accredited

laboratories according to a standardized method should produce identical results.

Reference materials (RMs) are used in various fields of testing to provide users

with accurate reference values and traceability for their measurements, to quantify

and to calibrate methods and equipment and to monitor the performance of the

laboratory. These tests can be complex, or the result is qualitative or is not readily

traceable to national or international standards. A reference material is by defini-

tion a material or substance one or more properties of which are sufficiently wellestablished to be used for calibration of an apparatus, assessment of a measure-ment method, or for assigning values to materials1.

RMs are widely used within the field of analytical chemistry, where they form the

basis for ensuring accuracy and for adhering to valid measurement systems. The

use of reference materials is gaining momentum in other fields of testing, too.

Certified reference materials (CRMs) are reference materials whose propertyvalues are certified by a technically valid procedure, accompanied by or traceableto a certificate or other documentation which is issued by a certifying body1. Inchemical laboratories CRMs correspond to the reference standards of physical

measurements. The use of CRMs has been encouraged and emphasized by the

1 definition, ISO GUIDE 35

calibration and traceability requirements in the international standards also for

accreditation of testing laboratories.

1.1 Definitions

There are a number of terms often used when dealing with quality assurance of

testing as well as with calibration of equipment and instruments. The definitions

below are collected from various standards to clarify further discussions in the

paper.

REFERENCE MATERIAL & CERTIFIED REFERENCE MATERIAL

* see above

WORKING STANDARD

* Reasonably homogeneous, previously analyzed materials, pure com-

pounds, solutions of pure elements etc. used instead of CRMs for fre-

quent checks and in situations when cost or availability limit the use of

CRMs.2

STANDARD REFERENCE MATERIAL (NIST Standard Reference Materials

Catalogue 1990-91, NIST Special Publication 260)

* NIST Standard Reference Material (SRM) is a certified reference

material issued by NIST.

2 Distinction between RMs and working standards is somewhat vague; oneway could be: RM is commercially available whereas a working standardis manufactured in-house.

-4-

-5-

REFERENCE STANDARD OF MEASUREMENT (NAMAS ACCREDIT

ATION STANDARD M10)

* Instrument or gauge used to calibrate measuring and test equipment.

The term also applies to a material used as a reference standard.

Note: The term standard reference material is also used in NAMAS

standards for a certified reference material.

CERTIFIED VALUE (ISO-GUIDE 30)

* For a CRM, the value that appears in the certificate or other documen-

tation accompanying the material, this value having been certified by a

technically valid procedure.

CONSENSUS VALUE (ISO-GUIDE 30)

* For a reference material, the value of the quantity obtained by

interlaboratory testing, or by agreement between appropriate bodies or

experts.

CALIBRATION (ISO GUIDE 25 : 1990)

* The set of operations which establish, under specified conditions, the

relationship between values indicated by a measuring instrument or

measuring system, or values represented by a material measure, and the

corresponding known values of a measurand.

VERIFICATION (ISO GUIDE 25 : 1990)

* Confirmation by examination and provision of evidence that specified

requirements have been met.

* NOTE - In connection with the management of measuring

equipment, verification provides a means for checking

that the deviations between values indicated by a mea-

-6-

suring instrument and corresponding known values of a

measured quantity are consistently smaller than the

maximum allowable error defined in a standard, regula-

tion or specification peculiar to the management of the

measuring equipment.

* The result of verification leads to a decision either to restore to service,

or to perform adjustments, or to repair, or to downgrade, or to declare

obsolete. In all cases it is required that a written trace of the verification

performed be kept on the measuring instruments individual record.

DIRECT VERIFICATION OF TESTING EQUIPMENT (ISO / EN standards)

* Method for checking the main functions of the machine.

* NOTE - This method shall be used:

When the machine is installed or removed or repaired.

When the result of the indirect verification is not satisfactory.

INDIRECT VERIFICATION OF TESTING EQUIPMENT (ISO / EN stan-

dards)

* Method suitable for overall checking of the machine. The indirect

verification method may be used on its own for periodic routine checking

of the machine in service.

* The indirect verification may be carried out by means of certified

reference materials (standardized, calibrated blocks).

VERIFICATION OF A REFERENCE MATERIAL (ISO standards)

* All the operations carried out, normally by a technically competent

third party, with the object of confirming the validity of the value(s)

stated in a reference material certificate.

-7-

CALIBRATION OF STANDARDIZED BLOCKS (EN standards)

* A specified method to be used to gain required traceability for the

reference material used for the indirect verification of the testing

machines.

1.2 Certification of reference materials

Reference materials can be gaseous, liquid or solid. Certification of CRMs can be

done according to three approaches:

DEFINITIVE METHOD, certification is done by one highly qualified laboratory

using the best possible methods to reach very high precision and very

low systematic error.

INDEPENDENT METHOD, usually done by one laboratory using two or more

reliable independent methods of measurement, each of which has been

shown to yield good results. The methods produce results which overlap

and all results by all the methods must fall within allowable end-user

uncertainty.

INTERLABORATORY METHOD, a number (5 - 20) of laboratories carry out

an interlaboratory test programme, and a statistical analysis of the results

is carried out. The programmes often consist of two or more rounds of

measurements to cut down the uncertainties due to different methods of

handling, measurement and analysis of the samples.

1.3 Organizations within the CRM schemes

In Europe the Community Bureau of Reference (BCR) is the largest establishment

dealing with certified reference materials (see appendix 1). There are also other

-8-

establishments with more limited fields of interest. In the US the National

Institute of Standards & Technology is federally responsible for the CRMs.

The ISO Committee on reference materials (REMCO) is harmonizing the work on

reference materials by organizing relevant research projects and by publishing ISO

GUIDES on the subject. REMCO has prepared an international database on

reference materials (COMAR, see chapter 6).

2 REFERENCES IN QUALITY AND TESTING STANDARDS

The quality-related standards either recommend or require the use of reference

materials to increase the reliability of testing activity. The following excerpts from

a number of standards show the level of requirements.

2.1 EN 45001

The EN 45001 standard General criteria for the operation of testing laboratories

describing the requirements for the accreditation of laboratories refers to reference

materials as follows:

* Reference materials shall where possible be traceable to national or

international standard reference materials. (5.3.3. Equipment)

* The Quality Manual shall contain at least:

. . .

f) where appropriate, reference to proficiency testing, use of

reference materials, etc.; (5.4.2. Quality System)

-9-

2.2 ISO/IEC Guide 25 : 1990(E)

The ISO/IEC Guide 25 General requirements for the competence of calibration

and testing laboratories describes the activities of both calibration and testing

laboratories as follows:

* The quality manual, and related quality documentation, shall state the

laboratorys policies and operational procedures established in order to

meet the requirements of this Guide, the quality manual and related

quality documentation shall also contain

. . .

n) reference to verification practices including interlaboratory

comparisons, proficiency testing programmes, use of reference

materials and internal quality control schemes. (5.2, Quality

system, audit and review)

* Reference materials shall, where possible, be traceable to national or

international standards of measurements, or to national or international

standard reference materials. (9.7, Measurement traceability and calibra-

tion)

2.3 ISO 9001 : 1987

The ISO 9001 standard Quality systems. Model for quality assurance in design/

development, production, installation and servicing describes the requirements for

inspection, measuring and test equipment as follows:

* The supplier shall control, calibrate and maintain inspection, measuring

and test equipment, whether owned by the supplier, on loan, or provided

by the purchaser, to demonstrate the conformance of product to the

specified requirements. Equipment shall be used in a manner which

-10-

ensures that measurement uncertainty is known and is consistent with the

required measurement capability. (4.11)

2.4 NAMAS accreditation standard Ml0

The NAMAS accreditation standard General Criteria of Competence for Calibra-

tion and Testing Laboratories describes the requirements for measurement

traceability and calibration as follows:

* Where the concept of traceability is not in practice, the Laboratory

shall provide satisfactory evidence of correlation of calibration or test

procedures, for example by participation in a suitable program of mea-

surement audit or inter-laboratory comparison or through the regular use

of standard reference materials. (7.5)

* Where the reference standard of measurement is a reference material

the Laboratory shall, wherever possible, obtain a certificate that provides

evidence of the characterization of the material and evidence of traceabil-

ity to national or international standards of measurement, or national or

international reference materials. (7.7)

2.4.1 NAMAS information sheets

NAMAS has published various information sheets as supportive media for

laboratories applying for or maintaining accreditation. These information sheets

give requirements, for example, for test performance, machine verification, use of

reference materials and also give sources of accredited verification and test

blocks. (NAMAS Information Sheets NIS 13 TRACEABILITY, HARDNESS

MEASUREMENTS)

-11-

2.5 Testing standards

Many testing standards referring to calibration or traceability do require reference

materials either as primary or secondary means of traceability. Examples of such

methods are measurement of surface hardness and impact testing of metallic

materials. These are chosen below as examples of test methods.

ISO or EN standards on mechanical testing usually have separate standards for the

verification of testing machines. For example hardness and impact testing

machines shall or may be verified by means of standardized blocks (=RMs). This

method is so-called indirect verification (see definitions).

2.5.1 Hardness test standards

ISO/EN

New EN standards will incorporate corresponding ISO standards and replace the

existing European national standards. These standards contain both direct and

indirect verification methods of testing machines. There are also separate stan-

dards for calibration of standardized blocks to be used for indirect verification of

hardness testing machines.

Direct verification (see definitions) of testing machines involves

-verification of test force,

-verification of the indenter and

-verification of the measuring device.

The hardness values given by the testing machine do not depend only on the

dimensions of the indenter, but also on the surface roughness and the seating of

the indenter. For this reason an indirect verification using RMs is considered

necessary. The indirect verification can be performed with two procedures.

According to the first procedure each hardness scale (Rockwell), different

-12-

hardness ranges and test forces or ball sizes (Brinell) will be verified periodically.

The other possibility is to verify prior to the test only one hardness value that

corresponds approximately to that of the tests to be performed.

ASTM

In ASTM standards instructions for a specific hardness testing method, verifica-

tion of machine and calibration of hardness blocks are in one standard. According

to ASTM standards the direct verification shall be used for new and rebuilt

machines. The indirect method using standardized test blocks may be used in

routine testing to assure the operator that a machine is operating properly.

2.5.2 Impact test standards

According to prEN 10 045 Part 2, the main method for verification of impact test

machines is the indirect verification using the Charpy V reference test pieces,

whose breaking energies are known. The dimensional requirements are more

accurate than the requirement for standard test specimens. The test pieces shall be

national test pieces traceable to the BCR test pieces or certified BCR test pieces.

The direct method shall be used, when the machine is being installed or repaired

or if the indirect method gives an incorrect or unacceptable result.

Contrary to the EN standard, the main method in the ISO 442 standard is the

direct verification. The indirect method is not even an alternative to complete

direct verification. The reference test pieces are unnotched and their thickness is

smaller than the standard specimens. The same material shall be tested in parallel

in a reference machine.

According to the ASTM E 23-88 standard, Charpy impact machines shall be

verified by testing with standardized specimens. If these produce values outside

the tolerances, those parts subjected to wear shall be inspected and if necessary

repaired or replaced. ASTM E 127l-88 gives very detailed instructions for manu-

-13-

facturing and qualifying Charpy verification specimens. Certified reference

material is delivered by the National Institute of Standards and Technology. It

must be noted that this material can be used only in machines made according to

ASTM E 23. Certain dimensions in ISO and EN standards differ from those in

ASTM.

3 CHOOSING AND MAINTAINING REFERENCE MATERIALS

The ISO-GUIDE 35 Certification of reference materials - General and statistical

principles describes the requirements for establishing certified reference

materials. Much importance is given to confirming homogeneity of the materials

and to the statistical methods to define the uncertainties.

Certification can be established by a definite method, by interlaboratory testing or

through a metrological approach.

The ISO-GUIDE 31 Contents of certificates of reference materials describes the

requirements for the certificates. There are unfortunately a number of commercial-

ly available reference materials whose certificates do not clearly state the level of

confidence, level of certification or the uncertainties. The users of CRMs have to

seek confirmation on the quality of reference materials before purchase.

3.1 Choosing a reference material

The user of a reference material must decide what properties are relevant to the

measurement process. The level (of property value) of the CRM should corre-

spond with the level of the intended measurement. The matrix should be close to

the matrix of the material to be subjected to the measurement process (carbon in

low-alloy steel or carbon in stainless steel). The form should correspond with that

of the intended samples (solid, liquid, gas...). The quantity of the CRM should be

-14-

sufficient; the need to obtain additional CRM during the experimental programme

should be avoided and batches of CRMs should partly overlap in use for continu-

ity. Stability of the CRMs should be ensured or changing of the properties should

be recorded in the certificate. The uncertainty of the certified value should be

acceptable when compared to requirements (legal limits, trade agreements and

commercial requirements, previous experience in the laboratory). Appendix 4

shows an example of a BCR reference material.

3.2 Working standards

Often the availability of certified reference material is limited, the use of the

reference material is extensive or the price of the certified reference material is

high. In these cases the commercial certified reference materials cannot be used

in everyday practice by the laboratories. The laboratories can and are also encour-

aged to produce in-house working standards.

Homogeneity of these working standard reference materials should be verified

with sufficient accuracy through in-house test series; the level of the parameters

of interest have to be calibrated against valid certified reference materials.

Uncertainty (95% confidence limits, for example) should be estimated for the

working standard.

Generally a well-prepared and evaluated working standard with traceability to

CRMs is of higher value to the quality assurance of a testing laboratory than a

commercial reference material with limited confidence and questionable certifi-

cate.

-15-

3.3 Household practices

Reference materials have to be treated as highly vulnerable instruments. The

CRMs have to be stored in a manner which will insure that none of the properties

of the CRMs alter more than the normal deterioration stated in the certificates,

if evident. Records on all uses of CRMs should be kept and all unqualified uses

of CRMs have to be discouraged.

Clear procedures should be maintained on how each reference material is to be

used and what records kept. CRMs, RMs or working standards should be used in

a continual manner to ensure quality and traceability of testing in a laboratory as

described in the following chapter.

4 REFERENCE MATERIALS IN QUALITY ASSURANCE OF TESTING

LABORATORIES

Two major uses of reference materials in a testing laboratory can be identified.

One use is the indirect verification and calibration of test instruments, while the

other is the continual monitoring of laboratory performance.3

The NIST catalogue gives the following reasons for use of referencematerials:

* To help develop accurate methods of analysis (reference met-hods);* To calibrate measurement systems used to:

(a) facilitate exchange of goods,(b) institute quality control,(c) determine performance characteristics, or(d) measure a property at the state-of-the-art limit; and

* To assure the long-term adequacy and integrity of measure-ment quality assurance programmes.

-16-

4.1 Indirect verification of test instruments

As mentioned above, verification of an instrument is understood as confirmation

by examination and provision of evidence that the instrument meets the specified

requirements set for it. Verification provides a means for checking that the

deviations between values indicated by a measuring instrument and the corre-

sponding known values of a measured quantity are consistently smaller than the

maximum allowable error defined in a standard, regulation or specification

characteristic of the measuring equipment.

Indirect verification usually consists of an overall checking of the working

conditions of the test instrument as well as carrying out the normal test pro-

cedures using samples of known level(s) of measured parameter(s). The results

from the tests are compared with the values given for the verification samples and

conclusions are drawn based on the deviations (differences of test results and

corresponding true value) and the requirements given for the instrument.

The result of verification leads to a decision either to restore to service, or to

perform adjustments, or to repair, or to downgrade, or to declare obsolete. In all,

cases it is required that a written trace of the verification performed shall be kept

on the measuring instruments individual record.

Often the standards that define indirect verification of test instruments require the

use of certified reference materials (CRMs) as samples with known levels of the

parameters of interest.

A reduced level of reliability can be allowed for the reference material; this

depends on the requirements set for the instruments, on the level of uncertainty of

the certified reference material (traceable, verified, calibrated material) and on the

performance of the instrument. The laboratory can manufacture its own working

standard reference materials or use reference materials without a certificate as

long as the laboratory has verified and documented the production and the

-17-

traceability of the working standard. ISO-GUIDE 33 encourages laboratories to

study possibilities to supplement CRMs with working standards when the required

CRMs have a limited availability and relatively high cost.

4.1.1 Calibration

Calibration is often (but not necessarily) an integral part of verification of a test

instrument. Calibration defines the relationship between values indicated by a

measuring instrument or measuring system, or values represented by a material

measure, and the corresponding known values of a measurand. The known value

here can be given through CRM or RM as above and through instruments whose

calibration is traceable to national and international standards.

4.2 Controlling laboratory performance

Controlling the laboratory performance should be understood as a philosophy, a

system and a set of techniques for controlling and improving testing processes. In

a sense the process is related to the statistical process control (SPC) (Juran 1974)

generally used in the production and service industries. In SPC the normal

production is evaluated in an organized manner by measuring critical attributes of

the production afterwards.

Much of the testing is destructive or testing affects the attributes measured; often

adjustment, verification or calibration of the instruments with reference material

(or instruments) is required prior to testing. The possibility to re-test the samples

is thus very unlikely and valuable information is received on performance of the

test process by recording the measurements (of known values of attributes) done

prior to the test.

-18-

The process described above is very open and has to be included in the normal

testing procedures. Another way to receive material for performance control is to

include blind samples in the testing process. The open process has the possibility

of biased or specially good practice in testing as the CRMs are naturally

identified; blind testing can be difficult to organize and in time samples of known

values of attributes are identified by the personnel. Often it is advisable to

incorporate not only testing but also sample preparation in the blind testing

programmes.

An integral part of successful control is good planning of the sampling and testing

plan along with the number of tests, levels and intervals. Control charts measure

the process over time. There are many ways to draw up these charts; below two

choices are described.

4.2.1 X - R charts

The conventional SPC control chart is constructed as a pair of graphs showing

both mean and range of measurements. Sample size for SPC charts is usually

between 2 and 10. The means and control limits (three standard deviations) are

calculated (Fig 1). The next step is to determine whether the process is under

control or not. A simple test sets conditions for out-of control process as follows:

* any points above the upper control limit or below the lower control

limit in either graph or

* eight points in a row above (or below) the centreline in either graph.

This test tells only if the process is under control, not if the measurements are

within required tolerances. The tolerance limits have to be added to the graphs to

justify evaluation on whether the results are within specifications.

Nordtest Nordtest

-19-

Fig. 1

X-bar

8 LCL6

4

2

0

R2.5 | UCL

X-R chart of a process showing both upper and lower controllimits set at three standard deviations from the average valuelines. Sample lot size is 5 per data point.

4.2.2 Value-time charts

A simpler and probably more used method within testing is recording individual

measurements of the attributes of known value i.e. testing CRMs. Fig. 2 shows a

hypothetical chart of this kind. A test is done each morning using a CRM of

known level and uncertainty according to procedures and standards. The graph

shows the uncertainty of the CRM and the (in-house) specification limits for the

tests.

Nordtest Nordtest

-20-

4.3 Using the control charts

Fig. 2 a) shows a value-time chart with measurements at equal intervals and

sample size 1, while limits of allowed deviations are also included. Fig. 2 b)

shows increasing scatter of results; this phenomenon can be due to loss of calibra-

tion/adjustment or even wear of the instruments. All in all, the verification

interval of the equipment is clearly overdue.

VALUE12

10

8

6

4

2

0

0 20 40 60 80 100

VALUE12

10

8

6

4

2

00 20 40 60 80 100

VALUE12

10

8

6

4

2

00 20 40 60 80 100

VALUE .1 2

10

8

6

4

2

00 20 40 60 80 100

Fig. 2 Value-time charts showing control measurement data.Explanations of the figures in the text

Figure 2 c) shows a sudden increase in scatter in the results, which could be due

to a mechanical fault in the instrument. Further analysis of the data in figure 2 d)

shows that when the measurements by two operators are separated, the increase

in scatter is probably due to the test being conducted by an operator less skilful

than the initial operator.

A process has to be under control before the quality of the results can be

managed. Timely recording of testing performance and the process charts provide

-21-

simple practical means of finding the level of control of the operations, the

quality of testing and pinpointing the forthcoming problems possibly causing

quality nonconformance.

4.4 Interlaboratory tests in estimating laboratory performance and the uncertainty

of testing

Interlaboratory tests can be used to study the reliability of test methods (ISO

5725), performance of testing laboratories (proficiency testing) and to estimate the

uncertainty of laboratory performance. Interlaboratory tests need to have a

relatively high number of participants to yield statistically valid results. Thus such

test programmes can be arranged only at relatively long intervals. Proficiency

testing and estimation of uncertainty of test performance can be carried out

between the interlaboratory test programmes through the use of materials whose

performance (level of parameters, repeatability, reproducibility, long-term stability

etc) has been established in statistically valid comparisons (interlaboratory tests).

Such materials can, as mentioned above, be certified reference materials. Thus

CRMs can be used in laboratories to estimate the development of uncertainty of

laboratory performance.

The repeatability value (r) defines the value below which the absolute difference

between two single test results obtained under identical (repeatability) conditions

may be expected to lie with a probability of 95%, whereas reproducibility value

(R) gives the equivalent limit value when using the same method and test material

but in different conditions (time of test, equipment, personnel, laboratory)4.

The within-laboratory long-term reproducibility value can be used as a conserva-

tive estimate for uncertainty of laboratory performance when using the test

method.

4 definition, ISO 5725

-22-

5 SOME SOURCES OF CERTIFIED MATERIALS

There is several sources to get certified reference materials. Some of the best

known are the Community Bureau of References, BCR, the National Institute ofStandards and Technology, NIST, Laboratory of the Government Chemist, LGCand Analytical Quality Control Services, AQCS, which is related with Interna-tional Atomic Energy Agency, IAEA.

5.1 BCR

The role of the Community Bureau of References (BCR) programme of the

European Community is to solve the major technical difficulties which arise in the

measurements done in the implementation of Community Directives or in apply-

ing standardized methods of testing.

BCR operates through collaborative programmes (often interlaboratory test

programmes). The programmes have multiple stages with the aim of harmonizing

the performance of the participating laboratories. Large numbers of laboratories

carry out a particular test or measurement, and the inherent results with a high

level of confidence are generally used to produce reference materials (CRMs). By

autumn 1991 the BCR programme had produced in approximately 550 projects

over 300 CRMs, and the CRMs are used in over 2500 laboratories.

For the mechanical tests of metals BCR can supply a reference material for

Charpy impact testing and is planning to certify a CRM for tensile testing

(NIMONIC 75, CRM 425, which is currently available as a creep testing refer-

ence material).

If a laboratory has a specific problem due to lack of reference materials it should

contact BCR and initiate a collaborative programme on the topic.

-23-

5.2 NIST

The National Institute of Standards and Technology (NIST) offers for sale

primarily NIST Standard Reference Materials (SRMs). These are well-character-

ized materials produced in quantity to improve measurement science. SRMs are

certified for specific chemical or physical properties. In their certificates NIST

reports the results of the characterization and indicate the intended use of

material.

NIST has the function to develop, produce, and distribute SRMs that provide a

basis for comparison of measurements on materials, and that aid in the control of

production processes. To perform this function, the Office of Standard Reference

Materials evaluates the requirement of science, industry, and government for

carefully characterized reference materials, and their production and distribution.

6 COMAR DATABASE

The Reference Material Committee (REMCO) of ISO adapted and started jointly

to develop the French reference material database originated by Service de

Matriaux de Rference at the Laboratoire National dEssais (LNE). The original

goal was to provide answers to the question Where can I buy a CRM with the

following properties...? without the need to consult all the catalogue s of the

many suppliers. (Klich 1991)

Code of Reference Materials COMAR has been developed by three contractors;

LNE (France) as the main contractor and LGC (UK, replaced NPL 1988) and

BAM (Germany) as associate contractors. COMAR consists of over 3500 CRMs

from 50 producers; the database contains

* name and general description of the material,

* name and address of the producer,

* form of the material and

* certified properties.

-24-

The COMAR database is delivered on floppy disks and a standard IBM-compat-

ible (286 processor or better, min 30Mb hard disk) computer can run the database.

The user interface is rather easy to learn through a step-by-step menu system

though there seem to be still a number of intrinsic bugs and anomalities.

The database is quite comprehensive and thus searching for relevant reference

materials can be tedious. Good knowledge and experience of the database and

also the tests performed and CRMs needed is necessary to efficiently use the

database.

The database is now reaching the level of international commercial utilization.

Currently the contractors are unfortunately not able to quote uniform prices and

terms for delivery and updating the database. Hopefully an agreement will be

reached in the REMCO meeting in May 1992.

7 NORDIC QUESTIONNAIRE ON REFERENCE MATERIALS

In order to get information on the general knowledge of reference materials and

on use of the RMs in testing laboratories (destructive testing of metals) a ques-

tionnaire was sent to 82 testing laboratories in the Nordic countries. Questions on

the use of testing standards and reference materials for 12 different test methods

was asked. Answers were received from 16 laboratories.

The distribution of the test methods and the reference materials used in these

laboratories is presented table 1.

-25-

Table 1 The test methods used in the laboratories

Test Number Number of labsof laboratories using RMs

Brine1l hardness testing 14 9Rockwell hardness testing 13 9Vickers hardness testing 16 9Tensile testing 13 2Compression testing 5 -Bend testing 12 -Flattening testing 4 -Charpy impact testing 13 4Drop weight testing (DWT) 4 -Drop weight tear testing (DWTT) 2 -Fatigue testing 7 -Fracture mechanical testing 7 -

According to the answers, reference materials are quite widely used in hardness

testing and most of reference materials had some certification; there were 7 supp-

liers. Most of the certificates did not fully correspond with the ISO GUIDE 31-

1981 Contents of certificates of reference materials. Some certificates referred

to national standards (DIN, BSI) and national requirements and were thus mainly

intended for local use; certificates were not very clear in general.

The most popular (and generally also the most trusted) hardness reference blocks

come from Staatliches Materialprfungsamt Nordrhein-Westfalen (MPA-NRW),

whose certificates give good information on the reference blocks and their use.

Calibration (and certification) of these blocks is according to DIN 51303.

The British and the French systems have accredited (or recognized) suppliers of

reference blocks and some of the certificates (most of the new certificates) refer

to the accreditation.

Many laboratories used reference blocks with no certificates; some RMs were

rather old, dating from the 1970s, so the use of the blocks had obviously not

been very frequent.

-26-

One laboratory manufactures in-house reference materials (working standards) for

tensile and Charpy impact testing (RMs according to ISO 442) and supplies one

participating laboratory with tensile test reference material.

There were a number of accredited laboratories from different Nordic countries

among the respondents. Some of these laboratories did not use reference materials

or used uncertified RMs; one laboratory stated that their equipment is traceably

verified once yearly according to standard verification methods and thus no

reference material is needed for quality assurance of testing. These answers

definitely show that the accreditation requirements between the Nordic countries

vary significantly as in some countries the use of CRMs in indirect verification

and between-verification control of equipment is compulsory.

The questionnaire showed that the significance of the use of reference materials

is not very well understood or even identified as a part of quality assurance

activity in a testing laboratory. Availability of reference materials and of general

information on reference materials seems very limited.

8 CONCLUSIONS

The reference materials are an integral part of the quality assurance of testing

laboratories. Test performance between laboratories can be harmonized through

use of certified reference materials, and individual laboratories can continually

follow up their own performance and check the instruments between verifications

and calibrations. Reference materials are used also to calibrate and tune instru-

ments which are difficult or impossible to calibrate with direct methods. RMs are

used as a transfer standard between a high-quality instrument and a working

instrument.

-27-

Recommendation: The importance of attention to quality and traceability of

measurements of testing laboratories cannot be stressed enough. Use of

reference materials in periodic checks, verifications and calibrations is an

essential part of quality assurance. Laboratories should judge for them-

selves how to organize the process; accreditation bodies should stress the

significance of RMs and if possible publish supportive information.

Related standards

There is a definite incompatibility between quality standards (EN 45001, ISO

GUIDE 25), testing standards and the quality of reference materials and partly

also the application of EN 45001 by the national accreditation bodies.

Recommendation: Test methods should be harmonized with the quality standards

and also the available reference materials (requirements in standards have

to be related to physical possibilities and available uncertainty).

Availabilitv of reference materials

The number and quality of reference materials are increasing, although informa-

tion on them has not been spread so well. The active role of BCR has had a

positive effect on the field; bringing laboratories and RM producers (Appendix 5)

together in collaborative programmes will increase knowledge and interest in the

related questions. All CRMs should be produced in a way which would allow for

international use and distribution, preferably through BCR in Europe.

Recommendation: International distribution of CRMs is encouraged; COMAR,

for example, should be updated and used actively.

-28-

Some of the reference materials available are still inadequately documented -

uncertainties are not given and it is difficult to know whether the material has

been adequately studied and the RM produced in a suitable manner. This uncer-

tainty has a negative effect on the willingness of laboratories to use reference

materials.

In some countries RMs are produced by accredited or national calibration labora-

tories according to national standards. Information on the certification/calibration

methods is generally available nationally but not internationally.

Recommendation: The producers of RMs and CRMs should be encouraged to

publish certificates according to ISO GUIDE 31 to help users (laborato-

ries) evaluate and compare possible CRMs.

COMAR database

The COMAR database is an effective and efficient way to find relevant certified

reference materials. The database should be at least nationally available (in France

30 Minitel subscribers are currently connected to COMAR). There are two major

concepts of organizing the use of COMAR nationally; on one hand a national

body (accreditation body, national technical information centre etc) could centrally

be responsible for giving out information on reference materials, on the other

hand national laboratories would know better their own technical field and could

thus better serve testing laboratories. A national solution is, however, necessary

to spread the information to all laboratories needing information on reference

materials.

Certification documents

-29-

REFERENCES

ASTM E 23-88. Standard test methods for notched bar impact testing for metallic

materials.

ASTM E 127l-88. Standard practice for qualifying Charpy verification specimens

of heat-treated steel

EN 45001:1989, General criteria for the operation of testing laboratories

ISO 442-1965. Verification of pendulum impact testing machines for testing

steels.

ISO 9001:1987, Quality systems. Model for quality assurance in

design/development, production, installation and servicing

ISO/IEC GUIDE 25:1990 (E) General requirements for the competence of

calibration and testing laboratories

ISO GUIDE 30:1981 (E), Terms and definitions used in connection with reference

materials

ISO GUIDE 31:1981 (E), Contents of certificates of reference materials

ISO GUIDE 33:1989 (E), Uses of certified reference materials

ISO GUIDE 35:1989 (E), Certification of reference materials - General and

statistical principles

Juran, J.M. 1974. Quality Control Handbook. 3rd ed. New York, McGraw-Hill

Book Company. 1780 p.

-30-

Klich, H. 1991. COMAR Database for Certified Reference Materials. Berlin,

BAM. 4p.

Klich, H. Personal communication and COMAR demonstration, 31.1.1992. Berlin,

BAM.

NAMAS Accreditation standard M10, General Criteria of Competence for

Calibration and Testing Laboratories. 1989. 24p.

NAMAS Information Sheet NIS 13, Traceability, hardness measurements. 1987.

5p.

NIST Standard Reference Materials Catalogue 1990-91, NIST Special Publication

260, U.S. DEPARTMENT OF COMMERCE, National Institute of

Standards and Technology

prEN 10 045 Part 2:1990, Metallic materials - Charpy impact test - Part 2:

Verification of the testing machine (pendulum impact)

Quality Management in the Laboratory, Espoo, Finland, 24. - 25.9.1991, National

Association of Testing Authorities, NATA Australia in conjunction with

the Technical Research Centre of Finland, VTT.

Excerpts from BCR REFERENCE MATERIALS CATALOGAppendix 1 1(2)

INTRODUCTION

For some ten years the Community Bureau of Reference (BCR), a department of the Commission of theEuropean Communities, has provided laboratories with a means of carrying out collaborative measurementstudies in Physical Measurements and Chemical Analysis.

To understand the role of the BCR it is important to remember that common written standards (Norms),although essential to achieve harmonization of measurements, are not sufficient by themselves to avoiddiscrepancies of results.

The principle of this contribution is to bring together some of the most expert laboratories of preferably all theMember States for intercomparison exercises. Results are discussed with all participants. Discrepancies aregiven particular attention to identify their causes. Subsequently, additional measurements are made to verifythat the causes of errors can be eliminated. When necessary several successive intercomparisons are done tostudy the effect of the different parameters.

Certification - Uncertainties - Uses

When the major sources of errors are eliminated and agreement is reached on a value which can be consideredas the most accurate in the current state of the art, it is essential to provide a means of maintaining theconsistency between laboratories and of disseminating the accurate value to other laboratories. The instrumentsor pieces of equipment used to transfer values are called transfer standards in Applied Metrology and referencematerials in Analytical Chemistry.

The Community Bureau of Reference undertakes the establishment of transfer standards and reference materialswhich can be used by any laboratory which wishes to verify its methods and calibrate its instruments.

The uncertainty is by definition the estimate of the range of values within which the true value is most likelyto be. The uncertainties given in this catalogue take into consideration the variations between laboratories andbetween methods and care is taken not to underestimate uncertainties. They are given at a confidence level of95%.

A certified reference material can be used by a laboratory to verify if the method used gives results in agreementwith the certified values. If the result is different the user is immediately warned that he has to look for errorsin his method and to find their causes.

For calibration the reference material is used to assign a value to the indication of his instrument.

Reference materials are used mainly for:

- calibration and verification of measurement processes in routine analytical conditions;

- internal quality control and quality assurance schemes;

- verification of the correct application of standardized methods;

- development and validation of new methods of measurements.

The BCR programme welcomes any suggestion from industrial organisations, official laboratories, researchestablishments or universities, concerning measurements of practical importance that need to be improved orfor which there is a need for reference materials. Further information on the BCR is available on request.

Report

Each sample is usually delivered with a certificate and a report which contains all the data which formed thebasis for the certification. It also gives details on the methods of measurements that were used, recommen-dations on the best methods to use, and advice for the use of the material.

Excerpts from BCR REFERENCE MATERIALS CATALOGAppendix 1 2(2)

Procedure for purchase of BCR reference materials

The certified reference materials which are made available by the BCR were established in response to a needto help laboratories to improve their measurements. Except when batches of reference materials are very limited,the BCR makes no restriction in their availability for these specific uses as described in its catalogue.

The BCR cannot supply quantities of these materials for any other types of uses.

1. Purchase orders for reference materials and requests for information should be addressed to:

COMMUNITY BUREAU OF REFERENCE (BCR)COMMISSION OF THE EUROPEAN COMMUNITIES

200, Rue de la loiB-1049 BRUSSELS

or by telexing:

BCR -BRUSSELS21877 COMEU B

All purchase orders should give: CRM numbers indicated in the catalogue, name of the material and numberof units required.

2. Delivery time: 3-4 weeks maximum after receipt of the purchase order by the BCR.

3. Shipment: prices include handling and normal postage (surface mail, air mail). If airfreight is requested, ithas to be paid by the customer.

4. Terms of payment: the payments shall be made preferably by bank transfer following instructions to befound in the debit note (or invoice) that the customer will receive after delivery of the samples.Methods of payments such as credit letter are not acceptable. Prices of reference materials are shown inthe price list attached. Payments are expected within 30 days of receipt of the debit note.

Appendix 2 l(2)

COMMISSION OF THE EUROPEAN COMMUNITIES

COMMUNITY BUREAU OF REFERENCE - BCR Ns. 27 to 56EI

CERTIFIED REFERENCE MATERIALCERTIFICATE OF MEASUREMENT

Batch E

CRM 016

IMPACT TOUGHNESSOF V-NOTCH CHARPY SPECIMENS

(Energy level 120 J Nominal)

Impact energy at 20 C Cenified value (1)

According ASTM E 23-82 124.8 J

According ISO 121.5 J

Uncertainty (2)

5.0J

4.7J

(1) This is the mean impact energy for the 5 specimensdelivered in one set

(2) The uncertainty includes mainly the effect of thevariation between samples

DESCRIPTION OF THE REFERENCE MATERIAL

A unit comprises 5 specimens which must be broken successively in the machine to verify.The specimens are packed in protective oil and cleaning must done as recommended.

Nordtest Nordtest

Appendix 2 2(2)

INSTRUCTTONS FOR USE

Specimens should be kept in their original packing until they have to be used.

Special attention is drawn to cleaning, for which vigorous methods should be avoided(e.g. brushes or ultrasonic). The following is recommended :

1. Wipe excess oil with cellulose paper2. Immerse specimens in a clean bath of degreasing solvent for about 5 minutes3. Wipe the specimens with cellulose paper4. Immerse the specimens in a bath of fresh absolute alcohol5. Wipe with paper and allow to dry in still air6. Keep at room temperature for 24 hrs prior to testing

PARTICIPATING LABORATORIES

National Physical Laboratory, Teddington (UK)Material Testing Laboratory, Watertown (USA)Cogne, Aosta (IT)Franco Tosi Industriale, Legnano (IT)Cermet, Bologna (IT)Equipos Nucleares SA, Santander (ES)Bundesanstalt fr Materialforschung und-prfung, Berlin (DE)Staatliches Materialprfungsamt, Dormund (DE)Ugine, Isbergues Research Centre (FR)Laboratoire National dEssais, Paris (FR)Aubert et Duval, Gennevilliers (FR)Aubert et Duval, Les Ancizes (FR)Centre de Recherches et dEtudes dArcueil (FR)Sollac, Dunkerque (FR)GTS Industries, Dunkerque (FR)

METHODS USED

Charpy Testing Machine according to ASTM E23-82Charpy Testing Machine according to ISO (R 442)

LEGAL NOTICE

This document was prepared under the sponsorship of the Commission of the EuropeanCommunities. Neither the Commission of the European Communities nor any personacting on its behalf makes any warranty or representation, express or implied, that theuse of any information, material, apparatus, method or process disclosed in this documentdoes not infringe privately owned rights ; or assumes any liability with respect to the useof, or for damages resulting from the use of, any information, material, apparatus,methods or process disclosed in this document.

CERTIFICATION REPORT

The work done to certify this reference material is described in a report which isavailable to users free of charge upon request.

Provisional address : rue de la Loi 200, B-1049 Brussels - Phone (32 2) 235.50.14 - Fax (32 2) 235.80.72

Nordtest Nordtest

Nordtest Nordtest

Nordtest Nordtest

Nordtest Nordtest

Nordtest Nordtest

Appendix 3 l(1)

Producers of RMs and CRMs:

- Acco Industries Inc. (ACCO)

- The American Oil Chemists Society (AOCS)

- Analytical Quality Control Services (AQCS)

- Centre Mcanique Chimie Matriaux (CMCM)

- Community Bureau of Reference (BCR)

- Eidgenssische Materialprfungs- und Forschungsanstalt (EMPA)

- Engineering & Scientific Equipment Ltd. (ESE)

- Forschungs- und Materialprfungsanstalt Baden-Wrttemberg (FMPA)

- Foundrax Engineering Products Ltd

- Georg Reichter, Eslinger/Neckar

- Laboratoire National dEsaais (LNE)

- Laboratory of the Government Chemist (LGC)

- National Institute for Environmental Studies (NIES)

- National Institute of Standard and Technology (NIST)

- National Research Centre for Certified Reference Materials (NRCCRM)

- National Research Council Canada (NRC)

- Shimadzu Kyoto

- Staatliches Materialprfungsamt Nordhein-Westfalen (MPA NRW)

- Werk

- Yamato Scientific

TECHNICAL REPORTS FROM THE PROGRAMME GROUP FOR QUALITY ASSURANCE

082

084

104

105

107

107:E

118

119

131

143

144

145

146

147

148

149

164

165

177

178

179

Larsen, B., NORDTEST Guidelines for development of Quality Manuals for testing laboratories. Helsinki1987. Nordtest, NT Techn Report 082. 60 p. (in English and Danish)

Fahln, P., Madsen, E., Johansen, M.M., Rautiainen, L., Siitonen, V., Herje, J. & Sandberg, P.I.,Guidelines for development of Quality Manuals in the field of building physics, heating & ventilationand adjacent technical areas. Helsinki 1987. Nordtest, NT Techn Report 084. 25 p.

Hyrynen, J., Draft NORDTEST guidelines for interlaboratory test comparison, Espoo 1988. Nordtest,NT Techn Report 104. 95 p.

Fremstad, D., Lundgren, B., Olsen. H. & Saarela, K., Guidelines for development of Quality Manuals inthe field of chemical analysis. Espoo 1988. Nordtest, NT Techn Report 105. 32 p.

Haglund, M., A check list concerning the cost/benefit of laboratory accreditation. Espoo 1989.Nordtest, NT Techn Report 107. 18 p. (in Swedish)

Haglund, M., Check-list concerning the advantages, cost and disadvantages of laboratoryaccreditation. Espoo 1990, NT Techn Report 107 Engl, Edition 2. 9 p.

Hyrynen, J., Interlaboratory testing. Espoo 1989. Nordtest, NT Techn Report 118. 11 p.

Quality assurance in testing laboratories, Espoo 1989. Nordtest, NT Techn Report 119. 156 p. (inDan/Nor/Swed/Engl)

Quality assurance in testing activities. Espoo 1990. Nordtest, NT Techn Report 131. 261 p. (inDan/Nor/Swed/Engl)

Larsen, J.B., An examination of the costs in establishment and maintenance of an accreditation andthe including quality control system. Espoo 1991. Nordtest, NT Techn Report 143. 106 p.

Ploug, H.H., Nordtest practical guidelines for round robin tests. Espoo 1991. Nordtest, NT TechnReport 144. 28 p.

Lindskov Hansen, S., Guidelines for the development of software to be used in test and measuringlaboratories. Espoo 1991. Nordtest, NT Techn Report 145. 46 p.

Ohlon, R., About procedures for internal quality audits of testing laboratories. Espoo 1991. Nordtest,NT Techn Report 146. 39 p.

Andersson, H., An introductory study of calibration in testing laboratories. Espoo 1991. Nordtest, NTTechn Report 147. 30 p.

nk, S. & Sillanp, J., Requirements on the personnel and organization of testing laboratories.Espoo 1991. Nordtest, NT Techn Report 148. 70 p. (in Swedish)

Forstn, J., A view on the assessment of the technical competence of testing laboratories. Espoo1991. Nordtest, NT Techn Report 149. 46 p.

Sillanp, J., Requirements of ISO 9002 for testing and inspection. Espoo 1992. Nordtest, NT TechnReport 164. 42 p.

Baade, S., Interpretation of EN 45 001 for fire testing laboratories, Espoo 1992. Nordtest, NT TechnReport 165. 25 p.

Hyrynen, J., Kallio, H. & Kosonen, A.-M., Reference materials in quality assurance of testinglaboratories. Espoo 1992. Nordtest, NT Techn Report 177. 35 p.

nk, S. Qualification system for the personnel of testing laboratories. Espoo 1992. Nordtest, NTTechn Report 178. 17 p.

Ohlon, R., Comparison of standards with requirements on calibration and testing laboratories, Espoo1992. Nordtest, NT Techn Report 179. 71 p.

nordtestMailing Address: Tekniikantie 12 FIN-02150 Espoo, Finland. Phone +358 9 455 4600, fax +358 9 455 4272

www.nordtest.org