a basic introduction to reference materials€¦ · pops strategy 2009-2010+ a basic introduction...

POPs Strategy 2009-2010+A basic introduction to reference materials

A tutorial A tutorial

16 September 200916 September 2009

Angelique Angelique BothaBotha

R&D metrologistR&D metrologist

Contents

�Why do we need reference materials?� comparability of results� metrological traceability

�What are reference materials?� Reference materials� Reference materials� Certified reference materials

�Uses of reference materials�How to select a reference material?

Traceability and UncertaintyReality in measurements?

IRMM /IMEP proficiency test studies

IMEP-16: Pb in Wine (Proficiency testing)

� Results submitted: December 2000� IMEP Report: September 2001� Published data: JAAS, (2001), 9: 1091-1100� Metrologia� Coordinator: IRMM, JRC European Commission� Participants: 129 laboratories, 38 countries

Reliable measurement results

Approach 1: Standardisation

“harmonised methods”

Approach 2: Metrology

“ultimate analysis”• Traceability• Measurement uncertainty

Prescribed methods/procedures

method-defined parameters

required quality assurance?

prescribed method-performance

characteristics

“SI traceable” parameters

Legislation

ISO 170251. Scope2. Normative references3. Terms and definitions4. Management requirements5. Technical requirements5. Technical requirements

5.6 Measurement traceability

5.6.1 General“ All equipment used for tests and/or calibrations…shall be calibrated before being put into service. The laboratory shall have an established programme and procedure for the calibration of its equipment.”

Metrological traceability

‘Property of a measurement result whereby the result can be related to a stated metrological reference through a documented unbroken chain of comparisons, all having stated measurement uncertainties .’

VIM 2007

Why?‘Only measurement results where the metrological traceability is established can be compared, independent from when and where they were obtained.’ - IUPAC

Traceability – key points

�Property of measurement result – methods are not traceable

�Closely linked to measurement uncertaintyUncertainty + traceability = 2 main factors assuring measurement reliabilityreliability

�Traceability to a common reference allows comparability of results

�Traceability chain = sequence of standards and comparisons that relate the result to a stated reference

Traceability – final reference

Important : the traceability chain shall be linked to a final reference :� a measurement unit� a measurement procedure� a reference material (“artefact”)

Traceability to a unit� Preferably to the SI unit

� e.g. “mass: 5 kg” means: the measurement result of the property “mass” has been obtained by comparing the mass of the item with the mass of the kg in Paris

� Exists for most physical measurements – independent of the methodthe method

� Not (yet) available for many chemical measurements:� “kg ≠ kg ”; may depend on the mehod employed� SI unit is not always relevant for the application (mass

vs. catalytic concentration of an enzyme)

Traceability to a procedure

A problem?No! These methods define the analytes

� “dietary fibre” has no real meaning� a certified value without a statement of the � a certified value without a statement of the

method is meaningless� the only useful reference is the

procedure

Traceability to a reference material (“artefact”)

Important for arbitrary units� use the reference material directly as

comparison� can be method-dependent or –independent� can be method-dependent or –independent� e.g. WHO standard: “by definition, this

sample has 5000 IU”

Traceability to institutes?‘The results are traceable to NIST’

Incorrect! – what is meant is for example:

“The mass obtained is traceable to the kg in Paris, “The mass obtained is traceable to the kg in Paris, because it has been compared with a weight from NIST, which in turn has been compared to the kg in Paris.”

What matters is the end-point of the traceability chain

Metrological traceability in practice…more complex?

Example of the preparation of a Cd calibration standard for AAS:“Weigh 10 g of cadmium metal, add nitric acid to dissolve the metal, transfer the dissolved metal to a volumetric flask, fill volumetric flask to the mark”

� Analytical system: complete set of operations� Weighing, heating, dissolving, quantitative transfer, fill, etc.� Weighing, heating, dissolving, quantitative transfer, fill, etc.

� Each operation has to be calibrated� Mass for balances� Not all operations are critical for a particular measurement result, i.e.

required accuracy of the reference depends on the impact of the operation on the final analytical result

�analyst’s (competent) judgement

� Material� Sufficiently homogeneous and stable� Fit for its intended use in a measurement proces� NOTES

� Generic term (family name)

Reference material (RM) - ISO Guide 30: 2008

� Quantitative and qualitative properties� Uses� Can only be used for a single purpose in a given

measurement� VIM3 definition

� Reference material� Metrologically valid procedure� Certificate� Certified value has an uncertainty and statement of

metrological traceability

Certified reference material (CRM) - ISO Guide 30: 2008

metrological traceability� NOTES

� Quantitative and qualitative properties and their uncertainties� Metrologically valid procedures can be found in ISO Guides 34

and 35� ISO Guide 32 gives guidance on the certificate� VIM3 definition

Reference materials

material characteristics

additional requirements

Other reference materials

Certified RMs

� Homogeneous� Appropriate stability


Metrologically valid establishment of property values

accompanying information

main applications

property values

Statements on homogeneity and stability

� property value(s) traceable to reference system

� measurement uncertainty� stated homogeneity and

stability� intended use

� calibration� trueness control� full method validation� all QA/QC issues

� performance controls� method development

CRMs for short-cutting the traceability chain

� CRM = RM + certificatecertified value with uncertaintystated traceability (to a final reference)

� CRMs to ensure traceability of 1. Measurement conditions, classical ‘calibration’1. Measurement conditions, classical ‘calibration’2. Results of final quantification (through calibration)3. Results of overall method (through validation)4. Results of daily work

Physical standards = classical ‘calibration’ in the lab� weights� Volumetric flasks� Thermometers� Pipettes and burettes, etc.

! Must be calibrated to ensure accuracy! Must be calibrated to ensure accuracy

1. Physical standards to ensure correctness of measurement conditions

� Correct weights, temperatures, volumes, ….� Basic requirements for all measurements� Ensures that these operations are the same all the time,

everywhere

2. CRMs for quantificationsample

Weiging, milling

extraction

?Clean up

Fill to volume

quantification

?

2. CRMs for traceability in final quantification

Analysis resultReference standardcomparison

• Standards with stated purity and uncertainty

ISO 17025 section 5.6.3.2: shall, where possible, be traceable to SI units

2. Matrix CRMs for calibration?

�Usually not encouraged (uncertainty too large)

�For some methods necessary� In the absence of pure substance materials� In the absence of pure substance materials

2. CRMs and traceability


2. Pure substance CRMs to ensure correctness and traceability of final quantification

� Substances must have stated identity, purity, concentration and uncertainty = must fulfil all concentration and uncertainty = must fulfil all requirements of CRMs

� Concentrations/purities must themselves be traceable: ISO Guide 31 demands traceability statement on the certificate

� Standards should be produced in conformance with ISO Guide 34. Third party assessment?

3. CRMs for validationsample

Weighing, milling

extraction

?

Clean up

Fill to volume

quantification

?

?

3. CRMs for validation

Analysis result Reference standard

sample

?comparison

?

Traceability in sample preparation

� Problem : every sample preparation step breaks the traceability chain� One is not entirely sure that what went into the sample

preparation step is the same that came out� Method assumes correctness, absence of losses, etc.

� Any evidence?� Any evidence?� Therefore: ways to “restore” the traceability chain are

needed: method validation� Matrix CRMs can be used for method validation

Restoring the traceability chain

sample

weigh the sample

extraction/digestion

balance calibration

CR

Ms

impossible to link sample to final extract

X

clean up

dilution to a certain volume

quantification

calibrated glassware

pure standards

Mat

rix C

RM

s

final extract

X

3. CRMs and traceability


2. Pure standard CRMs to ensure correctness and traceability of final quantification

3. Matrix CRMs in method validation prove 3. Matrix CRMs in method validation prove trueness , absence of losses and interferences and link the final quantification to the sample

ISO 17025 Section 5.9: These demonstrations must be regularly repeated – ‘regular use of CRMs’

4. CRMs in daily work

�Demonstration that conditions of validation study apply for this particular measurement: quality control charts

� In practice: CRMs often used� Guaranteed homogeneity� Guaranteed homogeneity� Better/documented stability compared to

normal samples� Ultimate test of accuracy and traceability of

results in each measurement series

4. CRMS in daily work1. Physical standards to ensure correctness of

measurement conditions2. Pure standard CRMs to ensure correctness

and traceability of final quantification3. Matrix CRMs in method validation prove

correctness, absence of losses and interferences and link the final quantification to the sample

4. (C)RMs in control charts prove that the method has been applied correctly and that the conditions from the validation study apply

Conclusions

sample

Weigh the sample

Extraction/digestion

Balance calibration

Mat

rix C

RM

sISO 17025 demands proof of traceability = comparabilityTraceability can be established by using several classes of RMs

Clean up

Dilution to a certain volume

quantification

Calibrated glassware

Pure standards

Mat

rix

Conclusions

sample

weigh the sample


physical standards

CR

Ms

�ISO 17025 demands proof of traceability = comparability�Traceability can be established by using several classes of CRMs


clean up

dilution to a certain volume

quantification pure standards

Mat

rix C

RM

sQ

CM

s

Reference materials

material characteristics

additional requirements

Other reference materials

Certified RMs



Metrologically valid establishment of property values

accompanying information

main applications

property values

Statements on homogeneity and stability

� property value(s) traceable to reference system

� measurement uncertainty� stated homogeneity and

stability� intended use

� calibration� trueness control� full method validation� all QA/QC issues

� performance controls� method development

Uses of reference materials�Calibration�Method development�Method validation

� Evaluation of trueness� Uncertainty estimation

�Proof of method performance�Proof of method performance� Lab internal quality control (‘charting’)� Operator or equipment qualification

�Proficiency testing� Training and verification of competence

(external benchmarking)

Calibration�RMs for calibration requires property values with

stated uncertainty, and stated metrological traceability

�Many laboratories prepare their own RMs for calibration, often denoted as "calibrants" or "calibrators“"calibrators“

�Different types of calibration:� Single point� Bracketing� Multi-point

Single point calibration

samplesample yx=

One calibrant (in this context the CRM) is used to calibrate the measurement equipment, which is then used to assign value(s) to the sample(s) measured

CRM

sample

CRM

sample

y

y

x

x=

Bracketing

� Two calibrants, one with a property value greater than the value(s) of the sample(s), and one with a property value smaller than those values

� By means of linear interpolation between the two calibrants, values are assigned to other samples

( ) 1112

12 xyyyy

xxx samplesample +−

−−=

Multi-point calibration

�Ordinary least squares (OLS)� EXCEL

�Generalised least squares (GLS)� XGENLINE

Method validation

Validation parametersValidation parameters

Concept of method validation

�Method validation is required to establish the fitness for purpose of a method for the specific requirements of customers when applied to a specific laboratory

�Method validation studies produce data on the �Method validation studies produce data on the overall performance or individual influence quantities associated with the results of a method in normal use in the laboratory. This data can then be used for the estimation of the uncertainty associated with the results

Concept of method validation (cont.)

� Data on overall method performance parameters are obtained from method development (optimisation), interlaboratory studies, in-house validation protocols.

� Individual sources of error or uncertainty are typically only investigated when significant compared to the overall precision.precision.

� Emphasis is on identifying and removing /reducing significant effects.

Validation parameters

�Repeatability standard deviation sr

�Reproducibility standard deviation s

Precision

�Reproducibility standard deviation sR

�Intermediate precision szi

Repeatability s r

�Short term precision�Same sample�Same analyst�Same analyst�Same instrument�Same method�Short period of time

Reproducibility s R

�Long term precision�Same sample�Different analysts�Different instruments�Different instruments�Different methods�Different times of the day�Different days�Different laboratories

Average

xx

n

ii∑

== 1

Arithmetic mean

nx i== 1

Standard deviation, s

When repeated measurements give different results, we want to know how widely spread the readings are to be able to judge the quality of the measurements

( ) ( )2

11

1∑

=

−−

=n

iii xx

nxs


�Studied through the use of reference materials or spiking studies.

�Analytical recovery (value observed divided by value expected).

Bias (“Accuracy”)

�Expected to be negligible or accounted for.� Important uncertainty contributor.

100exp

(%)covRe ×=ectedvalue

luemeasuredvaery

t-test: Comparison of an experimental

mean with a know value

( ) nx µ−

and any difference is randomxH =µ:0

µ≠xH :1 and the difference is significant

( )s

nxtcalc

µ−=

05,0;νttcrit = Level of confidence = 95%

Reject if critcalc tt >

Comparison of the means of two

samples• Step 1: Compare the two standard deviations (F-test)

22

210 : σσ =H

22

211 : σσ ≠H

the variances are not significantly different

the variances are significantly different211 : σσ ≠H

22

21

s

sFcalc =

05,0;; 21 ννFFcrit =

the variances are significantly different

Comparison of the means of

two samples• No significant difference

( ) ( )2

11

21

222

2112

−+−+−=

nn

snsns pool

21

21

21

11nn

s

xxt

pool

calc

+

−=

221 −+= nnDF

Comparison of the means of two

samples• Significant difference

22

21

21

n

s

n

s

xxstcalc

+

−=

21 nn+

( ) ( )

+

++

+

=

11 2

2

22

1

1

21

2

22

1

21

n

n

s

n

n

s

n

s

n

s

DF


�Not generally quantified; checked for by inspection or using significance tests for non-linearity(Correlation coefficient, F-test).

�Non-linearity corrected for by non-linear calibration

Linearity

�Non-linearity corrected for by non-linear calibration or a more restricted operating range.

�Remaining deviations from linearity accounted for by overall precision estimates covering several concentrations, or within any uncertainties associated with calibration.


�Determined to establish the lower end of the practical operating range of a method.

�The detection limit is not of direct relevance to

Detection limit

�The detection limit is not of direct relevance to uncertainty estimation

�However, uncertainties near the detection limit may require careful consideration and special treatment

Limit of detection

The concentration of analyte required to give a signal equal to the background (blank) plus three times the standard deviation of the blank

blankblankLOD syy ⋅+= 3 blankblankLOD

LODLOD xy →

y

xLOD say ⋅+= 3

Limit of quantification

For linear regression:

yLOQ say ⋅+= 10x

yLOQ say ⋅+= 10


�Perform tests to determine the effect of one or more parameter changes on the results.

� If the effect is significant a more detailed study is carried out to measure the size of the effect.

Robustness (ruggedness)

carried out to measure the size of the effect.

�Significant effect: the precision deteriorated compared to the precision of the ruggedness test (ANOVA, F-test).


�The degree to which a method responds uniquely to the required analyte.

�Typical studies investigate the effects of likely interferents by adding the potential interferent to both blank and fortified samples and observing the response.

Selectivity/specificity

response.�Normally used to demonstrate insignificant effects.�Can use the data to estimate the uncertainty

associated with potential interferences.

Quality control: the RM� An RM is needed with sufficient homogeneity and stability � Sufficiently stable at least for the period of time that the

precision checking measurements take � If necessary, specific precautions should be taken to monitor the

stability of the RM used � Stability monitoring with the use of a CRM, or by using another

measurement process of which the stability has been demonstrated demonstrated

� Although the RM used for precision taking should not necessarily have known, metrological traceable property values for the properties of interest, precision measures may be dependent on the nominal value of the measurand, so knowledge about the nominal values of the parameters of interest is usually required to assess the appropriateness of the RM selected for the checking

Quality control: the measurement

�The user should perform independent replicate measurements.

� "Independent", in a practical sense, means that a replicate result is not influenced by previous measurements.

�To perform replicate measurements means to �To perform replicate measurements means to repeat the whole procedure.

�For example, in the chemical analyses of a solid material, the procedure should be repeated from the weighing of the test portion to the final reading or calculating of the result

Quality control

�Data treatment� Check for outliers: Dixon’s Q-test, Grubb’s

test

�Assessment of precision�Assessment of precision� Calculate the average and standard

deviation� If you want an accurate nominal value

• Compare with a CRM

Quality control: control charts

nominal value

warning limit

action limit

average

2s

3s

warning limit

action limit

2s

3s

Selection of reference materials

� Intended use�Measurand and measurement range�Matrix match and potential interferences�Measurement uncertainty and traceability

Checklist

�Measurement uncertainty and traceability�Certification procedures�Quality assurance issues�Completeness and transparency of

information�Availability and cost

Intended use of the reference material

�Calibration or quality control?� Pure material vs. matrix material

�Trueness or precision?� Combined uncertainty vs. � Combined uncertainty vs.

microhomogeneity

�Momentary or long-term control?� Combined uncertainty vs. (long-term)

stability

Measurand and measurement range

�Is the measurand for which the RM is certified identical to the measurand of the method to be checked/calibrated?� Measurand definition unambiguous?� Element vs. species specific?� Method dependent of not?

�Measurement range compatible� Concentration?� Expected precision of your method?

Matrix match and potential interferences

�Matrix preferentially identical or similar?� Cod fish vs. cod powder?� Cod powder vs. fish powder? (fat content

similar?)� Fish muscles powder vs. pig meat powder? � Fish muscles powder vs. pig meat powder?

(matrix similar?)� Presence of potential interferences?

Measurement uncertainty and traceability�CRM uncertainty:

� Clear description?� In accordance with the GUM?� Proper consideration of homogeneity and

stability?� Uncertainty budget available?� Does the information provided allow

calculation of standard uncertainty?� Does it satisfy your needs and

requirements? (can this CRM e.g. be used for calibration?)

Measurement uncertainty and traceability

�Certified value� Identity structurally defined?

• If yes: is the quantity value traceabe to the SI

�Identity operationally defined?� E.g. according to a written standard, e.g. ISO � E.g. according to a written standard, e.g. ISO � If yes: is the quantity value

• Traceable to SI?• Traceable to an artefact, e.g. expressed in IU?

Quality assurance issues

� Producer accredited as a reference material producer� If not, quality system according to ISO Guide 34?

� Design and execution of certification process� Selection of collaborators

� ISO 17025 accreditation� ISO 17025 accreditation� Post certification activities (storage, monitoring,

distribution, corrective actions)� If not according to ISO Guide 34, does certification

report provide sufficient evidence to guarantee equivalent quality, e.g. by third party evaluation (Certification committee, Certification advisory panel, ERM panel?)

Completeness and transparency of information

�Availability of� Extended certificate?� Full certification report?

�Sufficient details on�Sufficient details on� Production process?� Individual measurements?� Data handling and treatment?

�Detailed instructions for use?�Application notes/training

Availability and cost

�Where to find it? At what cost?� www/catalogues

�Import/export limitations?� Local legislative restrictions?� Local legislative restrictions?

�Transport possible?� Perishable CRMs to distant countries� Local authorised distributor?

a basic introduction to reference materials€¦ · pops strategy 2009-2010+ a basic introduction...

Documents