greg miller harmonization norcal aacc 041812

From chaos to order: harmonization of clinical laboratory test results

Greg Miller, PhD Virginia Commonwealth University

Richmond, VA, USA

H rm nzati on ia o .net

Outline

Why do we need harmonized results

How to achieve results traceable to a reference system

Barriers to harmonized results

AACCs global harmonization initiative

Primary reasons for testing

To identify and manage diseases in an individual patient

To develop data from which to establish clinical practice guidelines for managing diseases

Good laboratory medicine requires:

Total error of measurement is small enough that a result reflects a patients biological condition

Comparable results that are independent of

where and when a test was performed

the measurement procedure used

Total Error

Calibration bias to an accepted reference (includes calibrator lot-to-lot variability)

Imprecision in a measurement procedure

Sample specific influences

non-specificity for the measurand

interfering substances

Why do we need comparable results

Clinical practice guidelines are established for interpretation of lab test results

If different measurements give different results for the same patient sample:

The interpretive guidelines become less useful

Patients may receive incorrect treatment

How to achieve comparable results

Calibration of all measurement procedures is traceable to a common reference system

Consistent performance is monitored and maintained by surveillance using PT or EQA

ISO 17511

In vitro diagnostic medical devices - Measurement of quantities in biological samples - Metrological traceability of values assigned to calibrators and control materials

Category Reference

measurement procedure

Primary (pure substance) reference material

Secondary (value

assigned) reference material

Examples

1 Yes Yes Possible Electrolytes,

glucose, cortisol

2 Yes No Possible Enzymes

4 No No Yes Proteins,

tumor markers, HIV

5 No No No Proteins, EBV, VZV

3 Yes No No Hemostatic factors

S

t

a

n

d

a

r

d

i

z

a

t

i

o

n

Harmonization

Traceability categories from ISO 17511

Terminology

Standardization: results are uniform among measurement procedures

traceability is established to SI and a reference measurement procedure

Harmonization: results are uniform among measurement procedures

NO reference measurement procedure and no pure substance reference material exists

Primary Reference Material

(pure substance)

SI unit

Traceability (based on ISO 17511)A reference system (ideal)

Primary Reference Measurement Procedure

(e.g. gravimetry)Primary Calibrator

Secondary Reference Material

(matrix)

Secondary Reference Measurement Procedure

(e.g. IDMS)


(NIST SRM 917b crystalline glucose)

SI unit (glucose, mmol/L)

Traceability (based on ISO 17511)A reference system for glucose

Primary Reference Measurement Procedure(gravimetry, calibrated with

NIST mass standards)Primary Calibrator(glucose in water, 1, 3, 6, 11 mmol/L)

Secondary Reference Material(NIST SRM 965b glucose in frozen human serum)

Secondary Reference Measurement Procedure

(IDMS)

Reference Procedure


(pure substance)

(e.g. IDMS)

SI unit

Traceability (based on ISO 17511)

Routine Procedure

Patient sample result

T

R

A

C

E

A

B

I

L

I

T

Y

P

a

t

i

e

n

t

s

a

m

p

l

e

r

e

s

u

l

t

s

a

r

e

e

q

u

i

v

a

l

e

n

t

t

o

t

h

e

r

e

f

e

r

e

n

c

e

p

r

o

c

e

d

u

r

e

r

e

s

u

l

t

s


(matrix)Mfr Selected Procedure

Mfr Product Calibrator

Mfr Working Calibrator

Mfr Standing Procedure

(calibrator)

Reference Procedure


(pure substance)

(e.g. IDMS)

SI unit


Routine Procedure


T

R

A

C

E

A

B

I

L

I

T

Y

P

a

t

i

e

n

t

s

a

m

p

l

e

r

e

s

u

l

t

s

a

r

e

e

q

u

i

v

a

l

e

n

t

t

o

t

h

e

r

e

f

e

r

e

n

c

e

p

r

o

c

e

d

u

r

e

r

e

s

u

l

t

s

Mfr Selected Procedure




(calibrator)Panel of patient

samples

Traceability requires commutable calibration

materials

Commutable means that values measured for a calibration material and for native clinical samples have the same relationship between two, or more, measurement procedures for the same measurand.

02

4

6

8

10

0 2 4 6 8 10Measurement Procedure 1

M

e

a

s

u

r

e

m

e

n

t

P

r

o

c

e

d

u

r

e

2

Clinical SamplesReference Materials

Commutable: same relationship for clinical samples and reference materials

02

4

6

8

10

0 2 4 6 8 10Measurement Procedure 1

M

e

a

s

u

r

e

m

e

n

t

P

r

o

c

e

d

u

r

e

2

Clinical SamplesReference Materials

Non-commutable: different relationship for clinical samples and reference materials

Use of a non-commutable material for calibration traceability will cause:

Incorrect value assignment for a routine (field) measurement procedure calibrator

Incorrect results for patient samples

Miller, Myers, Rej. Why commutability matters. Clin Chem 2006; 52: 553-4 .

cholesterolcreatinine

glucose

uricacidhomocysteine

urea ALTAST HbA1c

HCG

TSH PSA

Troponin

I

EpsteinBarrvirus

Picking the low-hanging fruit !omics

What happens when there is no reference measurement procedure

Mfr Selected Procedure





Routine Procedure


T

R

A

C

E

A

B

I

L

I

T

Y

P

a

t

i

e

n

t

s

a

m

p

l

e

r

e

s

u

l

t

s

a

r

e

t

r

a

c

e

a

b

l

e

t

o

a

r

e

f

e

r

e

n

c

e

m

a

t

e

r

i

a

l


(matrix)

Value assignment Commutability

Value assignment when there is no reference measurement procedure

International conventional reference material

Arbitrary e.g. U/L

Bioassay for hormone activity

An arbitrary designated comparison procedure

Traceable to an international conventional reference material

The true value is not known

The goal of harmonization is to have results that are comparable irrespective of the measurement procedure used

Clinical guidelines can still be implemented

Examples: traceable to a reference material

Human chorionic gonadotropin

Prostate-specific antigen

Thyroid stimulating hormone

Human immunodeficiency virus

(no reference measurement procedure)

What happens when there is both:

no reference measurement procedure

no reference material





Routine Procedure


T

R

A

C

E

A

B

I

L

I

T

Y

P

a

t

i

e

n

t

s

a

m

p

l

e

r

e

s

u

l

t

s

a

r

e

n

o

t

t

r

a

c

e

a

b

l

e

t

o

a

n

y

i

n

t

e

r

n

a

t

i

o

n

a

l

r

e

f

e

r

e

n

c

e

There is no coordination among manufacturers

Reference interval concept is used to interpret results

Examples: traceable to a manufacturers working calibrator

B-type natriuretic peptide

Epstein-Barr virus

Varicella zoster virus

(no reference material nor reference measurement procedure)

ISO 17511 category 5 needs practical procedures to achieve harmonization

Possibilities:

traceable to an all methods mean (outliers removed) of a panel of patient samples?

traceable to a designated measurement procedure (arbitrary, but with good correlation with clinical outcome)?

Practical Considerations

Panel of patient

samples

Reference Procedure

Primary Ref Material

(pure substance)(e.g. IDMS)

SI unit

Traceability (an application)


Routine Procedure

Results equivalent to those from

RMP

Value is assigned to produce traceable results for patient samples

Panel of patient

samples

Reference Procedure

Primary Ref Material

(pure substance)(e.g. IDMS)

SI unit

Traceability (an application)


Routine Procedure

Results equivalent to those from

RMP

May be non-commutable, but has a procedure-specific factor to correct

non-commutability bias

02

4

6

8

10

0 2 4 6 8 10Ma

n

u

f

a

c

t

u

r

e

r

s

r

o

u

t

i

n

e

p

r

o

c

e

d

u

r

e

Reference procedure

Clinical SamplesProduct Calibrator

The known matrix-related bias can be offset in the procedure-specific value assigned to product calibrators

so that the results for patients match those from the RMP

A manufacturer's product calibrator is intended for use with a specific measurement procedure

It cannot be used with a different manufacturers measurement procedure

Important


(calibrator)

Procedure nProcedure 3

Procedure 2Procedure 1

Patient Samples

Traceability to a Reference Material

Reference Material must be commutable with patient samples for all measurement procedures with which it will be used

Results nResults 3

Results 2Results 1

Many secondary reference materials are not commutable with native clinical samples for routine clinical laboratory procedures

The Problem


Historically, commutability of reference materials was frequently not validated for use with routine clinical laboratory measurement procedures

The Problem


A manufacturers standing procedure is frequently the same as the clinical laboratory procedure but calibrated with a master lot of calibrator that may be traceable to a non- commutable reference material

The Problem


A manufacturers standing procedure is frequently the same as the clinical laboratory procedure but calibrated with a master lot of calibrator that may be traceable to a non- commutable reference material

The Problem

A non-commutable calibration material

breaks the traceability chain


Even though manufacturers claim traceability, the process fails to provide equivalent results for patient samples when different measurement procedures are used

The Problem

Secondary reference materials may not be commutable

A reference material may not have been validated to be suitable as a calibrator for all routine measurement procedures

Limitation may apply to:

National metrology institutes (e.g. NIST, IRMM)

Other providers of standards and calibrators such as: WHO, professional organizations, commercial providers

TSH methods All traceable to IS 94/674 (WHO)

Thienpont et al. Clin Chem 2010; 56: 902-911.

Mean 95% CI for 40 patient samples

Calibration traceability does not ensure accuracy for an individual patient sample

Measurement procedure may not be specific for the measurand

Measurand may not be well defined

Molecular form(s) of clinical interest

Interfering substances present in a patients sample may influence the result

Other examples:

Follicle stimulating hormone (Clin Chim Acta 1998;273:103-17)

Prostate-specific antigen ( Clin Chem 2006;52:59-64)

C-peptide (Clin Chem 2008;54:1023-6)

Insulin (Clin Chem 2009;1011-1018, 2009)

Human chorionic gonadotropin (Clin Chem 2009;55:1484-91)

Cytomegalovirus (Clin Chem 2009;55:1701-10)

Troponin I (Pathology 2010;42:402-8)

Prostate-specific antigen (Clin Chem 2011;57:1776-7)

What about new biomarkers

A new biomarker is shown to be clinically useful

A measurement procedure is commercialized

A 2nd procedure is commercialized

If the 2nd and subsequent procedures do not give comparable results, the clinical trial interpretive guidelines become less useful

Sustainable calibration traceability needs to be part of the development process:

Definition of the measurand

Requirements for measurement specificity

Reference measurement procedure

Commutable reference materials

New biomarkers

What do we do?What do we do?

Must change practice to require commutability validation for reference materials intended for use with:

Manufacturers standing procedures

Routine clinical laboratory procedures

A guideline is available: CLSI C53-A Characterization and Qualification of Commutable Reference Materials for

Laboratory Medicine (2010)

Commutability: who is responsible

Reference material manufacturer

Cannot know all procedures in use

Should make a material likely to be commutable

Can address commonly used procedures

Measurement procedure manufacturer

Must confirm commutability for an intended use

Progress has been made!

However,

The approach has been ad-hoc based on individual interests

There is no central organizing body to coordinate the work of different organizations

A number of organizations are addressing standardization / harmonization around the world

IFCC has been a key leader in developing science

Many national societies are engaged

JCTLM

Credentialing function; maintains lists of reviewed and approved:

Reference measurement procedures

Reference laboratories

Reference materials Commutability information is not required for listing

Joint Committee for Traceability in Laboratory Medicine

International Committee of Weights and Measures International Federation for Clinical Chemistry and Laboratory Medicine International Laboratory Accreditation Cooperation

How is calibration traceability sustained

Certification programs

e.g. CDC Lipids, NGSP HbA1c

Surveillance by PT / EQA

Requires commutable materials

Conformance to ISO 17511

e.g. CE mark in the EU

Barriers to harmonization

Lack of a systematic process to identify and prioritize measurands

Materials are labeled as reference materials that have not been validated to be commutable for the intended measurement procedures

Inadequate definition of the measurand

Inadequate analytical specificity for the measurand

Lack of systematic procedures to implement harmonization, in particular:

when there is no reference measurement procedure

when there is no reference material

Clin Chem Lab Med 2008;46:1220-2.

Challenged clinical laboratories to address the issues associated with harmonization

have the courage to agree on pragmatic solutions

Prince Harmonization finds the Sleeping Beauty

Report from an AACC conference, October, 2010: Improving Clinical Laboratory Testing through

Harmonization: An International Forum

August 2011

The Roadmap

Develop an infrastructure to coordinate harmonization activities world wide to include:

1. Prioritization of analytes

2. Gap analysis for what needs to be done

3. Technical processes to achieve harmonization

4. Surveillance of success of harmonization

Cooperate

With other organizations already working to improve standardization / harmonization

Provide a communications portal and prioritization scheme among organizations and standardization / harmonization activities

Focus on measurands for which no reference measurement

procedure exists

Measurands in ISO 17511 categories 4 and 5 have been technically more difficult to address, thus there have been few effective procedures implemented for harmonization in these categories.

AN INFRASTRUCTURE FOR HARMONIZATION

Harmonization Oversight Group

Evaluate measurand proposals

Clinical practice groupsLaboratory practice groupsIVD manufacturers

Metrology institutesStandards organizationsRegulatory organizations

Solicit champion and funding Clinically affected entity Economically affected entity

Create a Harmonization Implementation Group Technical plan Surveillance plan Implement the plans Achieve JCTLM listing

If measurand is not being addressed, create aSpecialty Work Group

Review priority and feasibility using a checklist

Recommendation to Harmonization Oversight Group

, If work underway, refer

to that group If RMP possible, refer

to another group

Public health orgs

A Steering Committee is responsible to implement the infrastructure for harmonization

3 Task Forces are working to develop operating procedures for the major components

1. Establishing the harmonization oversight group

2. Developing checklists for prioritization and gap analysis

3. Developing a tool box of technical processes for assessment and harmonization of measurands

Path Forward

Guiding Principles

The goal is harmonized results that are fit for the intended clinical purpose. Pragmatic approaches will be required.

An open and transparent process is required.

Success will depend on collaboration among all stakeholders world wide.

www.harmonization.net

Communication center for the project

Status reports on measurands

Communication with stakeholders

Will become a general information portal for global standardization / harmonization activities

Useful technical information

Information on global activities

Links to other organizations

H rm nzati on ia o .net

Coming soon: International Program for Harmonization of Clinical Laboratory Results

From chaos to order: harmonization of clinical laboratory test results Greg Miller, PhDVirginia Commonwealth UniversityRichmond, VA, USAOutlinePrimary reasons for testing Slide Number 4Slide Number 5Slide Number 6Slide Number 7Slide Number 8Slide Number 9Slide Number 10Slide Number 11Slide Number 12Slide Number 13Slide Number 14Slide Number 15Slide Number 16Slide Number 17Slide Number 18Slide Number 19Slide Number 20Slide Number 21Slide Number 22Slide Number 23Slide Number 24Slide Number 25Slide Number 26Slide Number 27Slide Number 28Slide Number 29Slide Number 30Slide Number 31Slide Number 32Slide Number 33Slide Number 34Slide Number 35Slide Number 36Slide Number 37Slide Number 38Slide Number 39Slide Number 40Slide Number 41Slide Number 42Slide Number 43Slide Number 44Slide Number 45Slide Number 46Slide Number 47Slide Number 48Slide Number 49Slide Number 50Slide Number 51Slide Number 52Slide Number 53Slide Number 54Slide Number 55Slide Number 56Slide Number 57Slide Number 58Slide Number 59Slide Number 60Slide Number 61Slide Number 62Slide Number 63

greg miller harmonization norcal aacc 041812

Documents