“why do environmental health and regulatory programs ......epa methods update rule march 12, 2007...

““Why do Environmental Why do Environmental Health and Regulatory Health and Regulatory

Programs Ignore Laboratory Programs Ignore Laboratory QC data?QC data?””

Henry Leibovitz, Ph.D. Environmental QA/Lab Cert OfficerRI State Health Laboratories

Laboratory Data UsersLaboratory Data Users

Federal and State laws have created the reliance on laboratory testing to support environmental health, regulatory and monitoring programs.

CWA, SDWA, RCRA, CERCLA, Brownfields, CAA, etc.

Relying on Laboratory ResultsRelying on Laboratory Results

Federal, state and municipal Environmental Health and Regulatory Programs utilize laboratory test results to inform decisions affecting public health and the environment.

Environmental programs and utilities may use their own laboratories or contract the testing to laboratories that meet requirements specified in

methods.

Laboratory CertificationLaboratory Certification

Most federal, state and municipal environmental programs require support

laboratories to be certified by a recognized Accreditation Authority.

Quality Assurance and ControlQuality Assurance and Control

Environmental programs and laboratories each maintain and follow their own Quality Assurance Plans to

ensure mission and data quality objectives are met.

Programs Expect LaboratoriesPrograms Expect Laboratories

• Maintain certification and reliability• Follow approved methods• Meet method requirements including

quality controls• Produce accurate and precise data reports

Laboratory Quality AssuranceLaboratory Quality Assurance

Systematic process laboratories follow to produce reliable and reproducible

data.

Laboratory Quality ControlLaboratory Quality Control

Quality control samples are tested with environmental samples in the

laboratory to demonstrate testing is in control (accurate and precise.)

Laboratory Reports Include:Laboratory Reports Include:

• Sample test results (program required)

• QC test results (program discretion)

Lab Reports will include QC results at no extra cost to programs.

Reasons Laboratory QC Data Are Reasons Laboratory QC Data Are IgnoredIgnored

• Assumption - if laboratory is certified then no need to verify data.

• Most federal and state programs don’t require QC or sample data submittal.

• Data review not defined in program Quality Plans

• Programs choose not to use QC data

Laboratory Certification Laboratory Certification Assures CapabilityAssures Capability

• Quality Assurance Plan• Method Detection Limit (MDL) studies• Initial Demonstration of Capability• Standard Operating Procedures• Annual Proficiency Testing• Training Documentation• On-site assessments every 2-3 years

Laboratory Certification Laboratory Certification Does Not Assure AccountabilityDoes Not Assure Accountability

• Are sample testing Quality Controls met?• Are test results and reports reviewed?• Is compliance and monitoring data

acceptable?• Did the laboratory follow approved

methods?

Are Laboratory Results Acceptable Are Laboratory Results Acceptable --Who Decides ?Who Decides ?

• Laboratories decide data is acceptable for public health decision making.

• Programs forfeit this responsibility

EPA Methods Update RuleEPA Methods Update Rule

March 12, 2007 – 40 CFR Parts 122, 136, 141, 143, 430, 455 and 465

“This rule modifies the testing procedures approved for analysis and sampling under the Clean Water Act and Safe Drinking Water Act.”

EPA March 2007 Methods Update RuleEPA March 2007 Methods Update Rule

Most compliance methods require rejection of data that fail QC criteria.

EPA March 2007 Methods Update RuleEPA March 2007 Methods Update Rule

However, to do so when there is not opportunity for reanalysis, resamplingor corrective action may cause a violation of (compliance) reporting requirements.

EPA March 2007 Methods Update RuleEPA March 2007 Methods Update Rule

Regulatory authority should decide whether to reject or accept data that have failed QC specifications in the method, or when problems have occurred during sample collection.

Data Review, Verify or Validate?Data Review, Verify or Validate?

Environmental Programs may accept or reject sample results based on review, verification or validation of QC data results.

Data Quality Objectives (DQO)Data Quality Objectives (DQO)

Program Quality Plans

• Define how laboratory QC data will be used to meet DQO,

• Describe level of review necessary to determine if DQO are met.

EPA OIG Report on EPA OIG Report on Laboratory FraudLaboratory Fraud

• Real concerns about the reliability and quality of laboratory data

• Findings based on improper laboratory practices

• Failure of laboratory to follow method requirements

Unscrupulous LaboratoriesUnscrupulous Laboratories

• Fail to use approved methods• Fail to perform QA/QC testing • Fail to calibrate instruments properly• QA/QC testing “after the fact” to satisfy an

audit or data submission request

What to do about Improper What to do about Improper Laboratory Practices?Laboratory Practices?

• Addressed by data review and verification or validation

• Identify options and resources to assure the laboratories are certified and sample results are acceptable

Program Resources are LimitedProgram Resources are Limited

Many programs do not have the resources or the training necessary to review, verify or validate laboratory data.

Support using 3Support using 3--Way TiesWay Ties

Programs and laboratories rely on each other and on the Laboratory Certification Officer –Build ties between all three

““Solutions to Analytical Chemistry Problems Solutions to Analytical Chemistry Problems with Clean Water Act Methodswith Clean Water Act Methods””

((““Pumpkin BookPumpkin Book”” EPA 821EPA 821--RR--0707--002) 002) March 2007March 2007

Strongly recommends that the supporting QA/QC data be submitted

along with analytical results

EPA EPA ““Pumpkin BookPumpkin Book”” ––Principles of Data ReviewPrinciples of Data Review

Applicable to laboratory methods:• Drinking water methods (500 – series)• SW-846 methods (RCRA methods)• ASTM International Committee D19 (water

methods)• Standard Methods

Laboratory Accountability –Require QC Results in Reports

Problem with Retrieving QC Only Problem with Retrieving QC Only When Questions AriseWhen Questions Arise

• Programs rely upon laboratories to keep QA/QC records unless requested

• QA/QC data maybe components of analytical results for a variety of clients

• Resolving questions of compliance is time consuming and confusing for the program

Retrieving Data on RequestRetrieving Data on Request

Data provided by laboratories may have been compliant . . .

. . . if QC data becomes misplaced there is no way to prove the data was compliant.

Including QC Data in ReportIncluding QC Data in Report

• Quality of the data evaluated directly• Program decides to accept or reject data• Supporting data becomes property of

program• Programs can defend their decisions by

verifying supporting data

Laboratory Laboratory -- Program Accountability Program Accountability --Finding the ResourcesFinding the Resources

• Develop data review capability through training workshops

• Use laboratory and Certification Officer expertise as learning resource

• Utilize new electronic data review software to verify or validate electronic reporting

• Third Party Validation

Data Review Data Review -- What Level?What Level?

Programs must decide and define the level of data review they will require in in their Quality Plan prior to the onset of the project.

Programs should seek a predefined level Programs should seek a predefined level they can achieve with their resourcesthey can achieve with their resources

1. Review QC results only2. Review and verify QC and sample data3. Validate the entire calibration, QC and

sample data.

Random Data AuditsRandom Data Audits

Programs may randomly request entire data set including original QC, calibration and sample data for assessment.

Advantage – Laboratories must always be prepared for data validation by the program.

Standardized Laboratory QA/QC Standardized Laboratory QA/QC (reviewed by certification officer)(reviewed by certification officer)

• Method detection limit (MDL) and minimum level of quantitation

• Initial precision and recovery• Ongoing precision and recovery• Statements of data quality from the

laboratory

Standardized Laboratory QCStandardized Laboratory QC(determined by validation)(determined by validation)

• Purity and traceability of reference standards

• Number of calibration points• Linearity of calibration• Calibration verification

Standardized Laboratory QCStandardized Laboratory QC(reviewed by program)(reviewed by program)

• Analysis of blanks (Method Blank)• Recovery of analytes spiked into a

laboratory reagent blank (LFB, LCS)• Recovery of analytes spiked into the

sample matrix (MS, LFM)

Method Flexibility vs. DeviationsMethod Flexibility vs. Deviations

Data review must distinguish between program allowed flexibility and improper deviations in method implementation.

Method Flexibility vs. DeviationsMethod Flexibility vs. Deviations

For unanticipated situations or where there is ambiguity in the regulations or analytical methods, try working mutual solutions with accreditation or regulatory authority.

Three Way SupportThree Way Support

Environmental Programs

LaboratoryCertification

Laboratories

Improving CommunicationImproving Communication

Environmental Programs

Rarely communicate with laboratories

Establish analytical and reporting requirements

Lack access to laboratories

Laboratory Certification

Routinely communicatewith all laboratories

Communicate program analytical and reporting requirements

Access to all laboratories

Lab Certification Reinforces/Verifies

Lab Capability of MeetingProgram Requirements

Laboratories Provide

Verifiable Data

ProgramsEstablish Requirements, Verify Data Quality –

Approve or Reject Data

Program Support Through CommunicationProgram Support Through Communication

Overlapping ResponsibilityOverlapping Responsibility

Programs

LaboratoriesLaboratory Certification

DATA QUALITY

It Takes a Laboratory!It Takes a Laboratory!

Have you ever received QC data with your routine clinical laboratory test result?

It Takes a Laboratory!It Takes a Laboratory!

ClinicalRequires 1000 tests to determine the public health impact on 1000 people

It Takes a Laboratory!It Takes a Laboratory!

Environmental -ONE test to determine the public health impact to an

entire population!

Programs and laboratories together are engaged in the responsibility to protect public health.

Programs ultimately are responsible to make the decisions that impact it.

Laboratory accountability is necessary to protect public health and the environment against damages caused by the questions surrounding laboratory reliability and integrity that will always be a factor in making the correct decisions.

The final decision on accepting or rejecting laboratory results belongs to Environmental Programs.

Questions or Comments to:Questions or Comments to:

Henry Leibovitz, Ph.D.

RI State Health Laboratorieshenry.leibovitz@health.ri.gov

Thank you!