DoD Environmental Data DoD Environmental Data Quality Workgroup UpdateQuality Workgroup Update

Presented to:Presented to:Joint Services Environmental Management Joint Services Environmental Management

ConferenceConferenceApril 2005April 2005

Edward HartzogEdward HartzogDirector, NAVSEA Laboratory Quality & Accreditation OfficeDirector, NAVSEA Laboratory Quality & Accreditation Office

Chair, DoD EDQWChair, DoD EDQWDoD Representative, IDQTFDoD Representative, IDQTF

TopicsTopics

• OverviewOverview

• Key Quality Systems Drivers Key Quality Systems Drivers

• Policy and GuidancePolicy and Guidance

• Quality Systems ImplementationQuality Systems Implementation– Joint Laboratory OversightJoint Laboratory Oversight– Perchlorate Sampling and TestingPerchlorate Sampling and Testing– Training and OutreachTraining and Outreach

Data Quality Issues Affect Data Quality Issues Affect Mission & OperationsMission & Operations

• Environmental decisions are based Environmental decisions are based on sampling and testing dataon sampling and testing data

• 95% of DoD lab testing is outsourced 95% of DoD lab testing is outsourced

• Requirements vary at federal, state, Requirements vary at federal, state, local levelslocal levels

• DoD has unique issues (e.g. DoD has unique issues (e.g. emerging contaminants, munitions, emerging contaminants, munitions, vessel discharges)vessel discharges)

DoD Inspector General Audit Report, DoD Inspector General Audit Report, Feb ‘97Feb ‘97Laboratory Support Services for Laboratory Support Services for Environmental TestingEnvironmental Testing• Audit ResultsAudit Results

– ““Organizations reviewed were not efficient and Organizations reviewed were not efficient and effective in the procurement and administration effective in the procurement and administration of environmental testing contracts”of environmental testing contracts”

• RecommendationsRecommendations– Consolidate requirements of environmental testsConsolidate requirements of environmental tests– Develop standardized quality assurance Develop standardized quality assurance

proceduresprocedures– Establish adequate procedures to administer Establish adequate procedures to administer

contractor performancecontractor performance– Include government remedy clauses in Include government remedy clauses in

environmental testing contractsenvironmental testing contracts

EPA Inspector General Report, Sept ‘98EPA Inspector General Report, Sept ‘98Superfund Quality Assurance Program (Regions 1-Superfund Quality Assurance Program (Regions 1-10)10)

• Audit ResultsAudit Results– EPA had developed many critical elements EPA had developed many critical elements

necessary for a strong and effective QA programnecessary for a strong and effective QA programBUTBUT

– Managers had not demonstrated commitment to Managers had not demonstrated commitment to fully develop and implement the programfully develop and implement the program

• RecommendationsRecommendations– Design a strategy to institutionalize QA agency-Design a strategy to institutionalize QA agency-

widewide– Improve oversightImprove oversight– Develop minimum QA requirements Develop minimum QA requirements project planning, oversight, training, tools/resources, authorityproject planning, oversight, training, tools/resources, authority

Overview - EDQWOverview - EDQW

• Established by DUSD(ES) 1996, Navy Established by DUSD(ES) 1996, Navy chairchair

• Members: Army, Navy, Air Force, DLA, Members: Army, Navy, Air Force, DLA, OSDOSD

• Tasks:Tasks:– Develop and recommend DoD policy for Develop and recommend DoD policy for

environmental data qualityenvironmental data quality– Respond to legislative/regulatory initiativesRespond to legislative/regulatory initiatives– Represent DoD on Intergovernmental work Represent DoD on Intergovernmental work

groups (e.g. IDQTF, NELAC)groups (e.g. IDQTF, NELAC)– Respond to DoD IG recommendationsRespond to DoD IG recommendations

Ensure the generation of environmental data of Ensure the generation of environmental data of known and documented qualityknown and documented quality

Overview - EDQW Overview - EDQW OrganizationOrganization

• Principal (voting) representativesPrincipal (voting) representatives– Ed Hartzog – Navy (acting chair) Ed Hartzog – Navy (acting chair) – Rosemary Gaffney – Army (USACHPPM)Rosemary Gaffney – Army (USACHPPM)– Tim Mukoda – Air Force (SAF/IEE)Tim Mukoda – Air Force (SAF/IEE)

• OSD LiaisonOSD Liaison – – Ed MillerEd Miller

• Technical representativesTechnical representatives– Laboratory Oversight, lead – Fred McLean (NAVSEA)Laboratory Oversight, lead – Fred McLean (NAVSEA)– Quality Systems Implementation, lead – Skip Darley Quality Systems Implementation, lead – Skip Darley

(NAVSEA)(NAVSEA)– Data Review and Management, lead – Dave Koran (HQ – Data Review and Management, lead – Dave Koran (HQ –

USACE)USACE)

• Contractor supportContractor support– Perot Systems Government ServicesPerot Systems Government Services– Versar, Inc.Versar, Inc.

Overview - IDQTFOverview - IDQTF

• Established by EPA 1997, Federal Facility Established by EPA 1997, Federal Facility Restoration and Reuse Office (FFRRO) Restoration and Reuse Office (FFRRO) leadlead

• Members: OSWER, EPA QA Staff, EPA Members: OSWER, EPA QA Staff, EPA Regions, DOE, and DoDRegions, DOE, and DoD

• Tasks:Tasks:– Respond to EPA IG reportsRespond to EPA IG reports– Develop written consensus agreements for Develop written consensus agreements for

environmental quality systemsenvironmental quality systems– Outline roles and responsibilities for data Outline roles and responsibilities for data

quality oversight at Federal facilitiesquality oversight at Federal facilities

Key Quality Systems DriversKey Quality Systems Drivers

• DoD Information Quality GuidelinesDoD Information Quality Guidelines, , Ensuring Quality of Information Ensuring Quality of Information Disseminated to the Public by the Disseminated to the Public by the Department of DefenseDepartment of Defense

• Proposed DoD Instruction 4715.xxProposed DoD Instruction 4715.xx, , Environmental Data Quality Environmental Data Quality AssuranceAssurance

• Draft DoD Procurement PolicyDraft DoD Procurement Policy, , Implementing Minimum QS Implementing Minimum QS Performance Standards into DoD Performance Standards into DoD Acquisitions Involving Environmental Acquisitions Involving Environmental Sampling and TestingSampling and Testing

Data Quality ChallengesData Quality ChallengesEnsuring Quality of Information Disseminated to the Ensuring Quality of Information Disseminated to the Public by the Department of Defense, February 2003 Public by the Department of Defense, February 2003

InformationInformationQualityQuality

Utility Utility Information meets Information meets

the needs of its the needs of its intended usersintended users

ObjectivityObjectivityInformation is accurate, Information is accurate,

clear, complete, unbiased, clear, complete, unbiased, and reliableand reliable

Integrity Integrity Information is Information is protected from protected from

unauthorized access unauthorized access or revisionor revision

Data must be Data must be “capable of being substantially reproduced“capable of being substantially reproduced””

Proposed DoD Instruction Proposed DoD Instruction 4715.XX4715.XXEnvironmental Data Quality Environmental Data Quality AssuranceAssurance

• Assigns responsibilities for implementing Assigns responsibilities for implementing environmental quality systemsenvironmental quality systems

• Implements DoD Data Integrity PolicyImplements DoD Data Integrity Policy

• Authorizes the publication of key Quality Authorizes the publication of key Quality Systems documentsSystems documents

• Establishes programs to monitor progress Establishes programs to monitor progress in quality systems implementationin quality systems implementation

Final Draft – December ‘04Final Draft – December ‘04

Quality Systems Quality Systems ImplementationImplementationDoD Procurement PolicyDoD Procurement Policy

• Provides minimum performance standards for Provides minimum performance standards for acquisitions involving environmental sampling acquisitions involving environmental sampling and testingand testing

• Provides sample language to use in Provides sample language to use in solicitations and contractssolicitations and contracts

• Defines minimum qualifications for contractor Defines minimum qualifications for contractor quality assurance personnelquality assurance personnel

• Specifies prohibited laboratory practicesSpecifies prohibited laboratory practices

Final Draft 17 March 2005Final Draft 17 March 2005

Quality Systems Policy & Quality Systems Policy & GuidanceGuidance

• Uniform Federal Policy for Implementing Uniform Federal Policy for Implementing Environmental Quality Systems (UFP-QS)Environmental Quality Systems (UFP-QS)

• Uniform Federal Policy for Quality Uniform Federal Policy for Quality Assurance Project Plans (UFP-QAPP)Assurance Project Plans (UFP-QAPP)

• Federal Quality Systems Roles and Federal Quality Systems Roles and Responsibilities Guidance (R&R)Responsibilities Guidance (R&R)

• DoD Quality Systems Manual for DoD Quality Systems Manual for Environmental Laboratories (DoD QSM, Environmental Laboratories (DoD QSM, V3)V3)

UFP-QSUFP-QSFinal Version 1, January 2003Final Version 1, January 2003

• Requirements and guidelines for documenting Requirements and guidelines for documenting and implementing acceptable environmental and implementing acceptable environmental quality systems (based on ANSI/ASQ E4-2004)quality systems (based on ANSI/ASQ E4-2004)

• Developed to ensure:Developed to ensure:• Environmental data are of known and documented Environmental data are of known and documented

qualityquality• Data are suitable for their intended usesData are suitable for their intended uses• Data collection and technology programs meet Data collection and technology programs meet

stated requirementsstated requirements

• Can be used to develop new quality systems Can be used to develop new quality systems or evaluate the adequacy of existing quality or evaluate the adequacy of existing quality systemssystems

UFP-QAPPUFP-QAPPInterim Final, July 2004Interim Final, July 2004

• Promotes systematic planning; Promotes systematic planning; encourages ‘graded approach’encourages ‘graded approach’

• Provides instructions and worksheets Provides instructions and worksheets for preparing project-planning for preparing project-planning documentsdocuments

• Establishes minimum QA/QC criteria for Establishes minimum QA/QC criteria for hazardous waste sites hazardous waste sites

• Streamlines development, review, and Streamlines development, review, and approval of project-planning documents approval of project-planning documents

DoD Quality Systems ManualDoD Quality Systems Manual for Environmental Laboratories for Environmental Laboratories

(QSM V3)(QSM V3)• Defines DoD requirements for Defines DoD requirements for

laboratories (based on NELAC standards, laboratories (based on NELAC standards, which are based on ISO/IEC 17025:1999)which are based on ISO/IEC 17025:1999)

• Unifies common elements of Army, Navy, Unifies common elements of Army, Navy, and Air Force QSMs and Air Force QSMs

• Provides basis for shared DoD Provides basis for shared DoD assessments and oversightassessments and oversight

• Addresses method performance criteria, Addresses method performance criteria, laboratory ethics & prohibited practiceslaboratory ethics & prohibited practices

https://www.denix.osd.mil/denix/https://www.denix.osd.mil/denix/denix.htmldenix.html

Federal Quality Systems R&R Federal Quality Systems R&R GuidanceGuidance

Final, July 2004Final, July 2004

• EPA HQ:EPA HQ: Consistent quality systems Consistent quality systems implementation throughout EPA and implementation throughout EPA and statesstates

• EPA Regional offices:EPA Regional offices: Consistent review Consistent review and approval of QAPPsand approval of QAPPs

• Federal Departments/Agencies:Federal Departments/Agencies: Agency- Agency-specific quality systems implementationspecific quality systems implementation

• IDQTFIDQTF:: Outreach and assistance Outreach and assistance

Concurrence expected March 2005Concurrence expected March 2005

Quality Systems Quality Systems ImplementationImplementation

• Laboratory OversightLaboratory Oversight

• Quality Assurance Handbook Quality Assurance Handbook for Perchlorate Investigationsfor Perchlorate Investigations

• Training and OutreachTraining and Outreach

Laboratory OversightLaboratory Oversight‘NELAP Plus’‘NELAP Plus’

• NELAP accreditation provides baseline NELAP accreditation provides baseline laboratory credentialslaboratory credentials

• DoD resources focused on assessment DoD resources focused on assessment of conformance to DoD QSM and of conformance to DoD QSM and project-specific requirementsproject-specific requirements

• Components of DoD Laboratory Components of DoD Laboratory Assessment Protocol include:Assessment Protocol include:– Assessment checklists based on DoD QSMAssessment checklists based on DoD QSM– Joint assessmentsJoint assessments– Laboratory assessment databaseLaboratory assessment database

PerchloratePerchlorateDoD DoD Interim PolicyInterim Policy (Sept 2003)(Sept 2003)

• Components must collect and compile Components must collect and compile perchlorate data for:perchlorate data for:– Drinking water suppliesDrinking water supplies subject to Unregulated subject to Unregulated

Contaminant Monitoring Rule (UCMR)Contaminant Monitoring Rule (UCMR)– Wastewater dischargesWastewater discharges permitted under National permitted under National

Pollutant Discharge Elimination System (NPDES)Pollutant Discharge Elimination System (NPDES)– Environmental restoration sitesEnvironmental restoration sites where release where release

may have occurred and complete human may have occurred and complete human exposure pathway is likely to existexposure pathway is likely to exist

– Range assessmentsRange assessments conducted under Munitions conducted under Munitions Action Plan and Defense Planning GuidanceAction Plan and Defense Planning Guidance

Perchlorate:Perchlorate:Data Quality IssuesData Quality Issues

• Most existing data based on Most existing data based on various modifications of EPA various modifications of EPA Method 314.0Method 314.0– Method developed/validated for use Method developed/validated for use

in drinking waterin drinking water– Non-specific/subject to interferencesNon-specific/subject to interferences– MRL in drinking water = 4 ppbMRL in drinking water = 4 ppb– Not suitable for use in other matricesNot suitable for use in other matrices

Perchlorate:Perchlorate:DoD Quality Assurance HandbookDoD Quality Assurance Handbook

• Guides preparation of Guides preparation of project-planning project-planning documentsdocuments

• Guides selection of Guides selection of appropriate methods appropriate methods and qualified and qualified laboratorieslaboratories

• Implements DoD Quality Implements DoD Quality Systems Policy & Systems Policy & GuidanceGuidance

DoD Quality Assurance Handbook

for PERCHLORATE Investigations

Planning, Implementation, and Assessment

Final DraftMarch 2005

DoD Quality Assurance Handbook DoD Quality Assurance Handbook Perchlorate Sampling and TestingPerchlorate Sampling and Testing

• Laboratories must comply with the DoD Laboratories must comply with the DoD QSMQSM

• Results above regulatory or action limit Results above regulatory or action limit must be confirmed using must be confirmed using perchlorate-perchlorate-specific specific technologytechnology

• Ability to achieve MRL must be Ability to achieve MRL must be demonstrated in matrix of concerndemonstrated in matrix of concern

• Selected methods must meet specified Selected methods must meet specified method performance criteria (MPCs)method performance criteria (MPCs)

DoD Quality Assurance Handbook DoD Quality Assurance Handbook Perchlorate Sampling and TestingPerchlorate Sampling and Testing

Management and Review of DataManagement and Review of Data

• Field documentation requirementsField documentation requirements

• Minimum requirements for laboratory Minimum requirements for laboratory data deliverable packagesdata deliverable packages

• Internal and third-party data review Internal and third-party data review proceduresprocedures

Quality Systems Quality Systems ImplementationImplementation

Training and OutreachTraining and OutreachISEERB-approved ISEERB-approved training:training:•Environmental Quality Environmental Quality SamplingSampling•Interagency UFP-QAPP Interagency UFP-QAPP TrainingTraining

NAVSEA 04XQ training:NAVSEA 04XQ training:A2LA Laboratory A2LA Laboratory

AssessorsAssessors

NAVSEA 04XQ NAVSEA 04XQ outreach:outreach:

www.navylabs.navy.milwww.navylabs.navy.mil

DoD Environmental Monitoring DoD Environmental Monitoring and Data Quality (EM/DQ) and Data Quality (EM/DQ)

WorkshopWorkshop Myrtle Beach, SC April 4 – 8Myrtle Beach, SC April 4 – 8

• National forum for information exchangeNational forum for information exchange– Managing environmental data quality to Managing environmental data quality to

support DoD mission and operationssupport DoD mission and operations– Evolving policy, innovative strategies, Evolving policy, innovative strategies,

technical toolstechnical tools

• TrainingTraining– DoD QSM – DoD QSM – Understanding and Assessing the Understanding and Assessing the

‘Gray Boxes’‘Gray Boxes’– DoD QSM Version 3 - DoD QSM Version 3 - OverviewOverview– Technical Implementation of SEDDTechnical Implementation of SEDD

Sound Science: Sound Science: The PayoffThe Payoff

• The right data the first timeThe right data the first time

- means the right decision the first - means the right decision the first timetime

• Reduced costs for complianceReduced costs for compliance

• Streamlined restorationStreamlined restoration

• Reduced liabilityReduced liability

• Sustained operational readinessSustained operational readiness

• Public confidencePublic confidence

HartzogEB@navsea.navy.mil (Acting EDQW Chair)www.navylabs.navy.mil

www.denix.osd.mil/denix/DOD/Working/EDQW/edqw.html

Improving Environmental Data Improving Environmental Data QualityQuality… … Because the Right Decisions Require Quality Because the Right Decisions Require Quality DataData