guest speaker: jeremy howe little river band of ottawa indians the measuring sticks of air...
TRANSCRIPT
Audits Overview: NPAP, PEP, Annual PE, Flow Rate Audits
Guest Speaker:
Jeremy HoweLittle River Band Of Ottawa Indians
The Measuring Sticks of Air Monitoring
2
April 7: QA Systems, EPA definitions, PQAOs and common sense – Mike Papp
April 14: Routine Quality Control and Data Management (1-pt QC, flow rate, and instrument stability checks) – Travis Maki
April 21: Audits Overview (NPAP, PEP, Annual PE, Flow Rate Audits) – Jeremy Howe
April 28: Calculating Bias and Precision and AQS reports – Bill Frietsche
May 5: 40 CFR 58 App. A- Gaseous Pollutants – Glenn Gehring
May 12: 40 CFR 58 App. A- Ozone – Brenda Jarrell May 19: 40 CFR 58 App. A- PM filter and continuous
methods –Brandy Toft
Overall Course Overview:
3
Audit – “a systematic and independent examination to determine whether quality activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives”
Independent means using a device other than the one you use routinely at a minimum, and in some cases a completely independent system of instrument and operator
Audit definition:
4
Basically compare◦What is actually being done in the field and the office
◦Against what is stated in the QAPP and SOPs
◦Are done by you or someone outside your office, depending on the type of audit
All Audits (in some cases called “assessments”):
5
Number and types of assessments People or organizations doing the assessments
Schedule Criteria for assessments (we will
compare what we do against the SOP or QAPP)
Reporting and responsibility for follow-up (e.g., we will revise the SOP in pen for now, and retype it with changes within a month)
Describe in your QAPP
6
Program: CLOSE THE LOOP (fix problems and take action to make sure
they do not happen again)
And make sure it is documented
Responsibility for Follow-Up and Verification of Corrective Action
7
1. Six-month flow rate audits you make with non-routine standards (PM)
2. Annual Performance Evaluations (gas) 3. NPAP and PEP audits conducted by
certified auditors 4. Technical Systems Audits conducted by
EPA regions (every 3 years) To see how many/often of the 1st three types
you need for NAAQS-comparable data, see Calculations for Tribes Interested in Joining a Tribal PQAO - 2011-02-08A.xls
4 categories of audits:
QC Checks (and audits):Method Pollutants Frequency MQO
One-Point QC SO2, NO2, O3, CO Every 2 Weeks
O3 :
Precision = 7% Bias = 7%
SO2, NO2, CO :
Precision = 10%Bias = 10%
Annual Performance Evaluation (Audit)
SO2, NO2, O3, CO Once per Year <= 15% for each audit concentration
Flow Rate Verification (QC check)
PM10 (lo-Vol), PM2.5
Once every 4 weeks
<= 4% of Standard & 5% of Design Value
Flow Rate Verification (QC check)
PM10 (high-Vol), TSP
Once per quarter
<= 7% of Standard & 10% of Design Value
Semi-Annual Flow Rate Audit
PM10, PM2.5 Every 6 Months
<= 4% of Standard & 5% of Design Value
Semi-Annual Flow Rate Audit
PM10 (high-Vol), TSP
Every 6 Months
<= 10% of Standard & 10% of Design Value
Collocated Sampling PM10, TSP, PM2.5 15% of Network Every 12 Days
PM2.5, TSP, PM10 :
Precision = 10%
PM2.5 PEP Program
NPAP
PM2.5
SO2, NO2, O3, CO
Quarter *!Year (see Calculations…spreadsheet)
Bias = 10%
8
Who can do these, and with what?Method Pollutants Frequency Who? With What?
One-Point QC SO2, NO2, O3, CO
Every 2 Weeks Site operator Annual calibration standard, owned by tribe
Annual Performance Evaluation
SO2, NO2, O3, CO
Once per Year Preferably not site operator, no national certification required, not the same standard that is used for routine QC checks
Flow Rate Verification (not required to be reported to AQS)
PM10, PM2.5 Every 4 weeks Can be site operator, no national certification required, not the same standards that are used to calibrate your analyzer
Semi-Annual Flow Rate Audit
PM10, PM2.5 Every 6 Months Preferably not site operator, no national certification required, not the same standards that are used for monthly QC checks
PM PEP, NPAP PM2.5,
All gaseous
See Calculations…spreadsheet
OAQPS/TAMS certified auditor
OAQPS/TAMS equipment
9
10
First type: PM flow rate audits (counts as an audit, for bias, because it
is not the stnd used for precision FR checks):
Flow Rate Verification
PM10, PM2.5
automated and
manual
Every 4 weeks Can be site operator, no national certification required, the flow rate standard used must be certified annually
Flow Rate Verification
PM10 (high-Vol), TSP
manual
Once per quarter
Semi-Annual Flow Rate Audit
PM10, PM2.5
automated and
manual
Every 6 Months Preferably not site operator, no national certification required, not the same standards that are used to calibrate the analyzer, but can be calibrated to the same primary stnd as your cal stnd
11
A PM2.5 example, using the spreadsheet:
12
This yields, in the QC Checks tab:
13
And in the Audits tab:
14
Discussed in more detail in PM webinar Rely on owner’s manuals EPA guidance available: 2.12 "Monitoring PM2.5 in Ambient Air Using
Designated Reference or Class I Equivalent Methods" 1998 http://www.epa.gov/ttn/amtic/files/ambient/pm25/qa/m212covd.pdf
See docs in course webpage and AMTIC and CARB SOPs
1st type, cont.: PM Flow Rate Audits:
15
check the operational flow rate of the analyzer, before any flow rate adjustment
Randomize the time and day Use a flow rate transfer standard certified to
be traceable to an authoritative volume or other applicable standard (within 2% of NIST –Traceable Standard Part 50, App L Sec 9.2)
Reported to AQS as RA (bias) because it is made with a non-routine standard (the ones with your routine stnd are precision (RP)
% diff =[ (meas-audit)/audit ]*100
1st type, cont.: Semi-Annual Flow Rate Audits (PM):
16
3.2.2 Annual performance evaluation for SO2, NO2, O3, or CO
…The gas standards and equipment used for evaluations must not be the same as the standards and equipment used for calibration or calibration span adjustments…the auditor should not be the operator or analyst who conducts the routine monitoring, calibration, and analysis.
2nd type of audits: Annual Performance Evaluations (gas)
17
40 CFR 58 App A sec 3.2.2: If there are fewer than four analyzers for a
pollutant within a PQAO, P.E. annually (preferably in different quarters)
The evaluation should be conducted by a trained experienced technician other than the routine site operator
Current CFR guidance will change soon
2nd type of audits: Annual Performance Evaluations, gas, cont.:
18
A gas analyzers example, 3 at 1 site:
19
This yields, in the Checks tab:
20
And in the Audits tab, this yields 1 NPAP and 3 Annual PEs:
21
use…audit gas from at least three consecutive audit levels. The audit levels selected should represent or bracket 80 percent of ambient concentrations measured by the analyzer being evaluated:
2nd type of audits: Annual Performance Evaluations, gas, cont.:
22
New guidance for audit levels: Ten levels, the lowest as low as 4-6 ppb for
O3 Consecutive audit levels no longer required See memo: http://itep68.itep.nau.edu/itep_downloads/Q
A101_Resources/EPA%20guidance%20and%20other%20reference%20material/TechMemoforPEAuditLevels.pdf
2nd type of audits: Annual Performance Evaluations, gas, cont.:
23
3.2.2…Report both the audit and measured concentration
The % differences (di ) between these concentrations are used to assess the quality of the monitoring data, and to confirm your biweekly QC check results:
2nd type of audits: Annual Performance Evaluations, gas, cont.:
24
Allow…the audit gas to pass through all filters, scrubbers, conditioners, and other sample inlet components used during normal ambient sampling and as much of the ambient air inlet system as is practicable
< 15 % for each audit concentration, and EPA uses these (RA) values you report to
AQS to generate probability limits for your bias
2nd type of audits: Annual Performance Evaluations, gas, cont.:
25
2.4 National Performance Evaluation Programs. ….the consent of the monitoring organization for EPA to apply an appropriate portion of the grant funds, which EPA would otherwise award to the monitoring organization for monitoring activities, will be deemed by EPA to meet this requirement. For clarification and to participate, monitoring organizations should contact either the appropriate EPA Regional Quality Assurance (QA) Coordinator…
3rd type of audits: NPAP and PEP audits conducted by
certified auditors
26
PEP- $2000/audit◦ 5 audits/yr for PQAO with < 5 sites = 10K/year◦ All sites audited at least once every 6 years
NPAP- $2200/audit◦ Each site audited at least once every 5 yrs◦ At least 1 NPAP audit per PQAO network that
includes even 1 gas analyzer◦ Would need 10 sites for 2 audits a year◦ Each NPAP audit covers all gas analyzers at a site
These (rough) costs are for EPA contractors doing the audits
3rd type, cont.: NPAP and PEP audits conducted by certified auditors
(cont.)
27
Auditors (including tribal folks) can take training and exam
Equipment must be approved/provided as audit-specific equipment
SOPs available on AMTIC, http://www.epa.gov/ttn/amtic/npepqa.html
3rd type, cont.: PEP and NPAP audits
28
NPAP Audits (3rd type, for gas)• Separate from Cradle to Grave• For Gaseous Pollutants
• O3 + CO + SO2 + NOX
29
PEP Audits (3rd type, for PM2.5)
• Separate from Cradle to Grave• For Filter Based Pollutants
• PM 2.5 + Lead
PEP
CollocationPEP Audit
30
2.5 Technical Systems Audit Program. Technical systems audits of each ambient air monitoring organization shall be conducted at least every 3 years by your EPA regional office
Review of siting, documentation, procedures’ implementation
Assessment of whether the “QA function is independent”
Checklists that the region will use are available as QA Handbook Volume II, Appendix H
4th type of Audit: Technical Systems Audits conducted by EPA
regions (every 3 years)
31
At least 3 months notice, with written audit plan
Checklist sent ahead of time, see NAAM Technical Systems Audit Form in: http://itep68.itep.nau.edu/itep_downloads/QA101_Resources/EPA%20guidance%20and%20other%20reference%20material/
4th type, cont.: EPA Technical Systems Audits
32
All audits are “friendly” and intended to improve the quality of the data
Findings will always include insufficient documentation
Site visits are included Some AQS data are pulled, then records of
all the QC and original data are looked for in office
Important for small organizations: the “independency of the QA function”
How are audit results used?
33
CFR App. A: 2.2 …a quality assurance management function…includes strategic planning, allocation of resources and …assessing and reporting…
must have sufficient technical expertise and management authority to conduct independent oversight…
should be organizationally independent of environmental data generation activities (e.g., NOT the site operator or their manager)
Ideal QA manager is described at: http://www.epa.gov/ttn/amtic/files/ambient/monitorstrat/qamroles2.pdf
“Independent” QA Function:
34
Adds necessary quantitative and qualitative (TSAs) yardsticks for data defensibility
Adds value by exchanging information Course website:
http://itep68.itep.nau.edu/itep_downloads/QA101_Resources/
Consider joining the TAMS Steering Committee to help develop national tribal training to address these complicated issues (email us)◦ Jeremy Howe: [email protected]◦[email protected]◦[email protected]
Audits Conclusions: