using in-network precision data as a basis for cross-network comparisons warren h. white, nicole p....

Using In-Network Precision Data as a Basis for Cross-Network Comparisons

Warren H. White, Nicole P. Hyslop, and Charles E. McDade

AAAR Specialty Conference: Particulate Matter, Supersites Program & Related Studies

Atlanta, GA 11 February 2005

Module A (PM2.5, Teflon Filter)

Mesa Verde National Park, COProctor Maple Research Facility, VTOlympic National Park, WASaint Marks National Wildlife Refuge, FLSac and Fox Tribe, KSTrapper Creek (Denali National Park), AK Module B (PM2.5, Nylon Filter)

Big Bend National Park, TXBlue Mounds State Park, MNFrostburg Reservoir, MDGates of the Mountains Wilderness, MTLassen Volcanic National Park, CAMammoth Cave National Park, KY

Module C (PM2.5, Quartz Filter)

Everglades National Park, FLHercules Glades Wilderness, MOHoover Wilderness, CAMedicine Lake National Wildlife Refuge, MTSaguaro National Park (Western Section), AZSeney National Wildlife Refuge, MI Module D (PM10, Teflon Filter)

Houston, TX (STN urban site)Jarbridge Wilderness, NVJoshua Tree National Park, CAQuabbin Reservoir, MASwanquarter National Wildlife Refuge, NCWind Cave National Park, SD

collocated samplers in IMPROVE networkstarting summer 2003

STN/IMPROVE Monitoring Intercomparison Sites: Oct. 2001 – Sept. 2003

Mt. Rainier NPS

Phoenix

TontoNational Monument

Haines Point, NPS

Wash. DC

Official or designated STN site, host to IMPROVE sampler

Official IMPROVE site, host to STN sampler

Anderson RAAS 401 STN Samplers

Met One SASS STN Samplers

URG MASS STN Samplers

SeattleBeacon Hill

USDA FS Dolly SodsWildersness

Operated According to Each Network’s Protocols


Mt. Rainier NPS

Phoenix



Haines Point, NPS

Wash. DC






SeattleBeacon Hill


Operated According to Each Network’s Protocolscollocated monitoring by Speciation Network

uncertainty reporting started summer 2003STN/IMPROVE Monitoring Intercomparison

Sites: Oct. 2001 – Sept. 2003

Mt. Rainier NPS

Phoenix


Haines Point, NPS

Wash. DC






SeattleBeacon Hill




Mt. Rainier NPS

Phoenix



Haines Point, NPS

Wash. DC






SeattleBeacon Hill



STN data provided by Shelly Eberly, EPA OAQPS

obligatory introductory slide

• When should both parties be happy with the results of a comparison?

• First impressions – how do the collocated and routine IMPROVE results compare?

• First impressions – how do the STN and IMPROVE results compare?

0

0.001

0.002

0.003

0 0.001 0.002 0.003

Se, ug/m3

VAL/MDL > 3

0

0.001

0.002

0.003

0 0.001 0.002 0.003

IMPROVE routine

IMP

RO

VE

col

loca

ted

0

0.001

0.002

0.003

0 0.001 0.002 0.003

PMRF

SAFO

OLYM

0

0.002

0.004

0.006

0.008

0 0.002 0.004 0.006 0.008

IMPROVE

ST

N

Se, ug/m3

0

0.002

0.004

0.006

0.008

0 0.002 0.004 0.006 0.008

MASS/URG PUSO

RASS/AndersonDOSO, WASH

SASS/MetOne PHOE, TONT

Se, ug/m3

0

0.001

0.002

0.003

0 0.001 0.002 0.003

IMPROVE routine

IMP

RO

VE

col

loca

ted

0

0.002

0.004

0.006

0.008

0 0.002 0.004 0.006 0.008

IMPROVE

ST

N

Se, ug/m3

Se, ug/m3

0

0.001

0.002

0.003

0 0.001 0.002 0.003

PMRF

SAFO

OLYM

0

0.002

0.004

0.006

0.008

0 0.002 0.004 0.006 0.008

MASS/URG PUSO



0

0.001

0.002

0.003

0 0.001 0.002 0.003

Se, ug/m3

VAL/MDL > 3

0

0.001

0.002

0.003

0 0.001 0.002 0.003

PMRF

SAFO

OLYM

0

0.002

0.004

0.006

0.008

0 0.002 0.004 0.006 0.008

MASS/URG PUSO



V, ug/m3

0

0.005

0.01

0.015

0.02

0.025

0 0.005 0.01 0.015 0.02 0.025

IMPROVE

ST

NMASS/URG PUSO



V, ug/m3

0

0.0025

0.005

0 0.0025 0.005

IMPROVE routine

IMP

RO

VE

col

loca

ted

0

0.001

0.002

0.003

0 0.001 0.002 0.003

PMRF

SAFO

OLYM

0

0.002

0.004

0.006

0.008

0 0.002 0.004 0.006 0.008

MASS/URG PUSO



V, ug/m3

0

0.005

0.01

0.015

0.02

0.025

0 0.005 0.01 0.015 0.02 0.025

IMPROVE

ST

N

V, ug/m3

0

0.0025

0.005

0 0.0025 0.005

IMPROVE routine

IMP

RO

VE

col

loca

ted

0

0.001

0.002

0.003

0 0.001 0.002 0.003

PMRF

SAFO

OLYM

0

0.002

0.004

0.006

0.008

0 0.002 0.004 0.006 0.008

MASS/URG PUSO



Fe, ug/m3

0

0.25

0.5

0.75

0 0.25 0.5 0.75 1 1.25 1.5

IMPROVE

ST

N

Fe, ug/m3

0

0.025

0.05

0.075

0.1

0 0.025 0.05 0.075 0.1

IMPROVE routine

IMP

RO

VE

col

loca

ted

0

0.001

0.002

0.003

0 0.001 0.002 0.003

PMRF

SAFO

OLYM

0

0.002

0.004

0.006

0.008

0 0.002 0.004 0.006 0.008

MASS/URG PUSO



Fe, ug/m3

0

0.025

0.05

0.075

0.1

0 0.025 0.05 0.075 0.1

IMPROVE routine

IMP

RO

VE

col

loca

ted

0

0.25

0.5

0.75

0 0.25 0.5 0.75 1 1.25 1.5

IMPROVE

ST

N

Fe, ug/m3

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.01 0.1 1 10 100

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.01 0.1 1 10 100

(col

lo-r

outin

e)/s

qrt(

2),

ug/m

3

-75%

-50%

-25%

0%

25%

50%

75%

0.01 0.1 1 10 100

(col

lo-r

out)

sqrt

(2)/

(col

lo+

rout

)

-75%

-50%

-25%

0%

25%

50%

75%

0.01 0.1 1 10 100

-75%

-50%

-25%

0%

25%

50%

75%

0.01 0.1 1 10 100

(collocated+routine)/2, ug/m3

-75%

-50%

-25%

0%

25%

50%

75%

0.01 0.1 1 10 100

-75%

-50%

-25%

0%

25%

50%

75%

0.01 0.1 1 10 100


-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.01 0.1 1 10 100

(col

lo-r

outin

e)/s

qrt(

2),

ug/m

3

-75%

-50%

-25%

0%

25%

50%

75%

0.01 0.1 1 10 100

(col

lo-r

out)

sqrt

(2)/

(col

lo+

rout

)

IMPROVE SO4=

relative error

arithmetic error

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.01 0.1 1 10 100

(col

lo-r

outin

e)/s

qrt(

2),

ug/m

3

-15

-10

-5

0

5

10

15

0.01 0.1 1 10 100

obse

rved

diff

. / e

xpec

ted

diff.

IMPROVE SO4=

-15

-10

-5

0

5

10

15

0.01 0.1 1 10 100


-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

2.5% 50.0% 97.5%

ob

serv

ed

/

exp

ect

ed

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.01 0.1 1 10 100-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.01 0.1 1 10 100

(collo

-routin

e)/

sqrt

(2),

ug/m

3

observed expected

-5

-4

-3

-2

-1

0

1

2

3

4

5

0% 25% 50% 75% 100%cumulative percentage of observations

ob

serv

ed

/ e

xpec

ted

-15

-10

-5

0

5

10

15

0.01 0.1 1 10 100

-15

-10

-5

0

5

10

15

0.01 0.1 1 10 100

obse

rved

diff

. / e

xpec

ted

diff.

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

2.5% 50.0% 97.5%

ob

serv

ed

/

exp

ect

ed

cumulative distribution of observed / expected

-5

-4

-3

-2

-1

0

1

2

3

4

5


IMPROVE SO4=

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

2.5% 50.0% 97.5%cumulative percentage of observations

ob

serv

ed

/

exp

ect

ed

-5

-4

-3

-2

-1

0

1

2

3

4

5

0% 25% 50% 75% 100%

obse

rved

/

expe

cted

-5

-4

-3

-2

-1

0

1

2

3

4

5


unit norm

al distrib

ution*

* corresponding to normal error, unbiased with known precision

IMPROVE SO4=

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0


ob

serv

ed

/

exp

ect

ed

0

7.5

15

22.5

0 7.5 15 22.5

ST

N

0

7.5

15

22.5

0 7.5 15 22.5

RASS/Anderson DOSO, WASH


MASS/URG PUSO

0

7.5

15

22.5

0 7.5 15 22.5

ST

NRASS/Anderson DOSO, WASH


MASS/URG PUSO

0

7.5

15

22.5

0 7.5 15 22.5

IMPROVE

0

7.5

15

22.5

0 7.5 15 22.5routine IMPROVE

collo

cate

d IM

PR

OV

E

MACA

LAVO

BIBE

GAMO

0

7.5

15

22.5

0 7.5 15 22.5

MACA

LAVO

BIBE

GAMO

0

7.5

15

22.5

0 7.5 15 22.5

collo

cate

d IM

PR

OV

E

0

7.5

15

22.5

0 7.5 15 22.5routine IMPROVE

IMPROVE-IMPROVE STN-IMPROVE

SO4=

-6.0

-5.0

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0


obse

rved

/

expe

cted

STN

- IMPROVE

-5.0

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

5.0

2.5% 50.0% 97.5%

Se

V

Fe

Are observed STN-IMPROVE differences accounted for by the two networks’ reported uncertainties?

Yes!

no!

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0


ob

serv

ed

/

exp

ect

ed

PM2.5

PM10

IMPROVE:collocated - routine

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

2.5% 50.0% 97.5%

PM2.5

PM10

How about collocated-routine differences within IMPROVE?

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

2.5% 50.0% 97.5%


-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0


obse

rved

/

expe

cted

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

2.5% 50.0% 97.5%


-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

2.5% 50.0% 97.5%

EC

OC

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0


obse

rved

/

expe

cted

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

2.5% 50.0% 97.5%


-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

2.5% 50.0% 97.5%

SO4

NO3

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0


ob

serv

ed

diff

ere

nce

/ e

xpe

cte

d d

iffe

ren

ce V, 351/372 detects (94%)

Ni, 299/372 detects (80%)

As, 243/372 detects (65%)

Se, 339/372 detects (91%)

normal error,zero bias,accurateuncertainty

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

2.5% 50.0% 97.5%

ob

serv

ed

/

exp

ect

ed

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

2.5% 50.0% 97.5%

V, 351/372 detects (94%)

Ni, 299/372 detects (80%)

As, 243/372 detects (65%)

Se, 339/372 detects (91%)


-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

2.5% 50.0% 97.5%


-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0


ob

serv

ed

diff

ere

nce

/ e

xpe

cte

d d

iffe

ren

ceVAL/MDL > 3

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

2.5% 50.0% 97.5%

ob

serv

ed

/

exp

ect

ed

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

2.5% 50.0% 97.5%

V, 330/372 detects (89%)

Ni, 200/372 detects (54%)

As, 53/372 detects (14%)

Se, 241/372 detects (65%)


-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

2.5% 50.0% 97.5%


-5.0

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

5.0


ob

serv

ed

diff

ere

nce

/ e

xpe

cte

d d

iffe

ren

ce

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

2.5% 50.0% 97.5%

ob

serv

ed

/

exp

ect

ed

-5.0

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

5.0

2.5% 50.0% 97.5%

Ca, 372/372 detects (100%)

K, 372/372 detects (100%)

Ti, 356/372 detects (96%)

Fe, 372/372 detects (100%)


-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

2.5% 50.0% 97.5%


-5.0

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

5.0


ob

serv

ed

diff

ere

nce

/ e

xpe

cte

d d

iffe

ren

ce

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

2.5% 50.0% 97.5%

ob

serv

ed

/

exp

ect

ed

-5.0

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

5.0

2.5% 50.0% 97.5%

Al, 154/372 detects (41%)

Si, 338/372 detects (91%)

Mn, 362/372 detects (97%)

Fe, 372/372 detects (100%)


-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

2.5% 50.0% 97.5%


-5.0

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

5.0


ob

serv

ed

diff

ere

nce

/ e

xpe

cte

d d

iffe

ren

ce Al, 132/372 detects (35%)

Si, 338/372 detects (91%)

Mn, 343/372 detects (92%)

Fe, 372/372 detects (100%)


VAL/MDL > 3

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

2.5% 50.0% 97.5%

ob

serv

ed

/

exp

ect

ed

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

2.5% 50.0% 97.5%


-5.0

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

5.0

2.5% 50.0% 97.5%

Al, 132/372 detects (35%)

Si, 338/372 detects (91%)

Mn, 343/372 detects (92%)

Fe, 372/372 detects (100%)


-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0


ob

serv

ed

diff

ere

nce

/ e

xpe

cte

d d

iffe

ren

ce

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

2.5% 50.0% 97.5%

ob

serv

ed

/

exp

ect

ed

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

2.5% 50.0% 97.5%


-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

2.5% 50.0% 97.5%

H, 372/372 detects (100%)

S, 372/372 detects (100%)

Br, 372/372 detects (100%)

Pb, 368/372 detects (99%)


What I just said

• When should both parties be happy with the results of a comparison? When observed differences are consistent with reported uncertainties.

• First impressions – how do the collocated and routine IMPROVE results compare? We are generally ‘happy’, except with the ‘crustal’ elements and S.

• First impressions – how do the STN and IMPROVE results compare? See C.E. McDade et al., this room, just 1 hour from now.

using in-network precision data as a basis for cross-network comparisons warren h. white, nicole p....

Documents

expected slide

epa oaqps slide

obligatory introductory

vt olympic national

ca mammoth cave national

nc wind cave national

nv joshua tree national

stn data