TAbMEP Assessment: POLARCAT CO Measurements 1. Introduction Here we provide the assessment for the carbon monoxide (CO) measurements taken from five aircraft platforms during the summer 2008 POLARCAT field campaign [INSERT REFERENCE] This assessment is based upon seven wing-tip-to-wing-tip intercomparison flights conducted during the field campaign. Recommendations provided here offer a systematic approach to unifying the POLARCAT CO data for any integrated analysis. These recommendations are based upon the instrument performance demonstrated during the POLARCAT measurement comparison exercises and are not to be extrapolated beyond this campaign. 2. POLARCAT CO Measurements Five different CO instruments were deployed on the five aircraft. Table 1 summarizes these techniques and gives references for more information. Table 1. CO measurements deployed on aircraft during POLARCAT

Aircraft Instrument Reference NASA DC-8 DACOM

NASA P-3B COBALT

NOAA WP-3D VUVF

DLR FALCON Aerolaser CO (AeroCO)

ATR-42 FALCON

3. Summary of Results Table 2 summarizes the recommendations drawn from the intercomparisons. The following sections describe the processes that led to the recommendations. Table 2 recommends a bias correction (see section 4.1 for details) that can be applied to each data set to maximize the consistency between them. Note that this bias correction should be subtracted, so a negative bias indicates that the reported CO concentrations should be increased by the absolute value of that bias correction, and a positive bias indicates that the reported concentrations should be decreased. The recommended 2σ uncertainty in Table 2 is the larger of either twice the uncertainty reported by the PI (if 1σ) or the quadrature-sum of the recommended bias correction listed in Table 2 and twice the adjusted precision determined for each instrument (see Table 4). When there are multiple intercomparisons available for the same instrument, the maximum precision value is used.

Table 2. Recommended POLARCAT CO measurement treatment

Aircraft Instrument Reported 1σ Uncertainty

Recommended Bias Correctiona

Recommended 2σ Uncertainty

NASA DC-8 DACOM 2% or 2ppbvb 0.24 - 0.003 CODC-8 2% or 2ppbv

NASA P-3B COBALT 3% 0.151 - 0.002 COP-3B 6%

NOAA WP-3D VUVF 3% 3.42 - 0.0142 COWP-3D 6%

DLR FALCON AeroCO 10% - 3.78 - 0.003 CODLR 20%

ATR-42 FALCON not reported 9.33 - 0.091 COATR {(9.33 - 0.091 CO)2 +

(0.085)2}1/2 aThe “true CO mixing ratio” = measurement – recommended bias correction (as discussed in Section 4.1). b Reported uncertainty is 2σ. 4. Results and Discussion 4.1 Bias Analysis Figures 1 – 5 illustrate the need for quantifying the bias between instruments. The difference between the simultaneous measurements reported by two instruments is plotted against the CO mixing ratio reported by one of the instruments. The apparent biases in Table 3 are calculated from orthogonal linear regression analysis (shown in the correlation plots in Figs. A1–A5) used to approximate the bias between the paired instruments’ dependence on the CO mixing ratio. Apparent bias is defined as the difference in a measurement on one aircraft platform referenced to the same measurement made on the DC-8 (i.e. DC-8 – WP-3D). For convenience, the apparent bias is given in the form a + b*CODC8. In this form, it is easier to propagate the apparent biases so the best estimate bias can be used to calculate the uncertainties summarized in Table 2. It should be noted here that the intercept should not simply be interpreted as a measurement offset; instead it is used in conjunction with the slope to best describe the linear trend found in the data. The best estimate bias is defined as the difference between the instrument being analyzed and the true CO mixing ratio as a function of the instrument being analyzed. This can be calculated by subtracting the true CO mixing ratio from the respective apparent bias equation from Table 3 and putting the result in terms of the instrument being analyzed. The average of the apparent biases for four instruments (-0.24 ppbv + 0.003 CODC8) is assumed to be the best estimate of the “true CO mixing ratio” from the DC-8 CO measurement. The ATR-42 Falcon apparent bias is not included in the average due to its magnitude. In effect, this procedure assumes that the best estimate of the true CO mixing ratio is the average of the five instruments, and the apparent bias correction is used in calculations to most closely approximate the true CO mixing ratio for each instrument. It should be noted that the initial choice of the reference instrument is arbitrary, and has no impact on the final recommendations. The given bias corrections were based upon the instrument performance demonstrated during the intercomparison periods.

Table 3. POLARCAT CO bias estimates

Aircraft Instrument Apparent Bias1 (a ppbv + b CO)

Best Estimate Bias (a ppbv + b CO)

NASA DC-8 DACOM 0 0.24 - 0.003 CODC-8 NASA P-3B COBALT -0.0886 + 0.001 CODC8 0.151 - 0.002 COP-3B

NOAA WP-3D VUVF 3.14 - 0.0110 CODC8 3.42 - 0.0142 COWP-3D DLR FALCON AeroCO -4.01 + 0 CODC8 - 3.78 - 0.003 CODLR

ATR-42 FALCON 8.33 - 0.0810 CODC8 9.33 - 0.091 COATR 1 DC-8 is taken as an arbitrary reference. Apparent bias is reported as a linear function of CO on the DC-8. 4.2 Precision Analysis The instrument precision assessment is summarized in Table 4. The Internal Estimate of Instrument Precision (IEIP) analysis procedures were applied for all continuous fast instruments. The IEIP procedure is an effective method to estimate “short-term” precision, which accounts for signal variation during a short period of assumed constant CO measurements. Because this assumption is not always valid, the IEIP estimate tends to provide an upper limit of the instrument short-term precision. Over longer time scales, however, some instruments are subject to lower precision (i.e. larger variability), which includes variability that arises from uncorrected changes in the zero level or sensitivity of the instrument. These additional contributions to the variability are not likely reflected in the IEIP derived precision, but the intercomparison flights do provide a reasonable check on their influence. This effect was examined through the comparisons of the “expected variability" and "observed variability" given in Table 4. The expected variability is the quadrature-sum of the corresponding IEIP precisions. The observed variability is the standard deviation derived from the five intercomparisons shown in Figs. 6-10, denoting the relative difference between the paired instruments. Each standard deviation is expected to be equal to the quadrature-sum of the separate IEIP precisions of the two intercompared instruments. In five cases the observed variability is larger than the expected variability, which indicates that the IEIP derived (short-term) precision needs to be adjusted to reflect the longer term fluctuations. Table 4 contains estimates of this “adjusted” precision obtained by proportionally scaling the IEIP estimates so that the expected variability values would equal to that of the observed variability. For the cases that the observed variability is smaller, the adjusted precision (last column in Table 4) is set equal to the IEIP precision. Based on the results presented in Table 4, the worst "adjusted precision" (or the largest value) is taken as a conservative precision estimate for each POLARCAT CO instrument and is used for the derivation of the recommended 2σ uncertainty in the last column of Table 2. Table 3 shows that the measurement bias is a function of CO mixing ratio. Thus, the bias may have a significant impact on the observed variability. To minimize the effect of bias, we make corrections for bias before computing the observed variability. For instance, the observed variability in the case of DLR/ATR-42 on 7/14 was estimated at 8.2% without correction. This value was reduced to 2.55% when bias correction was applied. The observed variability values given in Table 4 are computed after the bias correction. The final analysis results are shown in

Table 2. Over 90% of the data falls within the combined recommended uncertainties for each intercomparison, which is consistent with the TAbMEP guideline for unified data sets. Table 4. POLARCAT CO precision (1σ) comparisons

Flight Platform IEIP Precision

Expected Variability

Observed Variability

Adjusted Precision

4/12 DC-8 0.30% 1.14% 1.65% 0.43% WP-3D 1.10% 1.58%

4/8 DC-8 0.30% 0.39% 1.28% 0.43%a P-3B 0.25% 1.21%

4/19 DC-8 0.30% 0.46% 9.62% 0.43% a P-3B 0.35% 9.61%

7/10 DC-8 0.45% 0.67% 0.99% 0.43% a P-3B 0.50% 0.90%

7/9 DC-8 0.30% 1.09% 2.17% 0.60% DLR Falcon 1.05% 2.09%

4/15 P-3B 0.45% 1.47% 3.06% 2.62% WP-3D 1.40% 1.58%b

7/14 DLR Falcon 1.05% 4.38% 2.55% 1.05% ATR-42 Falcon 4.25% 4.25%

aDC-8 adjusted precision held at 0.43%, the value for the DC-8 and WP-3D comparison, due to problems with the IEIP analysis of the P-3B data. bWP-3D adjusted precision held at 1.58%, the value for the DC-8 and WP-3D comparison, due to problems with the IEIP analysis of the P-3B data. Appendix A Figures A1 through A5 show the time series of the CO measurements and aircraft altitudes for each intercomparison flight as well as the correlations between the two CO measurements. References [INSERT REFERENCES]

Figures

-150

-100

-50

0

50

100

150

DC

-8 -

P-3B

CO

(ppb

v)

40035030025020015010050DC-8 CO (ppbv)

04/08/2008 Average = - 3.0 ± 1.8 ppbv 04/19/2008 Average = 3.1 ± 3.9 ppbv 07/10/2008 Average = - 0.83 ± 0.68 ppbv

Overall Average = - 0.27 ± 2.1 ppbv

Figure 1: Difference between CO measurements for the three DC-8/WP-3D intercomparison flights as a function of DC-8 CO.

-20

-15

-10

-5

0

5

10

15

20

WP-

3D -

DC

-8 C

O (p

pbv)

240220200180160140120DC-8 CO (ppbv)

Average = 0.0522 ± 2.77 ppbv

Figure 2: Difference between CO measurements for the DC-8/WP-3D intercomparison flight as a function of DC-8 CO.

-15

-10

-5

0

5

10

15D

LR -

DC

-8 C

O (p

pbv)

1501401301201101009080DC-8 CO (ppbv)

Average = -0.138 ± 2.39 ppbv

Figure 3: Difference between CO measurements for the DC-8/DLR Falcon intercomparison flight as a function of DC-8 CO.

-30

-20

-10

0

10

20

30

WP-

3D -

P3-B

CO

(ppb

v)

200180160140WP-3D CO (ppbv)

Average = 4.37 ± 5.03 ppbv

Figure 4: Difference between CO measurements for the WP-3D/P-3B intercomparison flight as a function of WP-3D CO.

-30

-20

-10

0

10

20

30D

LR -

ATR

-42

CO

(ppb

v)

11511010510095908580DLR CO (ppbv)

Leg 1 Average = - 4.577 ± 2.2336 ppbv Leg 2 Average = 6.2785 ± 6.4264 ppbv

Overall Average = 0.1992 ± 7.0767 ppbv

Figure 5: Difference between CO measurements for the DLR/ATR-42 Falcon intercomparison flight as a function of DLR CO.

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

(DC

-8 -

P-3B

) / D

C-8

CO

40035030025020015010050DC-8 CO (ppbv)

04/08/2008 Average = 0.000879 ± 0.01458 04/19/2008 Average = 0.009501 ± 0.09621 07/10/2008 Average = - 0.00012 ± 0.009895

Figure 6: Relative difference between CO measurements from the DC-8/P-3B intercomparison flights as a function of DC-8 CO. Corrections were made to the 04/08/2008 and 07/10/2008 flights to account for bias in the correlation with DC-8.

-0.10

-0.08

-0.06

-0.04

-0.02

0.00

0.02

0.04

0.06

0.08

0.10(D

C-8

- W

P-3D

) / D

C-8

CO

240220200180160140120DC-8 CO (ppbv)

Average = - 0.000148 ± 0.0165

Figure 7: Relative difference between CO measurements from the DC-8/WP-3D intercomparison flight as a function of DC-8 CO. Corrections were made to the WP-3D data to account for bias in the correlation with DC-8.

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

(DC

-8 -

DLR

) / D

C-8

CO

1501401301201101009080DC-8 CO (ppbv)

Average = - 0.00152 ± 0.0217

Figure 8: Relative difference between CO measurements from the DC-8/DLR Falcon intercomparison flight as a function of DC-8 CO. Corrections were made to the DLR Falcon data to account for bias in the correlation with DC-8.

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15(W

P-3D

- P-

3B) /

WP-

3D C

O

210200190180170160150140130WP-3D CO (ppbv)

Average = - 0.00572 ± 0.03063

Figure 9: Relative difference between CO measurements from the WP-3D/P-3B intercomparison flight as a function of WP-3D CO. Corrections were made to the P-3B data to account for bias in the correlation with WP-3D.

-80x10-3

-60

-40

-20

0

20

40

60

80

(DLR

- A

TR-4

2) /

DLR

CO

11511010510095908580DLR CO (ppbv)

Leg 1 Average = - 0.005696 ± 0.02111 Leg 2 Average = - 0.0004294 ± 0.03047

Overall Average = - 0.003379 ± 0.02551

Figure 10: Relative difference between CO measurements from the DLR/ATR-42 Falcon intercomparison flight as a function of DLR CO. Corrections were made to the ATR-42 data to account for bias in the correlation with WP-3D.

Appendix 6

5

4

3

2

1

0

Altitude (km

)

13:45 14:00 14:15 14:30 14:45 15:00Time (GMT)

170

160

150

140

130

120

CO

(ppb

v)

Begin13:49:00

End14:54:40

CO 04/08/2008 P-3B DC-8

160

150

140

130

P3-B

CO

(ppb

v)

160150140130DC-8 CO (ppbv)

P-3B vs DC-8 COODR Fit, y = a + bxa = 26.5 ± 0.6b = 0.827 ± 0.004R2 = 0.94

1:1

8

6

4

2

0

Altitude (km

)

20:00 20:05 20:10 20:15 20:20 20:25 20:30 20:35 20:40 20:45Time (GMT)

400

350

300

250

200

150

100

CO

(ppb

v)

Begin20:05:00

End20:37:30CO 04/19/2008

P-3B DC-8

400

350

300

250

200

150

100

P-3B

CO

(ppb

v)

400350300250200150100DC-8 CO (ppbv)

04/19/2008 P-3B vs DC-8ODR Fit, y = a + bxa = 16.079 ± 1.8b = 0.91786 ± 0.00671R2 = 0.92

6

5

4

3

2

1

0

Altitude (km

)

18:00 18:15 18:30 18:45 19:00 19:15Time (GMT)

115

110

105

100

95

90

CO

(ppb

v)

Begin17:54:12

End18:17:26

Begin18:54:30

End19:16:00

CO 07/10/2008 P-3B DC-8

104

102

100

98

96

94

92

P-3B

CO

(ppb

v)

10410210098969492DC-8 CO (ppbv)

07/10/2008 P-3B vs DC-8ODR Fit, y = a + bxa = - 81.853 ± 5.42b = 1.858 ± 0.0561R2 = 0.40

1:1

Figure A1: (left panels) Time series of CO measurements and aircraft altitudes from two aircraft on the three intercomparison flights between NASA DC-8 and NASA P-3B. (right panels) Correlations between the CO measurements on the two aircraft.

400

350

300

250

200

150

100

50

P-3B

CO

(ppb

v)

40035030025020015010050DC-8 CO (ppbv)

Combined P-3B vs DC-8 COODR Fit, y = a + bxa = - 0.088581 ± 0.308b = 1.0009 ± 0.00179R2 = 0.98

1:1

04/08/2008 04/19/2008 07/10/2008

Figure A2: Correlations between the CO measurements on the two aircraft for all three days.

6

5

4

3

2

1

0

Altitude (km

)

22:15 22:30 22:45 23:00 23:15 23:30Time (GMT)

260

240

220

200

180

160

140

120

100

CO

(ppb

v)

Begin22:22:00

End23:28:00CO 04/12/2008

WP-3D DC-8

220

200

180

160

140

WP-

3D C

O (p

pbv)

220200180160140DC-8 CO (ppbv)

WP-3D vs DC-8 COODR Fit, y = a + bxa = 3.14 ± 0.61b = 0.989 ± 0.004R2 = 0.95

1:1

Figure A3: (left panel) Time series of CO measurements and aircraft altitudes from two aircraft on the intercomparison flights between NASA DC-8 and NOAA WP-3D. (right panel) Correlations between the CO measurements on the two aircraft.

8

6

4

2

0

Altitude (km

)

13:30 13:45 14:00 14:15 14:30Time (GMT)

150

140

130

120

110

100

90

80

CO

(ppb

v)Begin

13:30:40End

14:21:20

CO 07/09/2008 DLR DC-8

140

130

120

110

100

90

DLR

CO

(ppb

v)

14013012011010090DC-8 CO (ppbv)

DLR Falcon vs DC-8ODR Fit, y = a + bxa = -4.01 ± 0.35b = 1.00 ± 0.003R = 0.97

1:1

Figure A4: (left panel) Time series of CO measurements and aircraft altitudes from two aircraft on the intercomparison flights between NASA DC-8 and DLR Falcon. (right panel) Correlations between the CO measurements on the two aircraft.

5

4

3

2

1

0

Altitude (km

)

45.0x10344.544.043.543.042.542.0Time (GMT)

120

110

100

90

80

70

CO

(ppb

v)

Begin11:40:00

End11:54:05

Begin12:10:55

End12:25:00

07/14/2008 DLR vs ATR-42 Falcon DLR Falcon ATR-42 Falcon (30s average)

120

110

100

90

80

ATR

-42

CO

(ppb

v)

1201101009080DLR CO (ppbv)

ATR-42 Falcon vs DLR FalconODR Fit, y = a + bxa = 12.021 ± 4.75b = 0.9195 ± 0.0519R2 = 0.66

1:1 Leg 1 Leg 2

Figure A5: (left panel) Time series of CO measurements and aircraft altitudes from two aircraft on the intercomparison flights between DLR Falcon and ATR-42 Falcon. (right panel) Correlations between the CO measurements on the two aircraft.