more quality control for weather station networks...12.2.2007 vesa hasu: more quality control for...

Vesa Hasu

More Quality Control for Weather Station Networks

12.2.2007

2Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory

Contents

IntroductionThe quality control motivationSome basic vocabularyQuality control overviewQuality control detailsFurther quality control problemsFrom measurement quality control to network quality control

12.2.2007


Introduction

The quality control research made in Dada‐project (1.6.2005‐31.1.2007)– Dada = Development of Data Fusion and Diagnostics Methods in Weather Station Networks (http://control.hut.fi/research/dada/)

– Dada includes also data fusion of multiple source weather measurements

– The sequel: Pipo (Quality and fusion of surface weather stations and dual‐polarization radar measurements)

Collaboration between Helsinki University of Technology (TKK), Finnish Meteorological Institute (FMI) and VaisalaDada is funded by Tekes and Vaisala

12.2.2007


Quality Control Motivation

Why quality control in weather station networks is just right now important?– A new generation of weather stations: cheaper and more accurate results →more dense measurements in spatial and temporal directions

– New types of forecast products – all measurements used in the forecasting should be reliable with rapid quality control

→ Number of measurements grows by decades → Automated procedures needed for quick QC decisions

12.2.2007



Consider meso/synoptic scale measurement network difference:– If the average station spacing is decreased by factor 4‐5, the number of measurement stations increases by factor 20

– If the average sampling period decreases by a factor 12, from 1 hour to 5 minutes

→ Number of measurements can easily grow approximately 200 times larger than before

12.2.2007



In this presentation: quality control = fault detection (of meteorological measurements)From control engineering point of view, the fault diagnosis of meteorological measurement can be tedious– Unknown process– Time varying system (e.g. seasonal changes)– Geography dependent measurements (e.g. inland/sea)The first conclusions: – Adaptation needed – Traditional fault detection methods do not apply

12.2.2007


Some Basic Vocabulary

A bit of vocabulary: some basic error types

The following fault detection method can detect the noise and spike errors

Spike Bias

Noise Drift

12.2.2007


Noise Quality Control Overview

The noise fault detection in general level is – Estimate a residual– Compare residual to the alarm thresholds– Update the residual estimation model and alarm threshold

A more accurate block diagram:

12.2.2007


Noise Quality Control Overview

Advantages of the solution: – Adaptation to the seasonal changes– Applicable to different conditions due to the adaptation– The diurnal changes in the measurement statistics fairly easy to cope with

– No need for a background field (from e.g. LAPS)– Applicable for a single station or a multiple station environment

– Possibility to estimate a few missing measurements

12.2.2007


Preprocessing

The preprocessing consists of the flagging of the most obvious errors The preprocessing flagging is made based on step and consistency checksThe preprocessing is essential, since large errors may bias the residual estimation of the actual fault detection

12.2.2007


Modeling and Filtering

Self‐tuning modeling of the measurement behavior– Autoregressive time series model i.e. the new measurement is assumed to depend on the previous measurements

– The model is updated by the recursive least squares method

Filtering done by a Kalman filter– Kalman filter offers optimal linear estimation– Covariance matrices required by Kalman filter are also estimated by recursive methods

12.2.2007


Residual Computation

The idea is to filter the measurement and use the filtering residual to the fault detectionThe model applied in Kalman filter is modeled with a self‐tuning recursive least squares estimation– The advantage: easy to implement for different measurements and robustness against the seasonal changes

12.2.2007


Measurement Flagging

A measurement is flagged as erroneous, if the corresponding residual is larger than three (or four) times the standard deviation of the residual– 3σ‐ and 4σ‐thresholds comes from the assumption of normally distributed residual

– According to the theory, the 3σ‐threshold corresponds to passing more than 99.5 % of measurements

– Since the residual does not follow exactly the normal distribution, the 4σ‐threshold should be used in the practice

12.2.2007


Filtering Example

An example of temperature filtering:

0 50 100 150 200 250

10

15

20

T (o C

)

Suvisaaristo with added noise

0 50 100 150 200 250-5

0

5

Iteration (5 min)

Diff

eren

ce (o C

)

Thick black = measured temperature

Thin black = temperature with added noise

Red = estimated temperature

Black = the added noise

Red = error of the estimated temperature

12.2.2007


Another Filtering Example

The difficulty of weather measurement filtering:

Black = measured barometric pressure

Purple = filtered barometric pressure

7100 7120 7140 7160 7180 7200 7220 7240 7260 7280

1000

1001

1002

1003

1004

1005

1006

1007

Svartviken

Iteration (5 min)

Bar

omet

ric p

ress

ure

(hP

a)

7100 7120 7140 7160 7180 7200 7220 7240 7260 7280

1000

1001

1002

1003

1004

1005

1006

1007

Svartviken

Iteration (5 min)

Bar

omet

ric p

ress

ure

(hP

a)

β = 0.9 β = 0.99

12.2.2007


Residual Examples

Suvisaaristo barometric pressure and its residual

12.7.2006 14.7. 16.7. 18.7.1000

1010

1020

1030Suvisaaristo A Barometric pressure

p (h

Pa)

12.7.2006 14.7. 16.7. 18.7.

-0.4

-0.2

0

0.2

0.4

0.6

Res

idua

l (hP

a)

12.2.2007


Residual Examples

Suvisaaristo temperature and its residual

12.7.2006 14.7. 16.7. 18.7.5

10

15

20

25

30Suvisaaristo A Temperature

T (o C

)

12.7.2006 14.7. 16.7. 18.7.

-2

-1

0

1

2

Res

idua

l (o C

)

12.2.2007


Time‐Dependent Alarm Thresholds

Since the measurement are very much environment dependent, the alarm thresholds must be adaptiveThe variation of some meteorological measurements may have diurnal dependencyThe taken approach: estimate the residual variation based on the time of day

12.2.2007


Alarm Thresholds: A Practical Recursive Version

The recursive update of residual variance, and also the alarm threshold, looks like

– Good for e.g. barometric pressure, which does not experience diurnal variance change

The update rule for a measurement with diurnal variance change looks like

– Good for e.g. temperature and relative humidity, which variance depends strongly on the lighting conditions

– Additional temporal smoothing must be done

( )( )22 2 ˆˆ ˆ( ) ( ) 1 ( ) ( )k k x k x kσ λσ λ= + − −

( )( )22 2 ˆˆ ˆ( ) ( 288) 1 ( ) ( )k k x k x kσ λσ λ= − + − −

12.2.2007


Residual Example

An example of relative humidity measurement

6 12 18 0 6 12

75

80

85

90

95

Rel

ativ

e H

umid

ity [%

]

6 12 18 0 6 12-2

-1

0

1

2

Res

idua

l [%

]

15.11.2005 16.11.2005

15.11.2005 16.11.2005

Black = measurement

Red = filtered measurement

Black = 3σ‐alarm threshold

Red = residual

12.2.2007


Residual Example

An example on the temperature measurement with erroneous behavior

0 6 12 18 0

17

18

19

20

21

22

Tem

pera

ture

[o C]

0 6 12 18 0-2

-1

0

1

2

Res

idua

l [o C

]

15.7.2006

15.7.2006

12.2.2007


Residual Example: Two Stations

0 1000 2000 3000 4000 5000 6000 7000 8000

10

15

20

25

T (o C

)

Juhanila A

0 1000 2000 3000 4000 5000 6000 7000 8000-1

-0.5

0

0.5

1

Iteraatio

Enn

uste

virh

e (o C

)

Juhanila stations August 2005

A‐station: 2 m –level

B‐station: 100 m ‐level

Juhanila A‐station

Tempe

rature

Residu

al

0 1000 2000 3000 4000 5000 6000 7000 8000 9000

10

15

20

25

30

T (o C

)

Juhanila B

0 1000 2000 3000 4000 5000 6000 7000 8000-10

-5

0

5

10

Iteraatio

Enn

uste

virh

e (o C

)

Juhanila B‐station

Residu

alTe

mpe

rature

12.2.2007


Further Quality Control Problems

As mentioned earlier, the current fault detection works well in detecting spikes and noise– Fault/weather phenomena –separation is always an issueDrift detection is much more tedious task – Detection seems possible using either neighboring stations or nowcasted values

– An exception: barometric pressureDuring heavy weather phenomena the measurements have special behavior that is not included in fault detection yet (e.g. wind, pressure and temperature in convection)

Fault detection of non‐continuous measurements: rain→ Room for further QC work

12.2.2007


From Measurement Quality Control to Measurement Network Performance

For maintenance and forecasting purposes, knowing the measurement station performance is also essential– Additional information for the end user about the measurement quality

– Maintenance operation planningIn order to gain information about the network condition, descriptive performance indices about each station can be used– For example: availability, accuracy, reliability, estimability, influence

12.2.2007


Descriptions of Measurement Network Performance

Short descriptions of performance indices :– Availability = ”are there missing measurements”– Accuracy = ”is the measurement accurate or not”– Reliability = ”can the measurement station be trusted”– Estimability = ”can the network compensate the measurement”

– Influence = ”how the measurement is influencing the network performance”

For maintenance

For data users

12.2.2007


A Performance Index Example

Example temperature measurements:

Measurement OK Low accuracy Low availability

12.2.2007


Conclusions

Automatic quality control is a necessity when increasing the mesoscale measurement network sizeTraditional quality control is the starting point, the newer type algorithms can improve the resultsUsing background field is not always a feasible assumptionQuality control gives an opportunity to form new metadata to maintenance decisions

more quality control for weather station networks...12.2.2007 vesa hasu: more quality control for...

Documents