Vesa Hasu
More Quality Control for Weather Station Networks
12.2.2007
2Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Contents
IntroductionThe quality control motivationSome basic vocabularyQuality control overviewQuality control detailsFurther quality control problemsFrom measurement quality control to network quality control
12.2.2007
3Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Introduction
The quality control research made in Dada‐project (1.6.2005‐31.1.2007)– Dada = Development of Data Fusion and Diagnostics Methods in Weather Station Networks (http://control.hut.fi/research/dada/)
– Dada includes also data fusion of multiple source weather measurements
– The sequel: Pipo (Quality and fusion of surface weather stations and dual‐polarization radar measurements)
Collaboration between Helsinki University of Technology (TKK), Finnish Meteorological Institute (FMI) and VaisalaDada is funded by Tekes and Vaisala
12.2.2007
4Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Quality Control Motivation
Why quality control in weather station networks is just right now important?– A new generation of weather stations: cheaper and more accurate results →more dense measurements in spatial and temporal directions
– New types of forecast products – all measurements used in the forecasting should be reliable with rapid quality control
→ Number of measurements grows by decades → Automated procedures needed for quick QC decisions
12.2.2007
5Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Quality Control Motivation
Consider meso/synoptic scale measurement network difference:– If the average station spacing is decreased by factor 4‐5, the number of measurement stations increases by factor 20
– If the average sampling period decreases by a factor 12, from 1 hour to 5 minutes
→ Number of measurements can easily grow approximately 200 times larger than before
12.2.2007
6Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Quality Control Motivation
In this presentation: quality control = fault detection (of meteorological measurements)From control engineering point of view, the fault diagnosis of meteorological measurement can be tedious– Unknown process– Time varying system (e.g. seasonal changes)– Geography dependent measurements (e.g. inland/sea)The first conclusions: – Adaptation needed – Traditional fault detection methods do not apply
12.2.2007
7Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Some Basic Vocabulary
A bit of vocabulary: some basic error types
The following fault detection method can detect the noise and spike errors
Spike Bias
Noise Drift
12.2.2007
8Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Noise Quality Control Overview
The noise fault detection in general level is – Estimate a residual– Compare residual to the alarm thresholds– Update the residual estimation model and alarm threshold
A more accurate block diagram:
12.2.2007
9Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Noise Quality Control Overview
Advantages of the solution: – Adaptation to the seasonal changes– Applicable to different conditions due to the adaptation– The diurnal changes in the measurement statistics fairly easy to cope with
– No need for a background field (from e.g. LAPS)– Applicable for a single station or a multiple station environment
– Possibility to estimate a few missing measurements
12.2.2007
10Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Preprocessing
The preprocessing consists of the flagging of the most obvious errors The preprocessing flagging is made based on step and consistency checksThe preprocessing is essential, since large errors may bias the residual estimation of the actual fault detection
12.2.2007
11Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Modeling and Filtering
Self‐tuning modeling of the measurement behavior– Autoregressive time series model i.e. the new measurement is assumed to depend on the previous measurements
– The model is updated by the recursive least squares method
Filtering done by a Kalman filter– Kalman filter offers optimal linear estimation– Covariance matrices required by Kalman filter are also estimated by recursive methods
12.2.2007
12Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Residual Computation
The idea is to filter the measurement and use the filtering residual to the fault detectionThe model applied in Kalman filter is modeled with a self‐tuning recursive least squares estimation– The advantage: easy to implement for different measurements and robustness against the seasonal changes
12.2.2007
13Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Measurement Flagging
A measurement is flagged as erroneous, if the corresponding residual is larger than three (or four) times the standard deviation of the residual– 3σ‐ and 4σ‐thresholds comes from the assumption of normally distributed residual
– According to the theory, the 3σ‐threshold corresponds to passing more than 99.5 % of measurements
– Since the residual does not follow exactly the normal distribution, the 4σ‐threshold should be used in the practice
12.2.2007
14Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Filtering Example
An example of temperature filtering:
0 50 100 150 200 250
10
15
20
T (o C
)
Suvisaaristo with added noise
0 50 100 150 200 250-5
0
5
Iteration (5 min)
Diff
eren
ce (o C
)
Thick black = measured temperature
Thin black = temperature with added noise
Red = estimated temperature
Black = the added noise
Red = error of the estimated temperature
12.2.2007
15Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Another Filtering Example
The difficulty of weather measurement filtering:
Black = measured barometric pressure
Purple = filtered barometric pressure
7100 7120 7140 7160 7180 7200 7220 7240 7260 7280
1000
1001
1002
1003
1004
1005
1006
1007
Svartviken
Iteration (5 min)
Bar
omet
ric p
ress
ure
(hP
a)
7100 7120 7140 7160 7180 7200 7220 7240 7260 7280
1000
1001
1002
1003
1004
1005
1006
1007
Svartviken
Iteration (5 min)
Bar
omet
ric p
ress
ure
(hP
a)
β = 0.9 β = 0.99
12.2.2007
16Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Residual Examples
Suvisaaristo barometric pressure and its residual
12.7.2006 14.7. 16.7. 18.7.1000
1010
1020
1030Suvisaaristo A Barometric pressure
p (h
Pa)
12.7.2006 14.7. 16.7. 18.7.
-0.4
-0.2
0
0.2
0.4
0.6
Res
idua
l (hP
a)
12.2.2007
17Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Residual Examples
Suvisaaristo temperature and its residual
12.7.2006 14.7. 16.7. 18.7.5
10
15
20
25
30Suvisaaristo A Temperature
T (o C
)
12.7.2006 14.7. 16.7. 18.7.
-2
-1
0
1
2
Res
idua
l (o C
)
12.2.2007
18Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Time‐Dependent Alarm Thresholds
Since the measurement are very much environment dependent, the alarm thresholds must be adaptiveThe variation of some meteorological measurements may have diurnal dependencyThe taken approach: estimate the residual variation based on the time of day
12.2.2007
19Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Alarm Thresholds: A Practical Recursive Version
The recursive update of residual variance, and also the alarm threshold, looks like
– Good for e.g. barometric pressure, which does not experience diurnal variance change
The update rule for a measurement with diurnal variance change looks like
– Good for e.g. temperature and relative humidity, which variance depends strongly on the lighting conditions
– Additional temporal smoothing must be done
( )( )22 2 ˆˆ ˆ( ) ( ) 1 ( ) ( )k k x k x kσ λσ λ= + − −
( )( )22 2 ˆˆ ˆ( ) ( 288) 1 ( ) ( )k k x k x kσ λσ λ= − + − −
12.2.2007
20Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Residual Example
An example of relative humidity measurement
6 12 18 0 6 12
75
80
85
90
95
Rel
ativ
e H
umid
ity [%
]
6 12 18 0 6 12-2
-1
0
1
2
Res
idua
l [%
]
15.11.2005 16.11.2005
15.11.2005 16.11.2005
Black = measurement
Red = filtered measurement
Black = 3σ‐alarm threshold
Red = residual
12.2.2007
21Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Residual Example
An example on the temperature measurement with erroneous behavior
0 6 12 18 0
17
18
19
20
21
22
Tem
pera
ture
[o C]
0 6 12 18 0-2
-1
0
1
2
Res
idua
l [o C
]
15.7.2006
15.7.2006
12.2.2007
22Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Residual Example: Two Stations
0 1000 2000 3000 4000 5000 6000 7000 8000
10
15
20
25
T (o C
)
Juhanila A
0 1000 2000 3000 4000 5000 6000 7000 8000-1
-0.5
0
0.5
1
Iteraatio
Enn
uste
virh
e (o C
)
Juhanila stations August 2005
A‐station: 2 m –level
B‐station: 100 m ‐level
Juhanila A‐station
Tempe
rature
Residu
al
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
10
15
20
25
30
T (o C
)
Juhanila B
0 1000 2000 3000 4000 5000 6000 7000 8000-10
-5
0
5
10
Iteraatio
Enn
uste
virh
e (o C
)
Juhanila B‐station
Residu
alTe
mpe
rature
12.2.2007
23Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Further Quality Control Problems
As mentioned earlier, the current fault detection works well in detecting spikes and noise– Fault/weather phenomena –separation is always an issueDrift detection is much more tedious task – Detection seems possible using either neighboring stations or nowcasted values
– An exception: barometric pressureDuring heavy weather phenomena the measurements have special behavior that is not included in fault detection yet (e.g. wind, pressure and temperature in convection)
Fault detection of non‐continuous measurements: rain→ Room for further QC work
12.2.2007
24Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
From Measurement Quality Control to Measurement Network Performance
For maintenance and forecasting purposes, knowing the measurement station performance is also essential– Additional information for the end user about the measurement quality
– Maintenance operation planningIn order to gain information about the network condition, descriptive performance indices about each station can be used– For example: availability, accuracy, reliability, estimability, influence
12.2.2007
25Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Descriptions of Measurement Network Performance
Short descriptions of performance indices :– Availability = ”are there missing measurements”– Accuracy = ”is the measurement accurate or not”– Reliability = ”can the measurement station be trusted”– Estimability = ”can the network compensate the measurement”
– Influence = ”how the measurement is influencing the network performance”
For maintenance
For data users
12.2.2007
26Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
A Performance Index Example
Example temperature measurements:
Measurement OK Low accuracy Low availability
12.2.2007
27Vesa Hasu: More Quality Control for Weather Station Networks TKK – Control Engineering Laboratory
Conclusions
Automatic quality control is a necessity when increasing the mesoscale measurement network sizeTraditional quality control is the starting point, the newer type algorithms can improve the resultsUsing background field is not always a feasible assumptionQuality control gives an opportunity to form new metadata to maintenance decisions