“validation of the h-saf precipitation...

of 60

Validation of the H-SAF precipitation products over Greece using rain gauge data

H-SAF VSA Program

“Validation of the H-SAF precipitation products

over Greece using rain gauge data”

Visiting – Associate Scientist Activity HSAF_CDOP2_VS14_01_UNIFE_DCP

Final Report

Haralambos Feidas

Aristotle University of Thessaloniki

July 2015

of 60


Acknowledgements

This work was made possible by the support and assistance of a number of people whom I would like to personally thank.

First, I would like to personally thank Dr Konstantinos Lagouvardos and Dr. Vasiliki Kotroni from the National Observatory of Athens for giving access to the rain gauge data of the meteorological station network of the Institute, making possible the completion of this study.

A special thank goes to Silvia Puca and Angelo Rinolo from the Italian Department of Civil Protection (DCP) for their determination to support the idea and take the responsibility of submitting the proposal for this Associate-Visiting Scientist Activity.

It is a great pleasure to thank prof. Federico Porcu from the University of Ferrara and Gianfranco Vulpiani for their precious help during the two missions in Ferrara and Rome.

Finally, I would like to thank Cap. Davide Melfi, Science Coordinator of the H-SAF program, for his support in technical aspects of this activity.

of 60


CONTENTS

1. Objectives and Rationale ............................................................................................................ 5

2. Tasks ............................................................................................................................................. 5

3. Data and Methods ....................................................................................................................... 6 3.1 Data ....................................................................................................................................... 6 3.2 Methodology .......................................................................................................................... 7

i. Evaluation of interpolation techniques ............................................................................. 7 ii. Validation analysis ........................................................................................................... 7

4. Results ........................................................................................................................................ 10 4.1 Evaluation of interpolation techniques ................................................................................. 10 4.2 Quality index of interpolated data ........................................................................................ 18

i. The impact of stations’ density ..................................................................................... 18 ii. The impact of elevation difference ............................................................................... 18 iii. The quality index .......................................................................................................... 21

4.3 Validation of the H03 and H15 products .............................................................................. 23 iv. Continuous statistics ................................................................................................. 23 v. Multicategorical statistics ........................................................................................... 28 vi. Optimum interpolation ............................................................................................... 34 vii. The impact of stations’ density .................................................................................. 38 viii. The impact of elevation ............................................................................................. 42 ix. The impact of elevation difference ............................................................................ 46 x. The impact of the quality of interpolated data ........................................................... 49 xi. One example ............................................................................................................. 52

5. Conclusions ............................................................................................................................... 56 References .................................................................................................................................59

of 60


VSA Summary

VSA title Validation of the H-SAF precipitation products over Greece using rain gauge data

VSA ID: HSAF_CDOP2_VS14_01_UNIFE_DCP

Objective category (if applicable):

Complementary validation

VSA Host institute: University of Ferrara/DPC

Related SAF products:

PR-OBS-3; PR-OBS-6

VSA supervisor: Federico Porcù/

Angelo Rinollo/

Silvia Puca

Related SAF WP: WP6100

Start date: 15 January 2015 End date: 31 March 2015

VS: Haralambos Feidas VS / AS: VS-AS

of 60


1. OBJECTIVES AND RATIONALE

The success of any satellite-based rainfall product depends greatly on the quantitative understanding of its performance for various seasons, regions and climatological regimes. Huffman (1997) and Kummerow et al. (2000) have pointed out the need for adequate validations on a regional basis instead of using global approaches. In this context, the Mediterranean region, having various climatic regions and a diverse topography with abrupt alternations of land and sea masses, provides a suitable platform for validating satellite-derived precipitation data. An extensive comparison of the H-SAF satellite rainfall products over the Greece is lacking.

Moreover, the complex topography in Greece, together with the abrupt alternation between land and sea induced by the large coastlines and the many islands of this area, is a challenge to many satellite rainfall products. Greece has been the subject of several validation studies of satellite-based rain estimation algorithms (Feidas, 2006, 2010; Feidas et al., 2005, 2007, 2009; Kamarianakis 2008). Thus, a specific validation for this region would be useful to get some level of confidence in using the H-SAF satellite estimates for this area.

The main objective of the VSA is to conduct an extensive validation and intercomparison of two H-SAF Precipitation Products (PR-OBS-3 and PR-OBS-6), at the product time resolution for a one year period, and evaluated at the monthly timescales, using gauge observations from a relatively dense network of 233 stations over Greece. The dependence of validation results on the geophysical characteristics of the station sites will be also considered in the validation exercises. This will be linked to a comparative study of different interpolation techniques for rain gauge data, and a quality evaluation for the interpolated rain gauge data.

2. TASKS

The activity involves two main tasks:

Task 1. Spatial and temporal matching of precipitation products (satellite and gauge).

The first task aims at producing spatially continuous reference rainfall maps starting from punctual rain gauge data. Different interpolation techniques will be tested, in comparison with the technique adopted by the H-SAF PPVG, in order to check the liability of them over the Greek area. A method for the calculation of a quality index of interpolated rain gauge data will also be proposed.

Task 2. Evaluation and stratification of validation statistics.

In this Task, validation will be performed using the common codes developed by the PPVG, and results will be stratified by the geophysical characteristics of the terrain in order to provide a location-dependent comparison of the H-SAF precipitation products and terrain in Greece, a country characterized by a large geographical diversity. The dependence structure between validation statistics and geophysical characteristics of the terrain will be investigated and stratified according to each geographical feature to further elucidate the nature of the errors.

of 60


3. DATA AND METHODS

3.1 Data

The validation area covers the Greek peninsula, located in south-eastern Europe, an area of complex terrain and with a large coastline (Figure 1). A large number of small islands are interspersed mainly in the Aegean Sea, east of the continental Greece. The main rainy season is during the cold period of the year (October to April) whereas warm season (May to September) is dry with winter rainfall being at least three times the summer rainfall. Western Greece receives the majority of rainfall, more than 1500 mm/year, while Eastern Greece, along with the islands of Aegean and Crete (except the western part of the island), have considerably lower precipitation totals.

The validation is based on a H-SAF reference period (June 2012 – May 2013). Precipitation rates provided by the H-SAF precipitation products are validated and inter-compared against rain gauge data. Two instantaneous (PR-OBS-3 and PR-OBS-6) rain products are validated over Greece. Validation is conducted at the product time resolution and evaluated on monthly, seasonal and annual basis.

Rain gauge data from ~233 stations will be used as rainfall information to validate the satellite products. Rain data for these days have been provided by the National Observatory of Athens as 10 min rainfall accumulation values. The station locations are shown in Figure 1. The average minimum distance (AMD) of the stations is 15.1 km. The spatial distribution of the stations is not homogeneous due to the complex terrain and the large number of islands. As a result there is a number of isolated stations situated manly in the islands of the Aegean Sea.

It is worth mentioning that 20 out of the 233 stations are deployed in locations with an altitude exceeding 1000 m, while 51 and 35 of them are coastal and island stations, respectively. Real time data as well as the database of the available measurements is provided at the institute web-site: http://www.meteo.gr/meteosearch (in Greek).

Figure 1. The network of the 233 automated surface stations operated by the National Observatory of Athens used in the study (http://penteli.meteo.gr/meteosearch/map.asp).

http://penteli.meteo.gr/meteosearch/map.asp

of 60


3.2 Methodology

i. Evaluation of interpolation techniques

Spatially continuous reference rainfall maps from the punctual rain gauge data are used as a reference to validate satellite estimates. Different interpolation techniques will be tested, in comparison with the technique adopted by the H-SAF PPVG, in order to check the liability of them over the Greek area.

Two different interpolation techniques have been tested in this study: the Barnes method (Barnes, 1964), and the Random Generator of Spatial Interpolation from uncertain Observations (GRISO). The GRISO (Pignone et al., 2010) is an improved Kriging-based technique implemented by the International Centre on Environmental Monitoring (CIMA-Research Foundation). Both methods have been tested by PPVG resulting in the choice of GRISO as the standard interpolation method in the common validation codes.

The decision of which interpolation model provides the best prediction is made by validating the interpolation field. The validation analysis first removes part of the data (test dataset) then uses the rest of the data (training dataset) to develop the validation model to be used for prediction. Models’ predictions are validated against the test dataset by computing and evaluating four statistical measures (mean error, absolute mean error, root mean square error and correlation coefficient).

The development of a quality index map associated to the reference ground data is important to evaluate the reliability of ground data which varies spatially, affecting also the validation results. Quality index, which is function of position, resumes into a number between 0 and 1 all the information useful to define the reliability of the ground data to which it is associated (Puca et al., 2013).

The quality index developed in this study is an empirical weighted function of two quality indices: (a) a density quality index which takes into account the number of stations in the vicinity of the grid point, and (b) the orography quality index, which is based on the average slope among the grid point and the stations in its vicinity.

Before developing the quality index, the impact of the uneven rain gauge station spatial distribution on the quality of the interpolated field is examined by validating the interpolated data in areas with different station densities. In addition, the impact of the elevation difference between grid points and gauges on the quality of the interpolated data is also investigated.

ii. Validation analysis

The study follows the common validation methodology defined by the PPVG (Precipitation Product Validation Group) and applies the related software developed by the group in order to guarantee the uniformity of the results with the other countries involved in the validation activity.

Comparisons (sat vs obs) are made on the satellite native grid. Rain gauge data are up-scaled to the satellite native grid based on the scanning geometry of the sensors according to the methodology developed by the PPVG. Ground truth data are averaged in time and space in order to produce coincident and collocated datasets of H-SAF products and rain gauge datasets for the area of Greece.

of 60


The validation methodology is based on rain gauge data comparisons to produce large statistic (multi-categorical and continuous) at monthly timescales.

The main steps of the validation procedure are:

Spatial and temporal matching of precipitation products (satellite and gauge) to produce comparable ground and satellite datasets. This involves the interpolation of gauge data in order to be spatially matched to the satellite native grid and the temporal match of the two dataset to ensure a direct comparison between the satellite and ground precipitation intensity.

Temporal matching: Both satellite rain estimates and gauge measurements are expressed as rain rates averaged in hourly intervals. The 10min rain gauge data were time-averaged on hourly intervals and stored in 12 files, corresponding to the 12 months of the period June 2012 – May 2013. Each file consists of 672 to 744 records (28-31 days x 24 hours) or instants.

PR-OBS-3 and PR-OBS-6, based on geostationary IR data, provide four instantaneous rain estimates each hour. In this case, an hourly average rain rate is estimated by averaging the measurements inside the validation hour, which is then compared with the corresponding rain gauge value. The hourly averaged PR-OBS-3 and PR-OBS-6 rain rate is computed as a weighted average of the 5 slots acquired at the nominal acquisition time of SEVIRI sensor (at 12, 27, 42 and 57 minutes) for each hour.

Spatial matching: In order to obtain a regular field for comparison with satellite products, the rain gauge measurements are interpolated onto a unique European grid, with grid cells of size 5 Km (similar to SEVIRI resolution). The interpolation of gauge data is based on the GRISO interpolation technique (Pignone et al., 2010) which is the default on the common validation codes used by the by the PPVG.

The resolution of the interpolated grid is nearly the same as IR-based satellite products PR-OBS-3 and PR-OBS-6. Thus, the satellite pixels and the interpolated rain gauge field grid points are spatially matched following a nearest-neighbor approach (Puca et al., 2013). In both cases, errors due to the displacement between satellite and ground data are neglected.

Evaluation of validation statistics. Monthly continuous verification and multi-categorical statistical scores are computed and evaluated for the reference period.

Once the ground data are temporally and spatially matched with satellite estimates, the validation is performed on satellite-ground data pairs. The statistical scores are evaluated on a monthly basis for the reference period.

Precipitation below the threshold of 0.25 mm/hr is classified as no-rain. For the measurements above this threshold, three precipitation classes are introduced (Table 1).

Table 1. Classes for instantaneous rainrates used to validate the products.

Class Rain rate (RR) products

Class 1(no-rain class) RR <0.25 mmh-1

Class 2 0.25 mmh-1 < RR < 1 mmh-1

Class 3 1 mmh-1 < RR < 10 mmh-1

Class 4 10 mmh-1 < RR

of 60


The following statistical scores are calculated (see Nurmi, 2003 for reference):

Continuous statistics: Mean Error (ME), Mean Absolute Error (MAE), Standard Deviation (SD), Root Mean Square Error (RMSE), Multiplicative Bias (MB), Correlation Coefficient (R);

Multi-category statistics: Contingency Table, Probability of Detection (POD), False Alarm Ratio (FAR), Probability of False Detection (POFD), Bias, Critical Success Index (CSI), Equitable Threat Score (ETS) and Hanssen and Kuipers (HK) score;

Moreover, rain rate Probability Distribution Functions are computed on monthly basis to evaluate the capability of the satellite products to describe the range of precipitation rates.

Stratification of validation statistics. The validation statistics are stratified according to the geophysical characteristics of the terrain (land, coastal, sea pixels, elevation) to further elucidate the nature of the errors.

Sensitivity of the validation analysis. (a) The impact of the interpolation model configuration on the validation statistics is examined by changing the radius of influence of the station in the interpolated data, and (b) the impact of the quality of the interpolated data on the validation results is also investigated based on the gauges’ density and the elevation difference between grid points and gauges. The possibility of using the quality index of the interpolated data as a filter in the validation procedure is also addressed.

of 60


4. RESULTS

4.1 Evaluation of interpolation techniques

The Barnes and GRISO interpolation models were validated by removing part of the data (test dataset) and then using the rest of the data (training dataset) to develop the validation model. The test dataset is used as a reference to validate the models based on the evaluation of statistical measures (mean error, absolute mean error, root mean square error and correlation coefficient).

In this study, the test dataset is produced by removing 13 out of the 233 rain gauge stations (Figure 2). These stations were selected based on the following criteria: (a) to be evenly distributed and (b) not to be isolated, meaning to have at least one station in a close distance (<30 km).

Figure 2. The 13 stations used as a reference for validating Barnes and GRISO interpolation models.

Table 2 presents the total validation statistics for GRISO and Barnes models for the entire period (June 2012-May 2013). There is no any significant difference between the statistics of the two models except for the mean error which indicates a strong underestimation of rain rates by the GRISO predictions. This underestimation is present in all months yet more pronounced during the rainy period of the year (October to March) (Figure 3). The MAE and RMSE are higher for GRISO for the half of the year and lower for the rest of the year. As a result these errors for the whole period are almost equal for both interpolation models. A slight improvement of the correlation coefficient is induced by Barnes interpolator.

The large underestimation of rainfall by the GRISO interpolation model can be attributed to the low density of the station network and the “bull eye” pattern of the interpolation surface in combination to the small (25km) radius of influence of each station. This is evident in Figure 4, in which the interpolated rain field is presented for an instant using first all the stations (233 stations) and then only the training dataset (220 stations). For example, the large values of rain recorded in stations numbered as 1, 2 and 3, which are located at the edges of the “bull eye”

of 60


cycles of interpolated data, cannot predicted adequately by the GRISO interpolation model. This indicates that the GRISO model underestimates rain at the edges of the radius of influence, especially when the network density is sparse. On the contrary, the Barnes code produces more spatially uniform predictions around the stations (Figure 5) and as a result predictions over the test stations are closer to the real values, even though the interpolated rain surface is less realistic that that of the GRISO.

The previous results indicate the impact of the radius of influence of each station to the GRISO predictions. This impact is reflected in the validation statistics when different radius of influence (25 km, 30km, and 35km) is used in the GRISO model (Figure 6). The results show a clear improved performance of the GRISO interpolation with increasing radius on influence, from the default 25 km radius used in the common validation codes of the PPVG to 30 km and 35 km.

The same sensitivity analysis was performed for three different analysis length scale (8km, 20km, and 60km) of the weight function used in the Barnes model. Validation results are optimized for a 20 km analysis length scale, a little higher than the default length scale of 8 km used in the common validation codes.

Table 2. Validation statistics for GRISO and Barnes models for the entire period (June 2012-May 2013).

Class GRISO Barnes

N 91873 103072 Mean test 0.13 0.12 Mean training 0.02 0.11 Mean Error -0.11 -0.02 Mean Absolute Error 0.12 0.12 Root Mean Square Error 0.85 0.91 Correlation Coefficient 0.36 0.40

The comparison of the two optimized GRISO and Barnes configurations does not exhibit any significant and distinct difference between them (Figure 6). More precisely, the GRISO configuration with a 35 km radius of influence presents the lower mean absolute error and the best correlation coefficient whereas the Barnes model with 20 km analysis length scale provides the lowest mean error and RMSE (although very close to that of the GRISO model).

Figure 7 shows the interpolated rain field is presented for an instant using the GRISO best configuration (35 km radius of influence), again first for all the stations (233 stations) and then only for the training dataset (220 stations). The improvement induced by the larger radius of influence is evident in the more realistic appearance of the spatial distribution of the rain field as a result of a less frequent occurrence of “bull eye” patterns. Consequently, the large values of rain recorded in stations numbered as 1, 2 and 3, are predicted more accurately this time by the GRISO interpolation model.

As slight more realistic appearance is also distinct on the interpolated field produced by the Barnes model with the 20 km analysis length scale (Figure 8). In this case, however, there are apparently false predictions over distant areas, such as those over the Ionian Sea. These predictions could be removed with a mask when they are used as a reference in a validation analysis. The Barnes model discard of some stations from the interpolation does not have any significant impact on the interpolated surface, indicating the robustness of the Barnes model.

of 60


Figure 3. Validation statistics (mean error, absolute mean error, root mean square error and correlation coefficient) at monthly scale for GRISO and Barnes interpolation models.

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

Me

an E

rro

r (m

m/h

)

Mean Error

GRISO

Barnes

0

0.05

0.1

0.15

0.2

0.25

Me

an A

bs.

Err

or

(mm

/h)

Mean Absolute Error

GRISO

Barnes

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

RM

SE (

mm

/h)

RMSE

GRISO

Barnes

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Co

rre

lati

on

Co

effi

cie

nt

Correlation Coefficient

GRISO

Barnes

of 60


Figure 4. An example of interpolation surfaces for GRISO model using data from (a) all stations (233 stations) and (b) the training dataset (220 stations) (default configuration). The 13 excluded stations used as a test dataset are marked with red cycles in (a). Red and yellow arrows indicate

three test stations (see text for details).

1

2

3

3

2

1

of 60


Figure 5. Same as Figure 4 but for the Barnes model (default configuration).

1

2

3

1

2

3

of 60


Figure 6. Validation statistics for three different configurations of the GRISO (radius of influence

25km, 30km, and 35km) and Barnes model (analysis length scale 8km, 20km, and 60km).

-0.12

-0.10

-0.08

-0.06

-0.04

-0.02

0.00

GRISO25km

GRISO30km

GRISO35km

Barnes8km

Barnes20km

Barnes60km

Me

an E

rro

r (m

m/h

)

Mean Error

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

GRISO25km

GRISO30km

GRISO35km

Barnes8km

Barnes20km

Barnes60km

Me

an A

bs.

Err

or

(mm

/h)

Mean Absolute Error

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

GRISO25km

GRISO30km

GRISO35km

Barnes8km

Barnes20km

Barnes60km

RM

SE (

mm

/h)

RMSE

0.00

0.10

0.20

0.30

0.40

0.50

0.60

GRISO25km

GRISO30km

GRISO35km

Barnes8km

Barnes20km

Barnes60km

Co

rre

lati

on

Co

effi

cie

nt


of 60


Figure 7. Same as Figure 4 but for the optimum configuration of the GRISO model (35 km radius

of influence).

1

2

3

1

2

3

of 60


Figure 8. Same as Figure 4 but for the optimum configuration of the Barnes model (20 km of the

analysis length scale).

1

2

3

1

2

3

of 60


4.2 Quality index of interpolated data

The development of a quality index of the interpolated data of the default GRISO model is important to evaluate the reliability of ground data, affecting also the validation results. It is expected that the quality of the interpolated field depends on parameters such as the density of the station network and the orography.

i. The impact of stations’ density

The impact of the uneven rain gauge station spatial distribution on the quality of the interpolated field is examined by validating the interpolated data in areas with different station densities. Again, the validation analysis first removes part of the data (test dataset) then uses the rest of the data (training dataset) to develop the validation model to be used for prediction. To this end, the number of the stations found in a circle of 15 Km radius around each of the 233 stations of the gauge network is counted and four test datasets are constructed by removing stations in different station density areas (1 station, 2 stations, 3 stations and more than 4 stations in the vicinity of the removed station).

GRISO model prediction is validated against each test dataset by computing and evaluating four statistical measures. The results are presented in Figure 9. There is a clear improvement of all the validation statistics in areas with increasing density of stations. A slight increase of errors is observed only from 3 to more than 4 stations.

It has to be noticed that the increase of the stations’ density results in the rise of the mean rain rate estimated by the interpolation model and consequently in the decrease of the ME. The low mean rain rates derived by the interpolation model for low station densities could be attributed to the “bull-eye pattern” of the interpolation field which in fact reduces the rain rates with the distance from the station.

ii. The impact of elevation difference

Precipitation generally tends to increase with elevation due to orographic lift. Mountains force air masses passing through to rise, the air cools adiabatically causing condensation, cloud formation and eventually precipitation.

Lack of high elevation stations may induce underestimation of precipitation rates by the spatial interpolation models over mountainous areas. This assumption can be justified by validating the interpolated data in areas with high terrain slopes. To achieve this, the mean slope (MS) of each station s with the neighboring stations i located in a 15 km distance, is calculated according to the following equation:

N

i i

is

d

zz

NS

1

1M

(1)

where zs is the elevation of the station in question s, zi is the elevation of the neighboring stations located in a 15 km distance from the station s, di is the distance between the station s and the neighboring stations i and N is the number of stations in a 15 km radius around the station s.

Three test datasets are constructed by removing stations with MS values belonging to three classes (0 – 0.05, 0.05 – 0.1, 0.1 – 0.2). GRISO model prediction is validated using the training dataset against each of the three test datasets. Only positive MS values were used (the station in

of 60


question is located in higher elevation than the surrounding stations) in order to have meaningful mean error score. Results are presented in Figure 10. There is an overall model performance degradation as the mean slope among the station increases. Only for ME this deterioration is not strictly proportional to the mean slope.

Figure 9. Validation statistics for the GRISO interpolation model using four test datasets constructed by removing stations in different station density areas (1 station, 2 stations, 3 stations and more than 4 stations in the vicinity of the removed station).

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

1 station 2 stations 3 stations >4 stations

Me

an R

ain

(m

m/h

)

Mean Raingaugesmodel

-0.12

-0.10

-0.08

-0.06

-0.04

-0.02

0.00

0.02


Me

an E

rro

r (m

m/h

)

Mean Error

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14


Me

an A

bs.

Err

or

(mm

/h)

Mean Absolute Error

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0


RM

SE (

mm

/h)

RMSE

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00


Co

rre

lati

on

Co

effi

cie

nt


of 60


Figure 10. Validation statistics for the GRISO interpolation model using three test datasets constructed by removing stations with mean slope values belonging to three classes (0 – 0.05, 0.05 – 0.1, 0.1 – 0.2).

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0-0.05 0.05-0.1 0.1-0.15

Me

an R

ain

(m

m/h

)

Mean slope

Mean Raingaugesmodel

-0.12

-0.10

-0.08

-0.06

-0.04

-0.02

0.00

0-0.05 0.05-0.1 0.1-0.2

Me

an E

rro

r (m

m/h

)

Mean slope

Mean Error

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0-0.05 0.05-0.1 0.1-0.2

Me

an A

bs.

Err

or

(mm

/h)

Mean slope

Mean Absolute Error

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0-0.05 0.05-0.1 0.1-0.2

RM

SE (

mm

/h)

Mean slope

RMSE

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0-0.05 0.05-0.1 0.1-0.2

Co

rre

lati

on

Co

effi

cie

nt

Mean slope


of 60


iii. The quality index

The quality index is function of position, resuming into a number between 0 and 1 all the information needed to define the reliability of the interpolated data. The quality index developed in this study is an empirical weighted function of two partial quality indices:

(a) a density quality index (DQI) which takes into account the number of stations in the vicinity of the grid point. To estimate the DQI the number of stations located in a 25 Km distance (the radius of influence of the GRISO model) is counted and:

If there are 4 or more stations then DQI=1 If there are 3 stations then DQI=0.75 If there are 2 stations then DQI=0.5 If there is only 1 station then DQI=0.25 If there is no station then DQI=0

(b) an orography quality index (OQI), which is based on the average slope among the grid point and the stations in its vicinity. For each grid point, the mean slope with all the stations located in a 25 Km distance is calculated as follows:

N

i i

igp

d

zz

NS

1

1M

(2)

where zgp is the elevation of the grid point, zi is the elevation of the neighboring stations located in a 25 km distance from the grid point, di is the distance between grid point and the neighboring stations i and N is the number of stations in a 25 km radius around the grid point.

The OQI is calculated as a function of MS:

If MS < 0.1 then OQI = 1 – (10MS) If MS > 0.1 then OQI = 0

The overall quality index (QI) is calculated as the product of the two partial quality indices QDI and ODI:

QI = DQI OQI (3)

The spatial distribution of the overall quality index QI along with the partial quality indices DQI and OQI is presented in Figure 11.

of 60


Figure 11. The spatial distribution of (a) the density quality index (DQI), (b) the orography quality index (OQI) and (c) the overall quality index (QI) of the interpolated data produced by the 233 stations in Greece.

of 60


4.3 Validation of the H03 and H15 products

The validation of the two satellite precipitation products (H03 and H15) is based on the comparison with the independent dataset derived by the interpolation of the 233 rain gauges over Greece. The analysis presented here has been performed on one year of data (June 2012-May 2013), aggregated at monthly and annual values.

iv. Continuous statistics

The continuous statistical indicators are computed only over the pixels where at least one rain value (satellite product or reference field) is greater than 0.25 mm h-1 (class 5), to avoid the contribution of the dominant amount of zero-zero samples.

The validation results of the H03 product show a yearly RMSE of around 2.5 mmh-1 and MAE of 1.35 mm h-1 (Figure 12). These values correspond to the 190% and 105% of the mean rain rate, respectively. The percentage RMSE (RMSE%) is high (around 260%) whereas the poor correlation

coefficient indicates virtually uncorrelated data (0.15). There is an overall tendency to underestimate the rain gauge rates (ME = -0.43 mm h-1 and Multiplicative Bias =0.66).

The H03 product estimates the same mean rain rates over terrains with different geophysical characteristics (land, coastal, sea) (Figure 12). Rain gauges derived mean rain rates, however, are different over land, sea and coastal pixels. There is no any distinct signal of change of validation statistics between land and sea or coastal grid points. ΜΕ show better results as we move from land towards coastal and sea grid points, whereas MAE and RMSE remain unchanged. The relative RMSE% error exhibits an opposite behavior to that of ME, increasing from land to coastal and sea grid points. This is the result of the different mean rain rates observed over those areas.

The yearly validation results for the H15 product report a RMSE of around 9.5 mmh-1 and MAE equal to 3.8 mmh-1 (Figure 13), corresponding to the 560% and 225% of the mean rain rate, respectively. These errors are double of those found for the H03 product. The percentage RMSE (RMSE%) is extremely high (around 1500%) whereas the almost zero correlation coefficient points out to uncorrelated data. In contrast to the H03 product, the H15 product significantly overestimates the rain gauge rates (ME = 2.1 mm h-1 and Multiplicative Bias =2.2).

The H15 mean rain rate estimates are different over land, coastal and sea grid points (Figure 13). As a result, all the validation statistics show better performance over land pixels and worst results over coastal and sea pixels.

According to the validation results on a monthly basis, H03 product has reached the best performance during the cold period of the year (December to April), in terms of MAE and RMSE (Figure 14). According to the ME and Bias scores, underestimation of precipitation is present all over the year, except for September and October. Similar results are obtained for the H15 product, with best performance obtained again during the cold period of the year (December to April) (Figure 15). In this case, very high MAE and RMSE errors are produced for autumn, during which the H15 causes a significant overestimation of the rain gauge rates as well according to the ME and Multiplicative Bias.

of 60


Figure 12. Continuous validation statistics for the H03 product for the whole period (June 2012 – May 2013) and class 5 (rain rate > 0.25 mm/h).

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

All Land Coast Sea

Me

an R

ain

(m

m/h

)

Mean Rain

gaugesH03

Class 5

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0.0

All Land Coast Sea

Me

an e

rro

r (m

m/h

)

Mean Error

Class 5

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

All Land Coast Sea

Me

an A

bso

lute

err

or

(mm

/h)

Mean Absolute Error Class 5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

All Land Coast Sea

RM

SE (

mm

/h)

RMSE Class 5

0

50

100

150

200

250

300

350

400

All Land Coast Sea

RM

SE (

%)

RMSE% Class 5

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

All Land Coast Sea

Mu

ltip

licat

ive

Bia

s

Multiplicative Bias Class 5

0

0.04

0.08

0.12

0.16

0.2

All Land Coast Sea

Co

rre

lati

on

Co

effi

cie

nt

Correlation Coefficient Class 5

of 60


Figure 13. Same as Figure 12 but for the H15 product.

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

All Land Coast Sea

Me

an R

ain

(m

m/h

)

Mean Rain gaugesH15

0.0

0.5

1.0

1.5

2.0

2.5

3.0

All Land Coast Sea

Me

an e

rro

r (m

m/h

)

Mean Error Class 5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

All Land Coast Sea

Me

an A

bso

lute

err

or

(mm

/h)


0

2

4

6

8

10

12

All Land Coast Sea

RM

SE (

mm

/h)

RMSE Class 5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

All Land Coast Sea

Mu

ltip

licat

ive

Bia

s

Multiplicative Bias

0

200

400

600

800

1000

1200

1400

1600

1800

2000

All Land Coast Sea

RM

SE (

%)

RMSE% Class 5

0

0.04

0.08

0.12

0.16

0.2

All Land Coast Sea

Co

rre

lati

on

Co

effi

cie

nt


of 60


Figure 14. Continuous validation statistics for the H03 product on a monthly basis (period June 2012 – May 2013) for class 5 (rain rate > 0.25 mm/h for all pixels.

0

0.5

1

1.5

2

2.5

Me

an R

ain

(m

m/h

)

Mean Rain

Gauges

H03

Class 5

-1.2

-1.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

Me

an e

rro

r (m

m/h

)

Mean ErrorClass 5

0.0

0.5

1.0

1.5

2.0

2.5

Me

an A

bso

lute

err

or

(mm

/h)

Mean Absolute ErrorClass 5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

RM

SE (

mm

/h)

RMSEClass 5

0

100

200

300

400

500

600

RM

SE (

%)

RMSE%Class 5

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

Mu

ltip

licat

ive

Bia

s

Multiplicative BiasClass 5

0

0.05

0.1

0.15

0.2

0.25

Co

rre

lati

on

Co

effi

cie

nt

Correlation CoefficientClass 5

of 60



0

2

4

6

8

10

12

Me

an R

ain

(m

m/h

)

Mean Rain

Gauges

H15

-4.0

-2.0

0.0

2.0

4.0

6.0

8.0

10.0

Me

an e

rro

r (m

m/h

)

Mean Error

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0

10.0

Me

an A

bso

lute

err

or

(mm

/h)

Mean Absolute Error

0.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

16.0

18.0

20.0

RM

SE (

mm

/h)

RMSE

0

500

1000

1500

2000

2500

3000

RM

SE (

%)

RMSE% Class 5

0.0

1.0

2.0

3.0

4.0

5.0

6.0

Mu

ltip

licat

ive

Bia

s

Multiplicative Bias

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

Co

rre

lati

on

Co

effi

cie

nt


of 60


v. Multicategorical statistics

Multi-categorical statistics has been also performed on the same validation period with rain gauge data.

A multi-categorical contingency table is a statistical table classifying observed data frequencies into several classes. Contingency tables are obtained by dividing precipitation events into four classes, as shown in Table 1. Each column of the table classifies the events detected by the rain gauges in a class into the four classes of the satellite product, while each row reports the rain rate classification of the satellite product into the classes of the rain gauge observations (Table 3 and 4). Each cell shows the frequency of occurrence of a class of satellite estimates into the four classes of the observations of the rain gauges. The percentages shown in a given column are computed with respect to the total number of satellite samples and represent the occurrence frequency of a rain rate class of gauge observations for each rain rate class of the satellite estimates. Ideal condition should be 100% of events in the main diagonal of the table.

Rain intensity distribution in the contingency table demonstrates that the H03 algorithm is able to discriminate efficiently the rain from the no rain events (Table 3). About 96% of no rain events (first class with rain rates < 0.25 mm/h) are correctly identified by H03. However, the percentages are very high also in the other cells of the first row in all the tables, indicating that a large number of rain events are missed by the H03 satellite product. It is also clear that this product tends to underestimate rain rate classes. H03 seems to better resolve low precipitation classes, with higher percentages in the first two cells of the main diagonal.

The H15 product focuses on the estimation of convective precipitation using NEFODINA system to define the areas of convection and redistribute the initial rainfall-rate estimation. By default, this product is not meant to discriminate the rain from the no rain events and this is evident in the low percentage (25-26%) of no rain events correctly identified by H15. In contrast, the percentages are small in the other cells of the first row in all the Tables, indicating that only a small number of rain events are missed by the H15 satellite product. As expected, H15 is more effective in classifying higher rain rates classes, with higher percentages in the last three cells of the main diagonal. No any significant bias is apparent in the contingency tables.

No any district differentiation is found between the contingency tables obtained for different geophysical characteristics of the terrain (pixels).

In Figure 16, the annual values of the categorical statistical scores for rain/no rain discrimination (rain rate threshold = 0.25 mm/h) are reported for H03, compared with rain gauge precipitation observations, for the whole period (June 2012 – May 2013) and stratified by the geophysical characteristics of the terrain (all, land, sea and coastal pixels). The most prominent feature is the very high FAR (more than 75%) and the relatively low POD (less than 40%). As a result, the CSI and the ETS scores are low (less than 0.16). According to the Bias score, the H03 algorithm overestimates the rainy pixels between 1.5 to 3.5 times. The Accuracy scores take very high values (94-96%) as a result of the efficiency of the algorithm to discriminate the rain from the no rain and the fact that the datasets are dominated by no rain rates. For the same reason the POFD is very low and the Accuracy gives very high values.

Validation statistics for the different geophysical characteristics of the terrain (pixels) show a slight better performance for land than for coastal pixels, whereas sea pixels provide slightly worse results.

of 60


The annual values of the categorical statistical scores for the H15 product are presented in Figure 17. POD scores are high enough (around 80%) whereas FAR and POFD take very large values (more than 70%). The high FAR and POFD means that there are many observed rain events as well as no-events that are false alarms. This leads to a very poor HK score, an indication of a poor algorithm ability to distinguish between rain and no-rain events. The CSI values indicate a better performance of the H15 algorithm compared to the H03. The H15 algorithm overestimates the rain events (more than 3 times) whereas the Accuracy, that is the fraction of the forecasts being correct, is rather low (less than 40%).

Validation statistics exhibit a significantly better performance for land than for coastal pixels, whereas sea pixels provide much worse results.

In Figure 18, the monthly values of the categorical statistical scores are presented for H03. The very high FAR values (more than 70%) exceed those of POD all over the year. POD is higher during late summer and autumn and lower during winter months. Worst FAR is obtained during spring and summer and best during winter. Bias score follows FAR with a large overestimation during spring, autumn and especially summer with values larger than 3. The CSI and its kindred ETS score present a simple annual cycle with better performance obtained for cold months. These scores have the advantage of not being dominated by the no-rain events, that is, CSI and ETS are only concerned with estimates that count. The HK score interannual variation follows that of POD with higher values during late summer and autumn and lower during winter months. HK score is a measure of the algorithm ability to distinguish rain events from no rain events, in fact is the difference between POD and POFD. Given that the datasets are dominated by no rain rates, the POFD is very low and as a result the HK score is determined by POD.

As for the H03 product, the better performances for H15 are obtained for cold months, according to the CSI and Accuracy scores (Figure 19). A large overestimation of the rain events is obtained during spring and especially in May. The HTS and HK scores are poor and almost zero during spring.

of 60


Table 3. Contingency table for the multi-categorical statistics of H03 as compared with rain gauges derived rain fields using all, land, sea and coastal pixels (rain rate threshold = 0.25

mm/h).

All

Gauge 1 Gauge 2 Gauge 3 Gauge 4 Gauge tot

Sat 1 96% 67% 54% 35% 82346027

Sat 2 3% 18% 21% 17% 2685242

Sat 3 1% 15% 24% 39% 1594590

Sat 4 0% 0% 1% 9% 24980

Sat tot 84599182 1323186 715634 12837 86650839

Land


Sat 1 96% 67% 53% 36% 44331012

Sat 2 3% 18% 21% 18% 1572995

Sat 3 2% 15% 25% 38% 997805

Sat 4 0% 0% 1% 8% 14321

Sat tot 45437504 933528 535310 9791 46916133

Sea


Sat 1 96% 66% 55% 30% 28898563

Sat 2 3% 18% 20% 14% 832733

Sat 3 1% 16% 23% 44% 443673

Sat 4 0% 1% 2% 12% 7637

Sat tot 29804528 263540 112762 1776 30182606

Coast


Sat 1 96% 68% 57% 35% 9116452

Sat 2 3% 17% 20% 18% 279514

Sat 3 1% 14% 22% 38% 153112

Sat 4 0% 0% 1% 9% 3022

Sat tot 9357150 126118 67562 1270 9552100

of 60


Table 4. Same as Table 3 but for the H15 product.


Sat 1 25% 21% 17% 10% 448264

Sat 2 47% 44% 37% 23% 858371

Sat 3 23% 28% 32% 28% 463884

Sat 4 8% 8% 13% 38% 111977

Sat tot 1468245 219490 188802 5959 1882496

Land


Sat 1 25% 21% 18% 11% 274321

Sat 2 48% 45% 38% 25% 549260

Sat 3 23% 27% 33% 28% 293624

Sat 4 4% 7% 12% 36% 64052

Sat tot 874770 157405 144643 4439 1181257

Sea


Sat 1 26% 20% 17% 7% 129154

Sat 2 45% 39% 35% 14% 229707

Sat 3 24% 29% 31% 31% 127211

Sat 4 6% 12% 17% 48% 35134

Sat tot 449378 42886 28029 913 521206

Coast


Sat 1 26% 21% 17% 12% 44789

Sat 2 46% 39% 35% 21% 79404

Sat 3 22% 28% 32% 27% 43049

Sat 4 5% 11% 16% 40% 12791

Sat tot 144097 19199 16130 607 180033

of 60


Figure 16. Categorical validation statistics (rain rate threshold = 0.25 mm/h) for the H03 product for the whole period (June 2012 – May 2013).


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

All Land Coast Sea

PO

D, F

AR

, PO

FD

POD, FAR, POFD

POD

FAR

POFD

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

All Land Coast Sea

CSI

, ETS

, HK

CSI, ETS, HK

CSI

ETS

HK

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

All Land Coast Sea

Bia

s

Bias

90%

91%

92%

93%

94%

95%

96%

All Land Coast Sea

Acc

ura

cy

Accuracy

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

All Land Coast Sea

PO

D, F

AR

, PO

FD

POD, FAR, POFD

POD

FAR

POFD

0.00

0.05

0.10

0.15

0.20

0.25

0.30

All Land Coast Sea

CSI

, ETS

, HK

CSI, ETS, HK

CSI

ETS

HK

0.0

1.0

2.0

3.0

4.0

5.0

6.0

All Land Coast Sea

Bia

s

Bias

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

All Land Coast Sea

Acc

ura

cy

Accuracy

of 60


Figure 18. Categorical validation statistics (rain rate threshold = 0.25 mm/h) for the H03 product

on a monthly basis (period June 2012 – May 2013).

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

PO

D, F

AR

, PO

FD

POD, FAR, POFD

POD

FAR

POFD

0,0

1,0

2,0

3,0

4,0

5,0

6,0

Bia

s

Bias

0,00

0,05

0,10

0,15

0,20

0,25

CSI

CSI

0,00

0,05

0,10

0,15

0,20

0,25

ETS

ETS

0,0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

HK

HK

0%10%20%30%40%50%60%70%80%90%

100%

Acc

ura

cy

Accuracy

of 60



on a monthly basis (period June 2012 – May 2013).

vi. Optimum interpolation

The impact of the interpolation model configuration on the validation statistics is examined by changing the radius of influence of the station in the interpolated data. According to the sensitivity analysis performed for different radius of influence (25 km, 30km, and 35km) of the GRISO interpolation model (see chapter 4.2), the optimum GRISO configuration was found for a 35 km radius of influence. To this end, the validation analysis of both algorithms (H03 and H15) was repeated using this optimum configuration for the GRISO interpolation model. A comparison of the validation statistics using the default and the optimum configuration of the GRISO interpolation model is presented in Figure 20 to Figure 23.

A slight improvement was found in the continuous validation statistics in terms of RMSE, correlation coefficient and especially RMSE%. The MAE remains virtually unchanged whereas only ME (and its kindred multiplicative Bias) is deteriorated. (Figure 20). A more distinct improvement is evident in all the statistical scores for the H15 algorithm Figure 21.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

PO

D, F

AR

, PO

FD

POD, FAR, POFD

POD

FAR

POFD

0,0

5,0

10,0

15,0

20,0

25,0

Bia

s

Bias

0,00

0,05

0,10

0,15

0,20

0,25

0,30

0,35

0,40

CSI

CSI

0,00

0,01

0,02

0,03

0,04

0,05

0,06

ETS

ETS

0,00

0,02

0,04

0,06

0,08

0,10

0,12

0,14

0,16

HK

HK

0%10%20%30%40%50%60%70%80%90%

100%

Acc

ura

cy

Accuracy

of 60


With reference to the categorical statistics, FAR is improved for H03 contrary to POD that has slightly decreased. As a result CSI, ETS and bias are slightly improved, with HK being the only score getting worse. Again, the signal of improvement is more marked in the H15 categorical validation statistics.

Figure 20. Continuous validation statistics for the H03 product for the whole period (June 2012 – May 2013) and class 5 (rain rate > 0.25 mm/h) for two different radius of influence of the GRISO

interpolation model (a) 25 km (default), and (b) 35 km (optimum).

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0.0

All Land Coast Sea

Me

an e

rro

r (m

m/h

)

Mean Error

25 km

35 kmClass 5

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

All Land Coast Sea

Me

an A

bso

lute

err

or

(mm

/h)

Mean Absolute Error

25 km

35 km

Class 5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

All Land Coast Sea

RM

SE (

mm

/h)

RMSE25 km

35 kmClass 5

0

50

100

150

200

250

300

350

400

All Land Coast Sea

RM

SE (

%)

RMSE%

25 km

35 km

Class 5

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

All Land Coast Sea

Mu

ltip

licat

ive

Bia

s

Multiplicative Bias25 km

35 km

Class 5

0

0.04

0.08

0.12

0.16

0.2

All Land Coast Sea

Co

rre

lati

on

Co

effi

cie

nt

Correlation Coefficient25 km

35 km

Class 5

of 60



0.0

0.5

1.0

1.5

2.0

2.5

3.0

All Land Coast Sea

Me

an e

rro

r (m

m/h

)

Mean Error

25 km

35 km

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

All Land Coast Sea

Me

an A

bso

lute

err

or

(mm

/h)

Mean Absolute Error25 km

35 km

0.0

2.0

4.0

6.0

8.0

10.0

12.0

All Land Coast Sea

RM

SE (

mm

/h)

RMSE25 km

35 km

0

200

400

600

800

1000

1200

1400

1600

1800

2000

All Land Coast Sea

RM

SE (

%)

RMSE%

25 km

35 km

Class 5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

All Land Coast Sea

Mu

ltip

licat

ive

Bia

s

Multiplicative Bias

25 km

35 km

0

0.04

0.08

0.12

0.16

0.2

All Land Coast Sea

Co

rre

lati

on

Co

effi

cie

nt

Correlation Coefficient25 km

35 km

Class 5

of 60


Figure 22. Categorical validation statistics (rain rate threshold = 0.25 mm/h) for the H03 product for the whole period (June 2012 – May 2013) for two different radius of influence of the GRISO

interpolation model (a) 25 km (default), and (b) 35 km (optimum).

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

All Land Coast Sea

PO

D

POD25 km

35 km

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

All Land Coast Sea

FAR

FAR 25 km

35 km

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16

0.18

All Land Coast Sea

CSI

CSI25 km

35 km

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16

All Land Coast Sea

ETS

ETS25 km

35 km

0.28

0.29

0.30

0.31

0.32

0.33

0.34

All Land Coast Sea

HK

HK25 km

35 km

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

All Land Coast Sea

Bia

s

Bias 25 km

35 km

of 60



vii.The impact of stations’ density

The impact of the uneven rain gauge station spatial distribution on the validation results is also examined by calculating validations statistics for grid points located in different station densities. To this end, the number of stations found in a circle of 25 Km radius around the grid point is counted and validation statistics are calculated for grid points with different station densities (1 station, 2 stations, 3 stations and more than 4 stations). The results are presented in Figure 24 to Figure 27. The default statistics using all the stations are presented as well for comparison.

The impact on the continuous statistics is equivocal (Figure 24 and Figure 25). A clear signal of improvement with increasing station density is evident only for RMSE% and correlation coefficient, for both products. ME increased and RMSE decreased for H03, but an opposite behavior is observed for H15. The station density has no impact on the MAE.

On the contrary, a distinct general improvement with increasing station density is obvious in the categorical statistics (Figure 26 and Figure 27). The station density has a notable impact on the FAR, which is improved substantially with increasing station density. This means that the number

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

All Land Coast Sea

PO

D

POD 25 km

35 km

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

All Land Coast Sea

FAR

FAR 25 km

35 km

0.00

0.05

0.10

0.15

0.20

0.25

0.30

All Land Coast Sea

CSI

CSI25 km

35 km

0.00

0.00

0.00

0.01

0.01

0.01

0.01

0.01

0.02

0.02

0.02

All Land Coast Sea

ETS

ETS25 km

35 km

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

All Land Coast Sea

HK

HK 25 km

35 km

0.0

1.0

2.0

3.0

4.0

5.0

6.0

All Land Coast Sea

Bia

s

Bias 25 km

35 km

of 60


of false alarms of observed rain events is reduced in areas with dense station network. Consequently, this induces a remarkable improvement in the CSI, ETS and bias score.

Figure 24. Continuous validation statistics for the H03 product for the whole period (June 2012 – May 2013) using grid points with different number of stations in a 25 km distance. The default

statistics using all the stations are presented as well for comparison.

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0.0

1 station 2 stations 3 stations >4 stations all

Me

an e

rro

r (m

m/h

)

Mean Error

Class 5

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6


Me

an A

bso

lute

err

or

(mm

/h)


0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0


RM

SE (

mm

/h)

RMSE

0

50

100

150

200

250

300

350

400


RM

SE (

%)

RMSE% Class 5

0

0.04

0.08

0.12

0.16

0.2


Co

rre

lati

on

Co

effi

cie

nt


of 60



`

0.0

0.5

1.0

1.5

2.0

2.5

3.0


Me

an e

rro

r (m

m/h

)

Mean Error Class 5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5


Me

an A

bso

lute

err

or

(mm

/h)


8.4

8.6

8.8

9.0

9.2

9.4

9.6

9.8

10.0

10.2


RM

SE (

mm

/h)

RMSE

0

200

400

600

800

1000

1200

1400

1600

1800

2000


RM

SE (

%)

RMSE% Class 5

0

0.04

0.08

0.12

0.16

0.2


Co

rre

lati

on

Co

effi

cie

nt


of 60


Figure 26. Categorical validation statistics (rain rate threshold = 0.25 mm/h) for the H03 product for the whole period (June 2012 – May 2013) using grid points with different number of stations

in a 25 km distance. The default statistics using all the stations are presented as well for comparison.


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


PO

D, F

AR

, PO

FD

POD, FAR, POFD POD

FAR

POFD

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40


CSI

, ETS

, HK

CSI, ETS, HK CSI

ETS

HK

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5


Bia

s

Bias

90%

91%

92%

93%

94%

95%

96%


Acc

ura

cy

Accuracy

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


PO

D, F

AR

, PO

FD

POD, FAR, POFDPOD

FAR

POFD

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40


CSI

, ETS

, HK

CSI, ETS, HK

CSI

ETS

HK

0.0

1.0

2.0

3.0

4.0

5.0

6.0


Bia

s

Bias

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


Acc

ura

cy

Accuracy

of 60


viii. The impact of elevation

It is well known that the complex orography can be an issue for the precipitation retrieval from satellite for the following reasons (Cattani et al, 2014):

• IR-based retrievals can have problems to identify warm orographic rainfall,

• MW-based retrievals rely on ice scattering over land, which can be moderate in case of warm orographic rain,

• the presence of snow or ice on the ground is a further difficulty for the MW-based retrieval

In order to assess the impact of the grid point elevation on the validation statistics, validation analysis was performed for grid points grouped in different classes of elevation (0 – 500m, 500 -1000m, 1000 – 1500m, 1500 – 2000m, and >2000m). Comparison results are reported in Figure 28 to Figure 29, in which validation statistics for all the land grid points are also shown for comparison.

As expected, gauge precipitation intensity increases with elevation. However, H03 estimates exhibit a remarkable stability which, in general, leads to an increase of all the errors (ME, MAE and RMSE) with elevation. In contrast, the relative RMSE% is decreased since it varies inversely with the observed mean rain rate.

The impact of elevation on the H15 validation statistics is different to that of H03. Mean rain rates estimates are diminished with elevation resulting to a general decrease of all the statistical errors.

A distinct general improvement with elevation is obvious in the categorical statistics (CSI, ETS and bias score) for both products (Figure 30 and Figure 31) mainly due to an improvement of FAR. This means that the number of false alarms of observed rain events is reduced in elevated areas.

of 60


Figure 28. Continuous validation statistics for the H03 product for the whole period (June 2012 – May 2013) using grid points with different elevation. The default statistics using all the stations

over land are presented as well for comparison.

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

Me

an R

ain

(m

m/h

)

Elevation

Mean Rain gaugesH03

Class 5

-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0.0

Me

an e

rro

r (m

m/h

)

Elevation

Mean Error

Class 5

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

Me

an A

bso

lute

err

or

(mm

/h)

Elevation


0.0

0.5

1.0

1.5

2.0

2.5

3.0

RM

SE (

mm

/h)

Elevation

RMSE

Class 5

210

220

230

240

250

260

270

280

290

300

RM

SE (

%)

Elevation

RMSE% Class 5

0

0.04

0.08

0.12

0.16

0.2

Co

rre

lati

on

Co

effi

cie

nt

Elevation


of 60



0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Me

an R

ain

(m

m/h

)

Elevation

Mean Rain

gaugesH15

0.0

0.5

1.0

1.5

2.0

2.5

Me

an e

rro

r (m

m/h

)

Elevation

Mean Error

Class 5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Me

an A

bso

lute

err

or

(mm

/h)

Elevation


0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0

10.0

RM

SE (

mm

/h)

Elevation

RMSE Class 5

0

200

400

600

800

1000

1200

1400

RM

SE (

%)

Elevation

RMSE% Class 5

0

0.04

0.08

0.12

0.16

0.2

Co

rre

lati

on

Co

effi

cie

nt

Elevation


of 60



for the whole period (June 2012 – May 2013) using grid points with different elevation. The default statistics using all the stations over land are presented as well for comparison.


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

PO

D, F

AR

, PO

FD

Elevation

POD, FAR, POFDPOD

FAR

POFD

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

CSI

, ETS

, HK

Elevation

CSI, ETS, HKCSI

ETS

HK

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0

Bia

s

Elevation

Bias

50%55%60%65%

70%75%80%85%90%95%

100%

Acc

ura

cy

Elevation

Accuracy

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

PO

D, F

AR

, PO

FD

Elevation

POD, FAR, POFD PODFARPOFD

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

CSI

, ETS

, HK

Elevation

CSI, ETS, HKCSI

ETS

HK

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

Bia

s

Elevation

Bias

0%10%20%30%

40%50%60%70%80%90%

100%

Acc

ura

cy

Elevation

Accuracy

of 60


ix. The impact of elevation difference

A large elevation difference between grid points and nearby stations may induce large differences in the precipitation rates estimated by the spatial interpolation models over mountainous areas. The impact of the elevation difference in the validation statistics is examined by validating both algorithms for grid points with different mean slopes. For each grid point, the mean slope with all the stations located in a 25 Km distance is calculated according to the equation 2. Grid points are classified to five classes according to the mean slope (0 – 0.05, 0.05 – 0.1, 0.1 – 0.15, 0.15 - 0.2 and >0.2).

Overall, there is no any clear trend of deterioration of the validation statistics with the increase of the mean slope. Only for the H03 product there is a distinct increase of ME, MAE and RMSE, when mean slope is increased from 0.0 to 0.2. The opposite behavior is observed for RMSE% and correlation coefficient. No any explicit conclusion can be drown by the categorical statistics.

Figure 32. Continuous validation statistics for the H03 product for the whole period (June 2012 – May 2013) using grid points with different mean slopes. The default statistics using all the

stations over land are presented as well for comparison.

0.0

0.5

1.0

1.5

2.0

2.5

Me

an R

ain

(m

m/h

)

Mean Slope

Mean Rain gaugesH03

Class 5

-2.0

-1.8

-1.6

-1.4

-1.2

-1.0

-0.8

-0.6

-0.4

-0.2

0.0M

ean

err

or

(mm

/h)

Mean Slope

Mean Error

Class 5

0.0

0.5

1.0

1.5

2.0

2.5

Me

an A

bso

lute

err

or

(mm

/h)

Mean Slope


0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

RM

SE (

mm

/h)

Mean Slope

RMSE

0

50

100

150

200

250

300

350

RM

SE (

%)

Mean Slope

RMSE% Class 5

0

0.04

0.08

0.12

0.16

0.2

Co

rre

lati

on

Co

effi

cie

nt

Mean Slope


of 60



0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Me

an R

ain

(m

m/h

)

Mean Slope

Mean Rain gaugesH15

Class 5

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

Me

an e

rro

r (m

m/h

)

Mean Slope

Mean Error

Class 5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

Me

an A

bso

lute

err

or

(mm

/h)

Mean Slope


0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0

10.0

RM

SE (

mm

/h)

Mean Slope

RMSE

0

200

400

600

800

1000

1200

1400

RM

SE (

%)

Mean Slope

RMSE% Class 5

0.00

0.05

0.10

0.15

0.20

0.25

Co

rre

lati

on

Co

effi

cie

nt

Mean Slope


of 60


Figure 34. Categorical validation statistics for the H03 product for the whole period (June 2012 – May 2013) using grid points with different mean slopes. The default statistics using all the

stations over land are presented as well for comparison.


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0-0.05 0.05-0.1 0.1-0.15 0.15-0.2 ≥ 0.2 land

PO

D, F

AR

, PO

FD

Mean Slope

POD, FAR, POFDPOD

FAR

POFD

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0-0.05 0.05-0.1 0.1-0.15 0.15-0.2 ≥ 0.2 land

CSI

, ETS

, HK

Mean Slope

CSI, ETS, HKCSIETSHK

0.0

0.5

1.0

1.5

2.0

2.5

0-0.05 0.05-0.1 0.1-0.15 0.15-0.2 ≥ 0.2 land

Bia

s

Mean Slope

Bias

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0-0.05 0.05-0.1 0.1-0.15 0.15-0.2 ≥ 0.2 land

Acc

ura

cy

Mean Slope

Accuracy

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0-0.05 0.05-0.1 0.1-0.15 0.15-0.2 ≥ 0.2 land

PO

D, F

AR

, PO

FD

Mean Slope

POD, FAR, POFD POD

FAR

POFD

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0-0.05 0.05-0.1 0.1-0.15 0.15-0.2 ≥ 0.2

CSI

, ETS

, HK

Qindex


0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

0-0.05 0.05-0.1 0.1-0.15 0.15-0.2 ≥ 0.2 land

Bia

s

Qindex

Bias

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0-0.05 0.05-0.1 0.1-0.15 0.15-0.2 ≥ 0.2 land

Acc

ura

cy

Qindex

Accuracy

of 60


x. The impact of the quality of interpolated data

An impact study of the introduction of a quality index of the interpolated data in the validation of the two satellite products over Greece has also carried out. The impact of the quality of the interpolated data on the validation results is examined by calculating validations statistics for grid points with different thresholds on quality index values (≥0, ≥0.2, ≥0.4, ≥0.6, ≥0.8).

The impact on the continuous statistics is opposite to what was expected (Figure 36 and Figure 37). A clear signal of improvement with increasing thresholds on quality index is evident only for RMSE% and correlation coefficient, for both products. All the others errors are slightly increased when threshold on quality index is increased from 0.0 to 0.8.

On the contrary, a distinct general improvement with increasing thresholds on quality index is evident in almost all the categorical statistics (Figure 38 and Figure 39). The quality of the interpolated data has a notable impact on the FAR, which is improved substantially when threshold on quality index is increased from 0.0 to 0.8. This means that the number of false alarms of observed rain events is reduced in areas with good quality of interpolated data. Consequently, this leads to a distinct improvement in the CSI, ETS and bias score.

Figure 36. Continuous validation statistics for the H03 product for the whole period (June 2012 – May 2013) using grid points with different thresholds on quality index values.

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

Me

an R

ain

(m

m/h

)

Qindex

Mean Rain

gaugesH03

Class 5

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0.0

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

Me

an e

rro

r (m

m/h

)

Qindex

Mean Error

Class 5

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

Me

an A

bso

lute

err

or

(mm

/h)

Qindex


2.0

2.1

2.2

2.3

2.4

2.5

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

RM

SE (

mm

/h)

Qindex

RMSEClass 5

0

50

100

150

200

250

300

350

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

RM

SE (

%)

RMSE%Class 5

0.00

0.05

0.10

0.15

0.20

0.25

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

Co

rre

lati

on

Co

effi

cie

nt


of 60



0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

Me

an R

ain

(m

m/h

)

Qindex

Mean Rain

gaugesH15

Class 5

0.0

0.5

1.0

1.5

2.0

2.5

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

Me

an e

rro

r (m

m/h

)

Qindex

Mean Error

Class 5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

Me

an A

bso

lute

err

or

(mm

/h)

Qindex


0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0

10.0

11.0

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

RM

SE (

mm

/h)

Qindex

RMSEClass 5

0

200

400

600

800

1000

1200

1400

1600

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

RM

SE (

%)

RMSE%Class 5

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16

0.18

0.20

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

Co

rre

lati

on

Co

effi

cie

nt


of 60


Figure 38. Categorical validation statistics (rain rate threshold = 0.25 mm/h) for the H03 product for the whole period (June 2012 – May 2013) using grid points with different thresholds on quality index values.


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

PO

D, F

AR

, PO

FD

Qindex

POD, FAR, POFD POD

FAR

POFD

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

CSI

, ETS

, HK

Qindex

CSI, ETS, HK CSIETSHK

0.0

0.5

1.0

1.5

2.0

2.5

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

Bia

s

Qindex

Bias

50%

55%

60%

65%

70%

75%

80%

85%

90%

95%

100%

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

Acc

ura

cy

Qindex

Accuracy

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

PO

D, F

AR

, PO

FD

Qindex

POD, FAR, POFD POD

FAR

POFD

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

CSI

, ETS

, HK

Qindex


0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

Bia

s

Qindex

Bias

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

≥ 0 ≥ 0.2 ≥ 0.4 ≥ 0.6 ≥ 0.8

Acc

ura

cy

Qindex

Accuracy

of 60


xi. One example

Image visualization is an important component of validating any rain estimation algorithm. To gain a visual impression of the performance of the two products, the precipitation rain rate maps obtained by the H03 and H15 products and the rain gauge rain field derived by the default GRISO interpolation model are compared, for one scene from one typical rainy day case over Greece (Figure 40 and 41).

Both rain algorithms reproduce quite well the pattern of the spatial rain distribution. However, the bull eye pattern of the interpolated rain field, resulted by the sparse gauge network in Greece, seems to affect the validation statistics mainly by producing too many false alarm rain events. Moreover, several differences are evident in particular on the Ionian Islands where the precipitation is more intense than forecast and on the central-eastern continental Greece where a false alarm rain event is evident in both products.

The H15 product captures quite well the convective precipitation; however, the redistributed rain is downgraded compared to the initial rainfall-rate.

Better agreement between H03 and gauge interpolated rain maps is observed when we use the more uniform and realistic pattern of the interpolated rain field obtained with the optimum configuration of the GRISO model (35 km radius of influence) (Figure 42).

of 60


Figure 40. Precipitation rain rate map obtained by (a) the H03, and (b) the rain gauge derived rain field using the default GRISO interpolation model, on 25/1/2013, at 11:00 UTC.

of 60



of 60


Figure 42. Precipitation rain rate map obtained by (a) the rain gauge derived rain field using the optimum configuration for GRISO interpolation model (35 km radius of influence), and (b) the H03 (b) on 25/1/2013, at 11:00 UTC.

of 60


5. CONCLUSIONS

The validation activity conducted in this study used in situ gauge observations from a relatively dense network of 233 stations over Greece to determine the reliability of two H-SAF Precipitation Products (PR-OBS-3 and PR-OBS-6), at the product time resolution for a one year period (May 2012 to June 2013).

Before that, one different interpolation technique has been tested (Barnes model), in comparison with the technique adopted by the H-SAF PPVG (GRISO model), in order to check the liability of them over the Greek area. Both interpolation models were validated by removing part of the data (test dataset) and then using the rest of the data (training dataset) to develop the validation model.

It was not found any significant difference between the statistics of the two models except for the mean error which indicated a strong underestimation of rain rates by the GRISO predictions. The large underestimation of rainfall by the GRISO interpolation model can be attributed to the low density of the station network and the “bull eye” pattern of the interpolation surface in combination to the small (25km) radius of influence of each station.

The impact of the radius of influence of each station to the GRISO predictions was pointed out by the sensitivity analysis performed using different radius of influence (25 km, 30km, and 35km). The results showed a clear improved performance of the GRISO interpolation with increasing radius on influence, from the default 25 km radius used in the common validation codes of the PPVG to 30 km and 35 km. The improvement induced by the larger radius of influence results in a more realistic appearance of the spatial distribution of the rain field as a result of a less frequent occurrence of “bull eye” patterns.

The same sensitivity analysis carried out for three different analysis length scale (8km, 20km, and 60km) of the weight function in the Barnes model, showed that validation results are optimized for a 20 km analysis length scale, a little higher than the default length scale of 8 km used in the common validation codes. The optimized Barnes model induced a slight more realistic appearance of the interpolated rain field but with more false predictions over distant areas.

The comparison of these two optimized GRISO and Barnes configurations has not exhibited any significant and distinct difference.

In order to develop a quality index map of the interpolated reference ground data, the impact of the uneven spatial distribution of the rain gauge stations on the quality of the interpolated field was first examined by validating the interpolated data in areas with different station densities. A clear improvement of all the validation statistics was found in areas with increasing density of stations. The impact of the elevation difference between grid points and gauges on the quality of the interpolated data was also investigated. An overall model performance degradation was found as the mean slope among the station increases.

Then, a quality index map of the interpolated reference ground data was constructed as an empirical weighted function of two quality indices: (a) a density quality index which takes into account the number of stations in the vicinity of the grid point, and (b) the orography quality index, which is based on the average slope among the grid point and the stations in its vicinity.

of 60


The continuous validation statistics of the H03 product showed in general high yearly statistical errors with RMSE and MAE corresponding to the 225% and 105% of the mean rain rate, respectively. The percentage RMSE (RMSE%) is also high (around 260%) whereas there is a slight overall tendency for underestimation of the rain gauge rates. The yearly validation results for the H15 product report MAE and RMSE errors that are the double of those found for the H03 product. In contrast to the H03 product, the H15 product significantly overestimates the rain gauge rates.

The poor correlation coefficients found for both algorithms indicate virtually uncorrelated satellite and gauge data. Both algorithms show better performance over land pixels and worst results over coastal and sea pixels.

According to the validation results on a monthly basis, both H03 and H15 products have reached the best performance during the cold period of the year (December to April). Significant overestimation of the rain gauge rates is observed for the H15 rain estimates during autumn.

The multicategorical statistics indicate that the H03 algorithm is able to discriminate efficiently the rain from the no rain events although a large number of rain events are missed. The most prominent feature is the very high FAR (more than 70%), the relatively low POD (less than 40%) and the overestimation of the rainy pixels (about 1.5 to 2 times).

H15 is more effective in classifying high than low rain rates classes. POD scores for the H15 product are high enough (around 83%) whereas FAR and POFD take very large values (more than 70%), an indication of a poor algorithm ability to distinguish between rain and no-rain events. This was expected since, by default, this product is not meant to discriminate the rain from the no rain events. The CSI values indicate a better performance of the H15 algorithm compared to the H03. The H15 algorithm overestimates the rain events (about 3 times).

As for the continuous statistics, both algorithms perform better in discriminating rain from no rain events for cold months and over land pixels.

In general, seasonal comparison shows that errors are lower for cold months than in the summer months, probably due to phenomena of lower intensity and inhomogeneity in winter. Another finding is that the RMSE% obtains very large values, especially for the H15 product. At this point, it has to be noted that the RMSE does not depend only by the error due to satellite estimation but also by the errors due to the rain gauges in ground and to the upscaling/downscaling and interpolation process. With regard to RMSE comparison is has been shown that for all the interpolation methods RMSE increases with distance of interpolated data (Petracca, 2011). The relatively sparse and inhomogeneous rain gauge network in Greece seems to have a significant contribution to the large RMSE% values found in this study.

A comparison of the algorithm validation statistics using the default and the optimum configuration of the GRISO interpolation model has shown a slight improvement in both algorithms which, in general, is more pronounced for H15. This indicates the impact the interpolation model has in the validation exercises of the two rain products over Greece.

The impact of the uneven rain gauge station spatial distribution, the elevation and the difference in elevation on the validation results was also examined. The impact of the uneven rain gauge station spatial distribution on the continuous statistics is equivocal. A clear signal of improvement with increasing station density is evident only for RMSE% and correlation coefficient, for both products. On the contrary, a distinct general improvement with increasing

of 60


station density is clear in the categorical statistics, induced mainly by the substantially improvement of FAR.

An increase of all the errors (ME, MAE and RMSE) with elevation was found for the H03 product, whereas the opposite behavior was observed for the H15 algorithm. A distinct general improvement with elevation was evident in the categorical statistics for both products mainly due to an improvement of FAR.

In general, there is no any clear trend of deterioration of the validation statistics with the increase of the mean slope of terrain between the grid point and the gauges. Only for the H03 product it was found a distinct increase of continuous errors ME, MAE and RMSE. No any explicit conclusion can be drown by the categorical statistics

A preliminary impact study of the introduction of quality index of interpolated data in the validation of the two satellite products has pointed out that the introduction of this quality information as a filter had a substantial impact only in the categorical statistics by reducing the number of false alarms of observed rain events in areas with good quality of interpolated data. Concerning the continuous statistics, a clear signal of improvement with increasing thresholds on quality index is evident only for RMSE% and correlation coefficient.

of 60


References

Cattani E., A. Merino, V. Levizzani (2014) Analysis of satellite monthly precipitation time series over East Africa. 7th International Scientific Conference on the Global Water and Energy Cycle, At World Forum, The Hague, The Netherlands.

Huffman GJ, Adler RF, Arkin P, Chang A, Ferraro R, Gruber A, Janowiak J, McNab A, Rudolph B, Schneider U (1997) The Global Precipitation Climatology Project (GPCP) combined precipitation dataset. Bul Amer Meteor Soc 78:5–20.

Feidas H. (2006). Validating three infrared-based rainfall retrieval algorithms for intense convective activity over Greece. International Journal of Remote Sensing, 27: 2787-2812.

Feidas H. (2010). Validation of satellite rainfall products over Greece. Theoretical and Applied Climatology, 99(1): 193-216.

Feidas H., Th. Kontos, N. Soulakellis, K. Lagouvardos (2007). A GIS tool for the evaluation of the precipitation forecasts of a numerical weather prediction model using satellite data. Computers and Geosciences, 33: 989-1007.

Feidas H., G. Kokolatos, A. Negri, M. Manyin, N. Chrysoulakis, Y. Kamarianakis (2009). Validation of an infrared-based satellite algorithm to estimate accumulated rainfall over the Mediterranean basin. Theoretical and Applied Climatology, 95: 91-109.

Kamarianakis Y., H. Feidas, G. Kokolatos, N. Chrysoulakis, V. Karatzias (2008). Evaluating remotely sensed rainfall estimates using nonlinear mixed models and geographically weighted regression. Environmental Modeling and Software. 23(12): 1438-1447.

Feidas H., K. Lagouvardos, V. Kotroni and C. Cartalis (2005). Application of three Satellite Techniques in Support of Precipitation Forecasts of a NWP Model. International Journal of Remote Sensing, 24: 5393-5417.

Kummerow C, Simpson J, Thiele O, Barnes W, Chang ATC, Stocker E, Adler RF, Hou A (2000) The status of the tropical rainfall measuring mission (TRMM) after two years in orbit. J Appl Meteor 39:1965–1982.

Nurmi, P. (2003) Recommendations on the verification of local weather forecasts. 11 ECMWF Tech. Memo. N. 430, 19pp

Pignone F., Rebora N., Silvestro F. and Castelli F. (2010) GRISO (Generatore Random di Interpolazioni Spaziali da Osservazioni incerte)-Piogge, Relazione delle attività del I anno inerente la Convenzione 778/2009 tra Dipartimento di Protezione Civile e Fondazione CIMA (Centro Internazionale in Monitoraggio Ambientale), report n° 272/2010, pp 353, 2010.

Petracca M. (2011) Evaluation on accuracy of precipitation data. Final Report on the Visiting Scientist Activities. H-SAF Program (http://hsaf.meteoam.it/documents/reference/ Final_Report_VS_accuracy-2.pdf)

Puca, S., Porcu, F., Rinollo, A., Vulpiani, G., Baguis, P., Balabanova, S., Campione, E., Ertürk, A., Gabellani, S., Iwanski, R., Jurašek, M., Kaňák, J., Kerényi, J., Koshinchanov, G., Kozinarova, G., Krahe, P., Lapeta, B., Lábó, E., Milani, L., Okon, L'., Öztopal, A., Pagliara, P., Pignone, F., Rachimow, C., Rebora, N., Roulin, E., Sönmez, I., Toniazzo, A., Biron, D., Casella, D., Cattani, E., Dietrich, S., Di Paola, F., Laviola, S., Levizzani, V., Melfi, D., Mugnai, A., Panegrossi, G., Petracca, M., Sanò, P., Zauli, F., Rosci, P., De Leonibus, L., Agosta, E., and Gattari, F. (2014) The

of 60


validation service of the hydrological SAF geostationary and polar satellite precipitation products, Nat. Hazards Earth Syst. Sci., 14, 871-889, doi:10.5194/nhess-14-871-2014, 2014.

“validation of the h-saf precipitation...

Documents