copernicus global land operations...figure 14: maximum number of consecutive days wrongly classified...
TRANSCRIPT
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Copernicus Global Land Operations
“Vegetation and Energy” ”CGLOPS-1”
Framework Service Contract N° 199494 (JRC)
SCIENTIFIC QUALITY EVALUATION 2019
SOIL WATER INDEX
VERSION 3.0
Issue I1.00
Organization name of lead contractor for this deliverable: TU Wien
Book Captain: Bernhard Bauer-Marschallinger
Contributing Authors: Tobias Stachl
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 2 of 78
Dissemination Level PU Public X
PP Restricted to other programme participants (including the Commission Services)
RE Restricted to a group specified by the consortium (including the Commission Services)
CO Confidential, only for members of the consortium (including the Commission Services)
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 3 of 78
Document Release Sheet
Book captain: Bernhard Bauer-Marschallinger
Sign Date 25.03.2019
Approval: Roselyne Lacaze Sign Date 03.04.2020
Endorsement: Michael Cherlet Sign Date
Distribution: Public
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 4 of 78
Change Record
Issue/Rev Date Page(s) Description of Change Release
25.03.2020 All First issue I1.00
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 5 of 78
TABLE OF CONTENTS
Executive Summary .................................................................................................................. 16
1 Background of the document ............................................................................................. 18
1.1 Scope and Objectives............................................................................................................. 18
1.2 Content of the document....................................................................................................... 18
1.3 Related documents ............................................................................................................... 18
1.3.1 Applicable documents ................................................................................................................................ 18
1.3.2 Input ............................................................................................................................................................ 18
1.3.3 Output ......................................................................................................................................................... 19
2 Review of Users Requirements ........................................................................................... 20
3 Review of the SWI quality .................................................................................................. 22
4 Scientific Quality Evaluation Method ................................................................................. 24
4.1 Global Analysis ...................................................................................................................... 24
4.2 Regional analysis ................................................................................................................... 26
4.3 Model Reference Products ..................................................................................................... 27
4.4 In-situ Reference Products ..................................................................................................... 28
4.5 Other Reference Products ...................................................................................................... 30
4.6 Data Availability .................................................................................................................... 31
4.7 Layer Comparison Matrix ....................................................................................................... 31
5 Results .............................................................................................................................. 32
5.1 Global analysis ...................................................................................................................... 32
5.1.1 Comparison with GLDAS Noah .................................................................................................................... 32
5.1.2 Comparison with ISMN ............................................................................................................................... 48
5.2 Regional Analysis .................................................................................................................. 62
5.2.1 Situation in southern Africa in 2019 ........................................................................................................... 62
5.2.2 Zambia ........................................................................................................................................................ 63
5.2.3 Australia ...................................................................................................................................................... 65
5.2.4 Europe ......................................................................................................................................................... 67
5.2.5 Spain ........................................................................................................................................................... 69
6 Conclusions ....................................................................................................................... 71
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 6 of 78
6.1 Global Analysis ...................................................................................................................... 71
6.2 Regional analysis ................................................................................................................... 73
7 Recommendations ............................................................................................................. 74
8 References ........................................................................................................................ 75
8.1 Scientific Literature ............................................................................................................... 75
8.2 News Footage ....................................................................................................................... 76
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 7 of 78
List of Figures
Figure 1: Map showing the regions used to define the regions of special interest for the plots of R
over time. ............................................................................................................................... 25
Figure 2: Map of used ISMN networks, showing all stations with data in the period 2007-2019. ... 29
Figure 3: Pearson’s correlation coefficient between SWI T=1 and GLDAS Noah, as well the
difference between the reference period (2007-01-01 until 2018-12-31) and the current
validation period (2019-01-01 until 2019-12-31). Green colours indicate performance
improvements in the current period. In the difference map only the statistically significant
differences are shown whereas the violin plots show all data points. The violin plots show the
estimated kernel density plot as well as the median (dashed line) and the lower and upper
quartile (dotted lines). ............................................................................................................ 33
Figure 4: Pearson’s correlation coefficient between SWI T=20 and GLDAS Noah, as well the
difference between the reference period (2007-01-01 until 2018-12-31) and the current
validation period (2019-01-01 until 2019-12-31). Green colours indicate performance
improvements in the current period. In the difference map only the statistically significant
differences are shown whereas the violin plots show all data points. The violin plots show the
estimated kernel density plot as well as the median (dashed line) and the lower and upper
quartile (dotted lines). ............................................................................................................ 34
Figure 5: Pearson’s correlation coefficient between SWI T=100 and GLDAS Noah, as well the
difference between the reference period (2007-01-01 until 2018-12-31) and the current
validation period (2019-01-01 until 2019-12-31). Green colours indicate performance
improvements in the current period. In the difference map only the statistically significant
differences are shown whereas the violin plots show all data points. The violin plots show the
estimated kernel density plot as well as the median (dashed line) and the lower and upper
quartile (dotted lines). ............................................................................................................ 35
Figure 6: Average 3-month Pearson’s correlation coefficient R between SWI T=5 and GLDAS
Noah, for the reference period (2007-01-01 until 2018-12-31, green line) and the current
validation period (2019-01-01 until 2019-12-31, purple line) over various areas. The standard
deviation for the reference is shown as light green surface. ................................................... 36
Figure 7: RMSD between SWI T=1 and GLDAS Noah, as well the difference between the
reference period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01
until 2019-12-31). Green colours indicate performance improvements in the current period. In
the difference map only the statistically significant differences are shown whereas the violin
plots show all data points. The violin plots show the estimated kernel density plot as well as
the median (dashed line) and the lower and upper quartile (dotted lines). .............................. 38
Figure 8: RMSD between SWI T=20 and GLDAS Noah, as well the difference between the
reference period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 8 of 78
until 2019-12-31). Green colours indicate performance improvements in the current period. In
the difference map only the statistically significant differences are shown whereas the violin
plots show all data points. The violin plots show the estimated kernel density plot as well as
the median (dashed line) and the lower and upper quartile (dotted lines). .............................. 39
Figure 9: RMSD between SWI T=100 and GLDAS Noah, as well the difference between the
reference period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01
until 2019-12-31). Green colours indicate performance improvements in the current period. In
the difference map only the statistically significant differences are shown whereas the violin
plots show all data points. The violin plots show the estimated kernel density plot as well as
the median (dashed line) and the lower and upper quartile (dotted lines). .............................. 40
Figure 10: Bias between SWI T=1 and GLDAS Noah, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-
12-31). Green colours indicate performance improvements in the current period. In the
difference map only the statistically significant differences are shown whereas the violin plots
show all data points. The violin plots show the estimated kernel density plot as well as the
median (dashed line) and the lower and upper quartile (dotted lines). ................................... 41
Figure 11: Bias between SWI T=20 and GLDAS Noah, as well the difference between the
reference period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01
until 2019-12-31). Green colours indicate performance improvements in the current period. In
the difference map only the statistically significant differences are shown whereas the violin
plots show all data points. The violin plots show the estimated kernel density plot as well as
the median (dashed line) and the lower and upper quartile (dotted lines). .............................. 42
Figure 12: Bias between SWI T=100 and GLDAS Noah, as well the difference between the
reference period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01
until 2019-12-31). Green colours indicate performance improvements in the current period. In
the difference map only the statistically significant differences are shown whereas the violin
plots show all data points. The violin plots show the estimated kernel density plot as well as
the median (dashed line) and the lower and upper quartile (dotted lines). .............................. 43
Figure 13: Percent of correctly classified freeze/thaw states compared to GLDAS Noah soil
temperature as well the difference between the reference period (2007-01-01 until 2018-12-
31) and the current validation period (2019-01-01 until 2019-12-31). Green colours indicate
performance improvements in the current period. In the difference map only the statistically
significant differences are shown whereas the violin plots show all data points. The violin plots
show the estimated kernel density plot as well as the median (dashed line) and the lower and
upper quartile (dotted lines). .................................................................................................. 45
Figure 14: Maximum number of consecutive days wrongly classified when comparing the SSF
against GLDAS Noah soil temperature as well the difference between the reference period
(2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31).
Green colours indicate performance improvements in the current period. In the difference map
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 9 of 78
only the statistically significant differences are shown whereas the violin plots show all data
points. The violin plots show the estimated kernel density plot as well as the median (dashed
line) and the lower and upper quartile (dotted lines). .............................................................. 46
Figure 15: Mean number of consecutive days wrongly classified when comparing the SSF against
GLDAS Noah soil temperature as well the difference between the reference period (2007-01-
01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31). Green
colours indicate performance improvements in the current period. In the difference map only
the statistically significant differences are shown whereas the violin plots show all data points.
The violin plots show the estimated kernel density plot as well as the median (dashed line) and
the lower and upper quartile (dotted lines). ............................................................................ 47
Figure 16: Pearson’s correlation coefficient between SWI T=1 and in-situ data, as well the
difference between the reference period (2007-01-01 until 2018-12-31) and the current
validation period (2019-01-01 until 2019-12-31). Green colours indicate performance
improvements in the current period. In the difference map only the statistically significant
differences are shown whereas the violin plots show all data points. The violin plots show the
estimated kernel density plot as well as the median (dashed line) and the lower and upper
quartile (dotted lines). ............................................................................................................ 49
Figure 17: Pearson’s correlation coefficient between SWI T=20 and in-situ data, as well the
difference between the reference period (2007-01-01 until 2018-12-31) and the current
validation period (2019-01-01 until 2019-12-31). Green colours indicate performance
improvements in the current period. In the difference map only the statistically significant
differences are shown whereas the violin plots show all data points. The violin plots show the
estimated kernel density plot as well as the median (dashed line) and the lower and upper
quartile (dotted lines). ............................................................................................................ 50
Figure 18: Pearson’s correlation coefficient between SWI T=100 and in-situ data, as well the
difference between the reference period (2007-01-01 until 2018-12-31) and the current
validation period (2019-01-01 until 2019-12-31). Green colours indicate performance
improvements in the current period. In the difference map only the statistically significant
differences are shown whereas the violin plots show all data points. The violin plots show the
estimated kernel density plot as well as the median (dashed line) and the lower and upper
quartile (dotted lines). ............................................................................................................ 51
Figure 19: RMSD between SWI T=1 and in-situ data, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-
12-31). Green colours indicate performance improvements in the current period. In the
difference map only the statistically significant differences are shown whereas the violin plots
show all data points. The violin plots show the estimated kernel density plot as well as the
median (dashed line) and the lower and upper quartile (dotted lines). ................................... 52
Figure 20: RMSD between SWI T=20 and in-situ data, as well the difference between the
reference period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 10 of 78
until 2019-12-31). Green colours indicate performance improvements in the current period. In
the difference map only the statistically significant differences are shown whereas the violin
plots show all data points. The violin plots show the estimated kernel density plot as well as
the median (dashed line) and the lower and upper quartile (dotted lines). .............................. 53
Figure 21: RMSD between SWI T=100 and in-situ data, as well the difference between the
reference period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01
until 2019-12-31). Green colours indicate performance improvements in the current period. In
the difference map only the statistically significant differences are shown whereas the violin
plots show all data points. The violin plots show the estimated kernel density plot as well as
the median (dashed line) and the lower and upper quartile (dotted lines). .............................. 54
Figure 22: Bias between SWI T=1 and in-situ data, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-
12-31). Green colours indicate performance improvements in the current period. In the
difference map only the statistically significant differences are shown whereas the violin plots
show all data points. The violin plots show the estimated kernel density plot as well as the
median (dashed line) and the lower and upper quartile (dotted lines). ................................... 55
Figure 23: Bias between SWI T=20 and in-situ data, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-
12-31). Green colours indicate performance improvements in the current period. In the
difference map only the statistically significant differences are shown whereas the violin plots
show all data points. The violin plots show the estimated kernel density plot as well as the
median (dashed line) and the lower and upper quartile (dotted lines). ................................... 56
Figure 24: Bias between SWI T=100 and in-situ data, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-
12-31). Green colours indicate performance improvements in the current period. In the
difference map only the statistically significant differences are shown whereas the violin plots
show all data points. The violin plots show the estimated kernel density plot as well as the
median (dashed line) and the lower and upper quartile (dotted lines). ................................... 57
Figure 25: SWI and in-situ data for SNOTEL-station CRAB CREEK ............................................. 59
Figure 26: SWI and in-situ data for USCRN-station Edinburg-17-NNE .......................................... 59
Figure 27: SWI and in-situ data for USCRN-station Mercury 3 SSW ............................................. 60
Figure 28: SWI and in-situ data for SCAN-station Pine Nut ........................................................... 60
Figure 29: SWI and in-situ data for RSMN-station Bacles ............................................................. 60
Figure 30: SWI and in-situ data for TERENO-station Gevenich ..................................................... 61
Figure 31: As Figure 30, but for SWI with T-value=100 ................................................................. 61
Figure 32: Monthly SWI anomalies over southern Africa in 2019. ................................................. 63
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 11 of 78
Figure 33: Monthly SWI anomalies (top) and Vegetation Health Index (VHI) data (bottom) over
Zambia in 2019. ..................................................................................................................... 64
Figure 34: Monthly SWI anomalies (top) and Vegetation Health Index (VHI) data (bottom) over
Australia in 2019. ................................................................................................................... 66
Figure 35: Monthly SWI anomalies over Europe in 2019. .............................................................. 68
Figure 36: Monthly SWI anomalies (top) and ECMWF rainfall anomalies (bottom, provided by FAO
(via the GIEWS) over Spain in 2019. ..................................................................................... 70
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 12 of 78
List of Tables
Table 1: Target requirements of GCOS for global near-surface soil moisture (up to 5cm soil depth)
as Essential Climate Variable (GCOS-154, 2011) .................................................................. 21
Table 2: SQE 2018: Compliance matrix for the comparison with GLDAS Noah. Percentages of grid
points that fulfill each requirement. Left the referenceperiod (2007-01-01 until 2017-12-31) /
right the validation period (2018-01-01 until 2018-12-31) ....................................................... 22
Table 3: SQE 2018: Compliance matrix for the comparison with ISMN in-situ data. Percentages of
grid points that fulfill each requirement. Left the referenceperiod (2007-01-01 until 2017-12-31)
/ right the validation period (2018-01-01 until 2018-12-31). .................................................... 23
Table 4: Possible values of the SSF and their meaning ................................................................ 26
Table 5: Classification scheme ...................................................................................................... 26
Table 6: Data availability and references for the used in-situ networks. ........................................ 30
Table 7: Available datasets for validation ...................................................................................... 31
Table 8: Which T-values were compared to which layer of the modelled and in-situ datasets ....... 31
Table 9: Mean / median R between SWI and GLDAS Noah as well as percentiles of the absolute
values of the differences. All values in m3/m3. ........................................................................ 35
Table 10: Mean / median RMSD between SWI and GLDAS Noah as well as percentiles of the
absolute values of the differences. All values in m3/m3. .......................................................... 40
Table 11: Mean / median Bias between SWI and GLDAS Noah as well as percentiles of the
absolute values of the differences. All values in m3/m3 ........................................................... 43
Table 12: Mean / median metrics of the SSF validation with GLDAS Noah as well as percentiles of
the absolute values of the differences. ................................................................................... 48
Table 13: Mean / median correlation coefficient between SWI and in-situ data as well as
percentiles of the absolute values of the significant differences ............................................. 51
Table 14: Mean / median RMSD between SWI and in-situ observations as well as percentiles of
the absolute values of the differences. All values in m3/m3. .................................................... 54
Table 15: Mean / median bias between SWI and in-situ observations as well as percentiles of the
absolute values of the differences. All values in m3/m3. .......................................................... 57
Table 16: Station details and climate classifications ...................................................................... 58
Table 17: Pearson’s correlation coefficient (R) and Root mean square deviation (RMSD) for the
selected in-situ stations. SWI T=1 was compared to in-situ soil moisture measured at a depth
of 0.05m. ................................................................................................................................ 59
Table 18: Compliance matrix for the comparison with GLDAS Noah and ISMN stations.
Percentages of grid points that fulfill the GCOS requirement of 0.04 m³/m³ for the reference
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 13 of 78
period (2007-07-01 until 2018-12-31) (left) / for the current validation period (2019-01-01 until
2019-12-31) (right). ................................................................................................................ 71
Table 19: Compliance matrix for the comparison with GLDAS Noah. Percentages of grid points that
fulfill each requirement. Left the reference period (2007-01-01 until 2018-12-31) / right the
current validation period (2019-01-01 until 2019-12-31). ........................................................ 72
Table 20: Compliance matrix for the comparison with ISMN in-situ data. Percentages of grid points
that fulfill each requirement. Left the reference period (2007-01-01 until 2018-12-31) / right the
current validation period (2019-01-01 until 2019-12-31). ........................................................ 72
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 14 of 78
List of Acronyms
AD : Applicable document
ATBD : Algorithmic Theoretical Basis Document
BIEBRAZA : In-Situ Network of the Instytut Geodezji i Kartografii, Poland.
CDF : Cumulative Distribution Function
CGLS : Copernicus Global Land service
COSMOS : The COsmic-ray Soil Moisture Observing System
CUR : Current validation period (i.e. 2019)
ECMWF : European Centre for Medium-Range Weather Forecasts
EFFIS : European Forest Fire Information System
ERA-Interim : European Reanalysis – Interim
EUMETSAT : European Organisation for the Exploitation of Meteorological Satellites
FAO : Food and Agriculture Organization of the United Nations
FMI : Finnish Meteorological Institute
GCOS : Global Climate Observing System
GIEWS : Global Information and Early Warning System (of FAO)
GLDAS : Global Land Data Assimilation System
GROW : In-Situ Network of the GROW Observation Network
HOBE : In-Situ Network of the Center for Hydrology – Hydrological Observatory
IFAD : International Fund for Agricultural Development
ISMN : International Soil Moisture Network
JRC : Joint Research Centre of the European Union
LSA-SAF : Land Surface Analysis Satellite Application Facility
MERRA : Modern-Era Retrospective Analysis for Research and Applications
Noah-LSM : Noah Land Surface Model
NASA : National Aeronautics and Space Administration
NF : News Footage
NRT : Near Real Time
OZNET : In-Situ Network of the University of Melbourne, Australia
PUM : Product User Manual
R : Spearman correlation coefficient (R)
REF : Reference validation period (i.e. 2007-2018)
RHEMEDUS : In-Situ Network of the Centro Hispano Luso de Investigaciones Agrarias, Salamanca, Spain
RMSD : Root Mean Square Difference
RSMN : In-Situ Network of the National Meteorological Administration, Romania
SCAN : Soil Climate Analysis Network
SMAP : Soil Moisture Active Passive
SNOTEL : In-Situ Network of the Snow Telemetry
SQE : Science and Quality Evaluation
SSF : Surface State Flag
SVP : Service Validation Plan
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 15 of 78
SWI : Soil Water Index
SWI10 : Soil Water Index 10-daily average
SWI-TS : Soil Water Index Time Series
TCI : Temperature Condition Index
TERENO : In-Situ Network of the Terrestrial Environmental Observatories
TUG : Technical User Group
UNFCCC : United Nations Framework – Convention on Climate Change
USCRN : U.S. Climate Reference Network
VCI : Vegetation Condition Index
VHI : Vegetation Health Index
WEGENERNET : In-Situ Network of the Wegener Center for Climate and Global Change, Graz, Austria
WFP : World Food Program
WGS : World Geodetic System
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 16 of 78
EXECUTIVE SUMMARY
The Copernicus Global Land Service (CGLS) is earmarked as a component of the Land service to
operate “a multi-purpose service component” that provides a series of bio-geophysical products on
the status and evolution of land surface at global scale. Production and delivery of the parameters
take place in a timely manner and are complemented by the constitution of long-term time series.
This document describes the Science and Quality Evaluation (SQE) of the Soil Water Index (SWI)
products for the year 2019. A global and a regional analysis were performed. The aim of the global
analysis in this report is to assess the performance of the SWI V3 NRT product in the period from
2019-01-01 until 2019-12-31 relative to the performance of the SWI archive available from the
Copernicus Global Land Service (CGLS) of the previous years, starting in 2007. The validation
presented in this report was performed according to the procedures defined in the Service
Validation Plan (SVP). The regional analysis focuses first on the continental-scale drought
conditions followed by unprecedented bushfire catastrophe in Australia, and severe drought
vegetation stress in central southern Africa, most notably in Zambia. Another focus is over Europe,
where, a second-in-a-row summer heat wave leaded in 2019 to extreme dry condition over large
parts of the continent. Distinguished rainfall patterns over Spain at the end of the year are also
documented.
For the global analysis, the SWI and Surface State Flag (SSF) were compared to the output of a
land surface model as well as in-situ data. The used model was GLDAS-Noah V2.1, whereas the
in-situ data was taken from the International Soil Moisture Network (ISMN). The SWI was
compared to different soil moisture layers of these reference datasets, whereas the SSF was
assessed using the first soil temperature layer of GLDAS. The compliance matrices in Table 19
and
Table 20 show very similar results to those from previous yearly evaluations (Table 2 and Table 3),
documenting a stable performance of the SWI product. Also the spatial patterns of agreement with
GLDAS for SWI and SSF are widely consistent with the previous analysis, with a weak
performance over tropical rainforests, subarctic areas, and deserts, and a good performance over
Europe, subtropical Africa, India, central and eastern Asia, non-arid Australia and the (non-polar)
Americas. Overall, the quality of the SWI and SSF products appears unchanged, with some
regional differences in both positive and negative direction.
In the in-situ analysis, overall scores are slightly reduced compared to last year and few less
stations achieve optimal and target quality when compared to the last SQE exercise, which itself
reported slightly better results than in the preceding year. As the magnitude of these changes is
little and in the range of previous years, a year-to-year fluctuation can be attested, with no
significant long term trend apparent. Also, a repeated analysis of selected in-situ time series
showed that over one station in Germany a distinct time lag may correspond with an inappropriate
attribution of T-value to soil depth.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 17 of 78
In the regional analyses over southern Africa, Australia, and Europe, monthly SWI anomalies for
2019 did highly agree with news footage and reference vegetation and rainfall datasets. Drought
conditions over central section of subtropical southern Africa from January to May are captured by
the SWI anomalies, as well as the abundant precipitation sums over Tanzania in late 2019. The
record-hitting drought and bushfire series of 2019 in Australia are well captured by the SWI
anomalies, as well as an intermission with heavy rain falls in Queensland. Finally, the SWI
dynamics recorded Europe dry anomalies over large parts of the continent, which was hit by an
exceptional hot spring and summer in 2019. Hydrological dynamics in Hungary and Spain are well
reflected. However, the overall magnitude of the moisture deficit for the continent is lower in the
SWI datasets as one would expect from reference datasets and media reports. Probably, this is
due to persisting dry conditions in the previous years, and the limited reference timeline for building
the anomalies (starting in 2007).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 18 of 78
1 BACKGROUND OF THE DOCUMENT
1.1 SCOPE AND OBJECTIVES
The document presents the results of the annual scientific quality evaluation of the operational SWI
V3.0 product.
The quality evaluation is performed on SWI data for the year from 2019-01-01 to 2019-12-31 with a
reference period from 2007-01-01 to 2018-12-31.
The objective is to check that the operational products keep the same level of quality in the period
under study than the fully validated products.
1.2 CONTENT OF THE DOCUMENT
This document is structured as follows:
Chapter 2 recalls the users requirements, and the expected performance
Chapter 3 reviews the results of previous validation reports
Chapter 4 describes the methodology for quality assessment, the metrics and the criteria of
evaluation
Chapter 5 presents the results of the analysis
Chapter 6 summarizes the main conclusions of the study
Chapter 7 makes recommendations based upon the results
1.3 RELATED DOCUMENTS
1.3.1 Applicable documents
AD1: Annex I – Technical Specifications JRC/IPR/2015/H.5/0026/OC to Contract Notice 2015/S
151-277962 of 7th August 2015
AD2: Appendix 1 – Copernicus Global land Component Product and Service Detailed Technical
requirements to Technical Annex to Contract Notice 2015/S 151-277962 of 7th August 2015
AD3: GIO Copernicus Global Land – Technical User Group – Service Specification and Product
Requirements Proposal – SPB-GIO-3017-TUG-SS-004 – Issue I1.0 – 26 May 2015.
1.3.2 Input
Document ID Descriptor
CGLOPS1_SSD Service Specifications of the Global Component of
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 19 of 78
the Copernicus Land Service.
CGLOPS1_SVP Service Validation Plan of the Global Land Service
CGLOPS1_ATBD_SWIV3 Algorithm Theoretical Basis Document of the SWI
V3.0 product
1.3.3 Output
Document ID Descriptor
CGLOPS1_PUM_ SWIV3-SWI10-
SWI-TS
Product User Manual summarizing all information about the
SWI V3.0, SWI10 and SWI-TS products.
CGLOPS1_SQE2018_SWIV3 Scientific Quality Evaluation report containing the results of
the scientific validation of the SWI V3.0 for 2018.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 20 of 78
2 REVIEW OF USERS REQUIREMENTS
According to the applicable document [AD2] and [AD3], the user’s requirements relevant for the
Soil Water Index are:
Definition: Amount of water (m3/m3) contained in soil layers identified according to their
depth measured from the top surface.
Geometric properties:
o Pixel size shall be defined on a per-product basis so as to facilitate the multi-
parameter analysis and exploitation
o The target baseline location accuracy shall be 1/3 of the at-nadir instantaneous field
of view.
o Pixel co-ordinates shall be given for the centre of the pixel.
Geographical coverage:
o Geographic projection: regular lat-long
o Geodetical datum: WGS84
o Pixel accuracy: minimum 10 digits
o Coordinate position: centre of pixel
o Window coordinates:
Upper Left:180°W-74°N
Bottom Right: 180°E 56°S
Ancillary information:
o the number of measurements per pixel used to generate the synthesis product
o the per-pixel date of the individual measurements or the start-end dates of the
period actually covered
o quality indicators, with explicit per-pixel identification of the cause of anomalous
parameter result
Ancillary information:
o the number of measurements per pixel used to generate the synthesis product
o the per-pixel date of the individual measurements or the start-end dates of the
period actually covered
o quality indicators, with explicit per-pixel identification of the cause of anomalous
parameter result
Accuracy requirements: wherever applicable the bio-geophysical parameters should meet
the internationally agreed accuracy standards laid down in document “Systematic
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 21 of 78
Observation Requirements for Satellite-Based Products for Climate”. Supplemental details
to the satellite based component of the “Implementation Plan for the Global Observing
System for Climate in Support of the UNFCCC”. GCOS-#154, 2011” (Table 1).
Table 1: Target requirements of GCOS for global near-surface soil moisture (up to 5cm soil depth) as
Essential Climate Variable (GCOS-154, 2011)
Variable Horizontal
Resolution
Temporal
resolution Accuracy Stability
Volumetric soil moisture 50km Daily 0.04 m3/m3 0.01m3/m3/year
GCOS notes that the targets above “are set as an accuracy of about 10 per cent of saturated
content and stability of about 2 per cent of saturated content. These values are judged adequate
for regional impact and adaptation studies and verification and development of climate models. It is
considered premature to consider global scale changes.” It adds that “stating a general accuracy
requirement is difficult for this type of observation, as this depends not only on soil type but also on
soil moisture content itself. The stated numbers thus should be viewed with some caution”.
We use the target GCOS requirement, set to 0.04m³/m³, as optimal goal in the compliance matrix,
which has to be taken as an upper bound.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 22 of 78
3 REVIEW OF THE SWI QUALITY
In the continuous evaluation process of the CGLS, the scientific quality of the SWI V3.0 2018
products was assessed in the last yearly cycle [see CGLOPS1_SQE2018_SWIV3]. A summary of
the results related to the user requirements is presented below.
In the SQE 2018 exercise, the SWI V3.0 product was compared to data from the Global Land Data
Assimilation System (GLDAS) and to in-situ data from the International Soil Moisture Network
(ISMN). Table 2 to Table 3 show the compliance matrices for 2018 products compared to products
for the reference time period between 2007 and 2017. It can be seen that, for e.g. GLDAS, 6 to 60
percent of the points reach for the SWI the optimal quality depending on the validation period and
the T-value.
Table 2: SQE 2018: Compliance matrix for the comparison with GLDAS Noah. Percentages of grid
points that fulfill each requirement. Left the referenceperiod (2007-01-01 until 2017-12-31) / right the
validation period (2018-01-01 until 2018-12-31)
Variable Threshold Target GCOS/optimal
Limit (Upper Bound) SWI 0.20 m
3/m
3
SSF 7 days
SWI 0.10 m3/m
3
SSF 3 days
SWI 0.04 m3/m
3
SSF 1 day
[%] REF / CUR REF / CUR REF / CUR
SWI T=1 100 / 100 80 / 86 10 / 30
SWI T=20 100 / 100 75 / 81 6 / 17
SWI T=100 100 / 100 91 / 96 33 / 60
SSF consecutive days
wrongly classified (max) 13 / 22 6 / 6 1 / 1
SSF consecutive days
wrongly classified (median) 74 / 82 21 / 50 1 / 8
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 23 of 78
Table 3: SQE 2018: Compliance matrix for the comparison with ISMN in-situ data. Percentages of grid
points that fulfill each requirement. Left the referenceperiod (2007-01-01 until 2017-12-31) / right the
validation period (2018-01-01 until 2018-12-31).
Variable Threshold Target GCOS/optimal
Limit (Upper Bound) SWI 0.20 m3/m
3 SWI 0.10 m
3/m
3 SWI 0.04 m
3/m
3
[%] REF / CUR REF / CUR REF / CUR
SWI T=1 96 / 98 58 / 70 6 / 13
SWI T=20 96 / 98 58 / 70 6 / 13
SWI T=100 97 / 97 70 / 81 12 / 34
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 24 of 78
4 SCIENTIFIC QUALITY EVALUATION METHOD
4.1 GLOBAL ANALYSIS
The aim of the global analysis in this report is to assess the performance of the SWI V3 NRT
product in the period from 2019-01-01 until 2019-12-31, relative to the performance of the SWI
archive available from the Copernicus Land Service of the previous years (starting in 2007). The
validation presented in this report was performed according to the procedures defined in the
Service Validation Plan [CGLOPS1_SVP], but are also shortly summarized here.
For the SWI validation, the following metrics are used:
Pearson’s correlation coefficient (R) with a p value < 0.01
Root mean square difference (RMSD)
Bias (SWI – reference dataset)
The reference datasets (GLDAS and ISMN situ data) were temporally matched to the SWI
observation timestamp using nearest neighbour matching. This was necessary since the SWI is a
daily dataset whereas the reference datasets have multiple observations per day. Since the SWI is
in a different unit than the reference products, data scaling was necessary to ensure consistent
results across reference products. The used approach was that of min-max scaling. With this
method, one dataset is scaled to have the same maximum and minimum values as the other
dataset. This approach was taken since it does not completely remove the bias as does e.g.
scaling to the same mean and standard deviation. At first, also experiments with linear CDF
matching after Liu et al., (2011) were made but it was discovered that the matching failed for high
T-values in very dry regions because the algorithm was unable to calculate the necessary
percentiles.
The SWI data was scaled into the data space of the models and the in-situ data. This approach
was taken to have the same units (m3/m3) for all results. This approach allows a better comparison
of the RMSD and Bias between the modelled data and the in-situ data. Scaling into the in-situ data
space is commonly done in the literature (Brocca et. al 2011), so it was decided that it would be
best to also calculate the error metrics of the modelled data sets in absolute units.
Before the calculation of the metrics, observations where the Surface State Flag showed frozen
were removed. If less than ten observations were available for a grid point, no metrics were
calculated.
For the Pearson correlation coefficient R, the 95% confidence intervals were calculated and used
to determine if there were significant differences between the reference period and the current
validation period (January-December 2019). Confidence intervals were estimated using the Fisher
z-transformation (Fisher, 1915). For each R-value, a confidence band is found with this method.
Two R-values are only significantly different when these confidence bands do not overlap.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 25 of 78
In addition to the R-maps, also seasonal/quarterly time series of average R-values over specific
regions are calculated. The regions were chosen to represent well different climate zones and
biomes, comprising subcontinent-scale areas (Continental USA, East Africa, Australia) as well as
state-scale areas (Patagonia, Italy, California).
For this, the temporally and spatially matched values from the SWI time series and the reference
time series of all points in a region per season/quarter are used to calculate an R-value for this
region over the three months periods. The chosen regions are of sub-continental- or country- scale
and are shown in Figure 1. The R-values are calculated for the reference period and the current
validation period. For both periods, only data from pixels were used where each period had valid
observations. However, this does not ensure the same number of values per year, since the
Freeze/Thaw masking is different from year to year. However, for the reference dataset, the
standard deviation of the quarterly R-values are computed per chosen region, and added to the
plots to give insight on the relevance of the difference of current to reference data.
Figure 1: Map showing the regions used to define the regions of special interest for the plots of R
over time.
For the SSF validation, the following metrics are used:
Percentage of correctly classified observations (percent_valid)
Period of consecutive misclassified days (min, max, mean, median) (wrong_days)
Classification of the SSF values (Table 4) into the binary correct-incorrect form was done using the
classification scheme outlined in Table 5. The soil temperature was taken from the first layer of the
used land surface model (GLDAS). The percent_valid metric was then calculated by calculating the
percentage of correctly classified observations as a fraction of all observations. Wrong_days was
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 26 of 78
calculated by finding consecutive periods of wrong classification and measuring their length in
days. Since each grid point can have multiple such periods the minimum, maximum, mean and
median length of these periods is stored. During the calculation of the Wrong_days-metric, every
year was taken by itself to make the results comparable to the current validation period.
Table 4: Possible values of the SSF and their meaning
SSF value Detected surface state
255 Could not be determined
0 Unknown
1 Unfrozen
2 Frozen
3 Temporary water on the surface
Table 5: Classification scheme
SSF value Soil temperature Classification
0, 1, 3 >= 0°C Correct
2 < 0°C Correct
0, 1, 3 < 0°C Incorrect
2 >= 0°C Incorrect
The SSF analysis bears the caveat that soil temperature is only a rough proxy for the freeze/thaw
state of the soil. However, it is to our knowledge the only possibility for SSF validation at the
moment. Several studies working with freeze/thaw data from active (Naeimi et al., 2012) and
passive (Youngwook et al., 2011) satellites as well as the upcoming SMAP satellite freeze/thaw
product (NASA, 2014) all rely on temperature observations for validation purposes. Since these
validation reports are due 2-3 months after the SWI data was produced, we must rely on datasets
that are updated in a timely manner. This adds to the complexity of finding an alternative dataset to
temperature for SSF validation. Unfortunately, from 2017 ongoing, we found numerous and erratic
data-gaps and -flaws in the ISMN soil temperature dataset (see Section 4.4), so that we evaluated
the SSF only against GLDAS.
4.2 REGIONAL ANALYSIS
For the regional analysis, we focus on five regions with extraordinary hydrological dynamics in
2019, comprising southern Africa (with focus Zambia), Australia, and Europe (with focus Spain). To
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 27 of 78
examine the hydrology of these regions, we built the monthly anomalies of the SWI10 decadal
product, allowing efficient and robust computation of monthly means. In more detail, we calculated
the anomalies as follows:
1. Building the monthly means for each month of the full SWI data period 2007-2019.
2. The monthly means of the reference period (2007-2018) are averaged again for each
calendar month, yielding per pixel 12 values, building the reference.
3. Again per calendar month, subtracting the reference monthly means from the 2019-
monthly-means.
The SWI10 monthly anomalies are then compared qualitatively against the Vegetation Health
Index (VHI, Kogan et al., 1995, Kogan et al., 2003) as provided on the website1 of the Food and
Agriculture Organization of the United Nations (FAO), to examine how soil moisture anomalies
observed by the SWI relate to crop and vegetation health status. The VHI is an indicator routinely
used by FAO that highlights anomalous vegetation growth and potential drought in arable land
during a given period (Rojas et.al 2014). For the investigation on how Spain’s dry period and the
subsequent transient rainfall events in 2019 are captured by the SWI product, we compare it with
ECMWF-rainfall-estimates (also provided by FAO) at T-value=1, reflecting the topmost soil
surface. Anomalies from ECMWF-rainfall-estimates describe the relative difference between the
monthly 2019 rainfall volume and the average level, which refers to the period 1989-2015. The VHI
is compared with SWI at T-value=5, to account for the importance of the upper soil layer to the
vegetation.
4.3 MODEL REFERENCE PRODUCTS
Modelled reference products to which the SWI product was compared against were taken from the
GLDAS (Global Land Data Assimilation System) Noah-LSM (Noah Land Surface Model), version
2.1. The GLDAS Noah Land Surface Model (Rodell et al., 2004) is produced by NASA. It includes
four soil layers with the bottom depth at 0.1, 0.4, 1.0 and 2.0m respectively, and simulates over 30
land surface parameters in total. Data is available with a temporal sampling of three hours on a
global 0.25° grid. The model soil moisture data was converted from kg/m² to m3/m3 by using the
formula SM[m3/m3] = SM[kg/m2] * 0.001 * 1/d, where d is the thickness of the soil layer in meters.
The factor 0.001 is due to the assumption that 1kg of water represents 1000cm3, which is 0.001m3.
The GLDAS model dataset were chosen for the validation because it provides consistent long
term, global data and is updated regularly. The consistency of the dataset also makes it likely that
future SQE exercises can use them as well. This would have been less likely if we had chosen e.g.
soil moisture products derived from other remote sensing platforms that are vulnerable to satellite
failure etc.
1 www.fao.org/gviews/earthobservation/
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 28 of 78
The soil moisture component of the model is of course not free of errors, but there are
comprehensive studies on the global quality of the modelled soil moisture. Dorigo et al. (2010)
investigated the quality of the GLDAS and ERA-Interim models with the triple collocation method
using remotely sensed soil moisture datasets as the third datasets. Similar error structures were
found for the two models, indicating that results obtained for the ERA-Interim model can be used
as a proxy for the GLDAS soil moisture output. Albergel et al. (2013) explored the performance of
the ERA-Land model, which is a pre-operational successor to ERA-Interim, and MERRA-Land
model against 196 in-situ stations from the ISMN and found mean correlation coefficients of 0.66
and 0.69 respectively. Chen et al. (2013) compared the GLDAS model output against in-situ
stations on the Tibetan plateau and found that the model underestimates surface soil moisture (0-5
cm) but was in good agreement with a deeper layer (10 – 40cm). Correlations of 0.69 (0-5cm) and
0.76 (10-40cm) were found. Because of these results suggesting a high agreement of the models,
and following review board’s recommendation, the ERA Interim is not used from the present SWI
evaluation. As a note, GLDAS features a higher resolution than ERA (0.25° vs 0.7°), which is
closer to the SWI’s resolution of 0.1°.
For the SQE 2019, the GLDAS-Noah Version 2.1 was used to cover the full evaluation period (as
in the previous SQEs 2017 and 2018).
4.4 IN-SITU REFERENCE PRODUCTS
In-situ data from the International Soil Moisture Network (ISMN) is used as a reference. The ISMN2
provides a harmonized repository of in-situ soil moisture observations. This harmonization makes it
feasible to use in-situ data from a wide array of networks. The data quality of the stations can vary
widely and is not guaranteed to be consistent in time for any station. Wild animals as well as
farmers or other natural phenomena can lead to sensor failure or shifts in the location which
invalidate the sensor calibration. These errors are partly addressed by various quality and
consistency checks that were recently introduced by (Dorigo et al., 2013) which aim to
automatically detect and flag jumps, plateaus and other unrealistic behaviour in the data. Also, it is
checked for realistic maximum and minimum values.
Spatial representativeness is an issue when comparing remotely sensed and in-situ data. This
means how well an in-situ observation, which is essentially a point measurement, represents the
data gathered by the much larger satellite footprint. This is also different for each station and
depends on topography, soil types and microclimates. The geographic distribution of the available
stations in the current validation period is shown in Figure 2, illustrating the inhomogeneous global
distribution of ground data, leaving many parts of the world unobserved. A more detailed view can
be seen on the interactive web portal of the ISMN.
2 https://ismn.geo.tuwien.ac.at/
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 29 of 78
Figure 2: Map of used ISMN networks, showing all stations with data in the period 2007-2019.
Each network has different data availability and references (see Table 6), but most of the networks
and stations used in SQE 2018 are used in SQE 2019, and also newly availably stations are
integrated (totalling 1026 stations).
To get the listed start- and end-date, the maximum and minimum date found in any station of the
network is given. It can be seen that most in-situ networks do not cover the full validation period.
Furthermore, the maximum date might not apply since this could be either for a time series that
was not soil moisture, or not in the right observation depth.
Another issue is that many networks or single stations do not measure soil temperature (in addition
to the soil moisture measurements). Even more critical is that since 2018, many soil temperature
time series are contaminated with inconsistent and erroneous data, especially in the winter season,
so that the ISMN dataset on soil temperature suffers from irregular coverage and quality. From
personal correspondence with ISMN technical staff, we got informed that this is due to unchecked
quality of raw ground temperature data in several networks. As a consequence, the analysis of
SSF against in-situ soil temperature data is skipped in this report.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 30 of 78
Table 6: Data availability and references for the used in-situ networks.
Network Name Start End Reference
COSMOS 2008-04-28 2019-12-31 http://cosmos.hwr.arizona.edu/, Zreda et al., 2008
FMI 2007-01-25 2019-12-31 http://fmiarc.fmi.fi/
GROW 2017-02-08 2019-10-08 https://growobservatory.org/index.html
HOBE 2009-09-08 2019-03-13 http://www.hobe.dk/
RHEMEDUS 2007-01-01 2019-12-31 http://campus.usal.es/~hidrus/
RSMN 2014-04-09 2019-12-31 http://assimo.meteoromania.ro/
SCAN 2007-01-01 2019-12-31 http://www.wcc.nrcs.usda.gov/scan/
SNOTEL 2007-01-01 2019-12-31 http://www.wcc.nrcs.usda.gov/, Leavesley et al., 2008
TERENO 2009-12-31 2019-12-31 http://teodoor.icg.kfa-juelich.de/, Zacharias et. al. 2011
USCRN 2007-01-01 2019-12-31 http://www.ncdc.noaa.gov/crn/, Bell et al., 2013, Diamond
et al., 2013
WEGENERNET 2007-01-01 2019-12-31 http://www.wegcenter.at/wegenernet
4.5 OTHER REFERENCE PRODUCTS
For the regional analysis, Vegetation Heath Index (VHI, Kogan et al., 1995, Kogan et al., 2003) and
ECMWF-rainfall-estimates from the FAO website3, as well as several news articles and bulletins
were used. The VHI is a FAO indicator that highlights anomalous vegetation growth and potential
drought in arable land during a given period (Rojas et.al 2014). It is a composite index describing
the health and vigorousness of crops and vegetation in general. It combines both the Vegetation
Condition Index (VCI) and the Temperature Condition Index (TCI). The TCI is calculated using a
similar equation to the VCI, but relates the current temperature to the long-term maximum, as it is
assumed that higher temperatures tend to cause a deterioration in vegetation conditions. A
decrease in the VHI following, for example, a decline in the VCI (relatively poor green vegetation)
and an increasing TCI (warmer temperatures) would signify stressed vegetation conditions, and
over a longer period would be indicative of drought. The VHI components (VCI and TCI) are given
equal weights when computing the index.
3 www.fao.org/gviews/earthobservation/
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 31 of 78
4.6 DATA AVAILABILITY
Table 7 shows the data availability of the SWI and the reference datasets. The GLDAS Noah
model data covers the complete validation period, whereas some of the ISMN networks do not
cover the full validation period.
Table 7: Available datasets for validation
4.7 LAYER COMPARISON MATRIX
The different T-values of the SWI products were compared to different layers of the GLDAS model
as well as in-situ data. This was done since higher T-values represent deeper soil layers. Table 8
shows which T-values were attributed in the comparison to which layer depths of the reference
data. General recommendations and insights on the attribution of T-values to soil depth are given
in the SWI PUM document [CGLOPS1_PUM_ SWIV3-SWI10-SWI-TS].
Table 8: Which T-values were compared to which layer of the modelled and in-situ datasets
SWI T-Value GLDAS layer depth In-situ observation depth
1, 5, 10, 15, 20, 40 0 - 0.1 m 0 - 0.1 m
60, 100 0.4 – 1 m 0.5 m
Dataset Start End Spatial Coverage, Resolution
Observation depth
Unit
SWI V3 NRT 2019-01-01 2019-12-31 Global, 25km T-value Percent
SWI V3 archive
2007-01-01 2018-12-31 Global, 25km T-value Percent
GLDAS 2007-01-01 2019-12-31 Global, 0.25° 0 - 0.1 m 0.4 – 1m
°C and Kg/m²
ISMN 2007-01-01 Different, max 2019-12-31
1026 stations 0 - 0.1 m 0.5 m
m3/m3
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 32 of 78
5 RESULTS
5.1 GLOBAL ANALYSIS
The comparisons were done for all eight T-Values but we will show in this report only the results for
T=1, 20 and 100 in detail, since nothing out of the ordinary was observed for any other T-Value.
The results are ordered first by reference dataset and second by metric, and are presented as
images well as in tabular form. The tables show the mean and median values of the computed
metrics as well as 50-, 70- and 80- percentiles of the absolute differences between the current
validation period and the reference period. The 70th percentile means for example that 70% of all
differences are smaller than the given value. All images map globally the metric measured during
the reference period as well as during the current validation period. Additionally, the difference
(reference - current) is mapped. Violin-plots show the estimated kernel density function of the
datasets as well as the median and the lower- and upper- quartiles. The violin-plots can be
interpreted similar to boxplots, but show in addition the estimated distribution of the data, which
provides more information than a simple boxplot would.
5.1.1 Comparison with GLDAS Noah
5.1.1.1 Results for SWI
5.1.1.1.1 Pearson’s correlation coefficient
The results for Pearson’s correlation coefficient R (Figure 3 to Figure 5) show that correlations are
different at the 95% confidence interval for some areas. Statistical significance means that the
calculated confidence intervals do not overlap.
Most importantly, the results for the reference period and 2019 show very similar patterns,
indicating a stable performance of the SWI product, not suffering from any substantial or
systematic quality degradation. In the violin-plots, one can see that R-values for the current
validation period are slightly more extreme, as that the distribution in the violin plot is more
stretched. Areas with positive R-values tend to be more positive and areas with negative R-values
tend to be more negative. This behavior was already observed in the previous year and is likely
due to the shorter observation period of only one year. Again in 2019, over desert areas, e.g. over
the Sahara, the Namibian deserts, or the Arabian peninsula, and also over the semi-arid Iranian
area the SWI shows weak or no agreement with the GLDAS model (albeit improved over the
Sahara and Arabia peninsula). This confirms previous findings that the ASCAT data does not
model soil moisture correctly in very arid areas.
However, large parts of the Earth show a slightly improved agreement with the model in 2019 than
in the reference period. The western section of the US, Europe, Turkey and its neighbors, the
Sahel area, and South America (most prominently Argentina) show better agreement in 2019 than
the previous years while the central parts of Australia, scattered parts of Russia, central US and
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 33 of 78
Alaska, and eastern Europe show worse performance. The negative differences appear stronger
for the T-value=20 (Figure 4) and again similar for T=100 (Figure 5). Overall, the spatial patterns
between the reference period and 2019 are very similar, so a consistent quality deprecation is
unlikely. Continuing the last year’s trend, the difference distributions lean again to the negative,
meaning that the R-values for 2019 are higher than for the reference period 2007-2018.
The mean and median values as well as the differences shown in Table 9 compare well with the
values obtained from the previous exercise performed on 2018 data, with mean and median
correlations being little higher, with biggest improvement for T=20 (mean R raised from 0.29 to
0.34)
Figure 3: Pearson’s correlation coefficient between SWI T=1 and GLDAS Noah, as well the difference
between the reference period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-
01 until 2019-12-31). Green colours indicate performance improvements in the current period. In the
difference map only the statistically significant differences are shown whereas the violin plots show
all data points. The violin plots show the estimated kernel density plot as well as the median (dashed
line) and the lower and upper quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 34 of 78
Figure 4: Pearson’s correlation coefficient between SWI T=20 and GLDAS Noah, as well the
difference between the reference period (2007-01-01 until 2018-12-31) and the current validation
period (2019-01-01 until 2019-12-31). Green colours indicate performance improvements in the
current period. In the difference map only the statistically significant differences are shown whereas
the violin plots show all data points. The violin plots show the estimated kernel density plot as well
as the median (dashed line) and the lower and upper quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 35 of 78
Figure 5: Pearson’s correlation coefficient between SWI T=100 and GLDAS Noah, as well the
difference between the reference period (2007-01-01 until 2018-12-31) and the current validation
period (2019-01-01 until 2019-12-31). Green colours indicate performance improvements in the
current period. In the difference map only the statistically significant differences are shown whereas
the violin plots show all data points. The violin plots show the estimated kernel density plot as well
as the median (dashed line) and the lower and upper quartile (dotted lines).
Table 9: Mean / median R between SWI and GLDAS Noah as well as percentiles of the absolute
values of the differences. All values in m3/m
3.
mean median 50th percentile 70th percentile 80th percentile
SWI T=1 0.41 0.57 0.20 0.28 0.34
SWI T=20 0.34 0.52 0.23 0.32 0.39
SWI T=100 0.33 0.54 0.35 0.53 0.66
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 36 of 78
Figure 6 shows the quarterly R-values over selected regions as shown in Figure 1. The yearly
evolution of the performance over Eastern Africa, and Contiguous USA including separately
analysed Californa, is similar for the reference and the current period, showing almost no deviation
in the annual SWI dynamics in these regions.
However, over Mainland Australia there is this year a lower agreement of the SWI product with
GLDAS in Q1 and especially Q4. This is confirming the decreased scores over the Australian dry
areas in the centre of the continent (as in Figure 3 to Figure 5)
On the contrary, for Patagonia, we observe much higher R-values throughout the year 2019,
showing consistency with the higher R-values found over that area (significantly higher in Q1-Q3)
than in the reference period in Figure 3 to Figure 5.
Repeating last year’s analysis, over Italy, featuring Mediterranean climate, one can see for both
periods that the seasonal correlation is higher later in the year, probably due to higher hydrological
dynamics induced by winter rainfalls. The same might be true for eastern Africa, as during the rain
seasons (Q2 and Q4) the correlation is stronger for both evaluation periods.
Figure 6: Average 3-month Pearson’s correlation coefficient R between SWI T=5 and GLDAS Noah,
for the reference period (2007-01-01 until 2018-12-31, green line) and the current validation period
(2019-01-01 until 2019-12-31, purple line) over various areas. The standard deviation for the reference
is shown as light green surface.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 37 of 78
5.1.1.1.2 Root mean square difference
Figure 7 to Figure 9 show the RMSD-results for the two periods and their differences on maps and
as violin-plots. Again, there seem to be no systematic differences. When consulting previous
exercises results, most differences can be interpreted as year-to-year fluctuations. Higher RMSDs
are observed this year over southeastern Canada, the southern Sahara area in Africa, and Eastern
Europe, for T=1 and T=20. Lower RMSDs are detected over many areas in high and mid-latitudes,
and in the tropical forests in South America, Africa and Indonesia, but the SWI data should be used
with caution in those regions, as due to the high vegetation density the ASCAT has only little
sensitivity (here Tropical Forest Mask of the SWI-STATIC collection is recommended).
Overall, the RMSD values are smaller in 2019 compared to the reference period, as the distribution
of the difference leans to positive (especially for T=1 and T=100), continuing the trend from SQE
2017 and 2018.
Table 10 shows the mean and median values of the RMSD as well as the absolute values of the
differences between the two periods. The values are identical to the values in last year’s SQE. Also
the shapes of the violin-plots for reference, current and difference are very akin to last years,
suggesting that there are no systematic changes in the SWI data.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 38 of 78
Figure 7: RMSD between SWI T=1 and GLDAS Noah, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31).
Green colours indicate performance improvements in the current period. In the difference map only
the statistically significant differences are shown whereas the violin plots show all data points. The
violin plots show the estimated kernel density plot as well as the median (dashed line) and the lower
and upper quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 39 of 78
Figure 8: RMSD between SWI T=20 and GLDAS Noah, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31).
Green colours indicate performance improvements in the current period. In the difference map only
the statistically significant differences are shown whereas the violin plots show all data points. The
violin plots show the estimated kernel density plot as well as the median (dashed line) and the lower
and upper quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 40 of 78
Figure 9: RMSD between SWI T=100 and GLDAS Noah, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31).
Green colours indicate performance improvements in the current period. In the difference map only
the statistically significant differences are shown whereas the violin plots show all data points. The
violin plots show the estimated kernel density plot as well as the median (dashed line) and the lower
and upper quartile (dotted lines).
Table 10: Mean / median RMSD between SWI and GLDAS Noah as well as percentiles of the absolute
values of the differences. All values in m3/m
3.
mean median 50th percentile 70th percentile 80th percentile
SWI T=1 0.06 0.05 0.02 0.03 0.03
SWI T=20 0.07 0.06 0.02 0.03 0.04
SWI T=100 0.04 0.03 0.02 0.03 0.04
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 41 of 78
5.1.1.1.3 Bias
The spatial patterns of the Bias shown in Figure 10 to Figure 12 correspond to the spatial patterns
seen for the RSMD. This is not surprising since the two metrics are closely related. On average,
the (absolute) biases are the same as for the reference period, but show less extreme values (less
stretched violin-plots), which can be expected due to the scaling of only one year of data for the
current validation period. With higher T-values, the bias-values are more and more small, owed to
the much smaller moisture dynamics in the deeper soil layers. Table 11 summarizes the statistics
on bias, with only the 80th-percentile value for T=1 raised by 0.01 against last year, supporting the
insight that there is no systematic difference in the product.
Figure 10: Bias between SWI T=1 and GLDAS Noah, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31).
Green colours indicate performance improvements in the current period. In the difference map only
the statistically significant differences are shown whereas the violin plots show all data points. The
violin plots show the estimated kernel density plot as well as the median (dashed line) and the lower
and upper quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 42 of 78
Figure 11: Bias between SWI T=20 and GLDAS Noah, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31).
Green colours indicate performance improvements in the current period. In the difference map only
the statistically significant differences are shown whereas the violin plots show all data points. The
violin plots show the estimated kernel density plot as well as the median (dashed line) and the lower
and upper quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 43 of 78
Figure 12: Bias between SWI T=100 and GLDAS Noah, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31).
Green colours indicate performance improvements in the current period. In the difference map only
the statistically significant differences are shown whereas the violin plots show all data points. The
violin plots show the estimated kernel density plot as well as the median (dashed line) and the lower
and upper quartile (dotted lines).
Table 11: Mean / median Bias between SWI and GLDAS Noah as well as percentiles of the absolute
values of the differences. All values in m3/m
3
mean median 50th percentile 70th percentile 80th percentile
SWI T=1 0.02 0.01 0.02 0.03 0.05
SWI T=20 0.03 0.02 0.02 0.04 0.05
SWI T=100 0.01 0.01 0.02 0.03 0.04
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 44 of 78
5.1.1.2 Results for Surface State Flag
The SSF was evaluated against the soil temperature data from GLDAS.
Figure 13, Figure 14, and Figure 15 show the percentage of correct SSF classifications, and the
maximum and mean in consecutive wrongly flagged days per grid point, respectively. The spatial
patterns for the percentage of correct SSF are globally almost identical for the current and
reference period, as well as with both results from SQE 2018. However, the difference highlights
reduced accuracy over Scandinavia and scattered parts of North America and central Asia. The
last years’ degradation over Canada was not continuing into 2019, with mostly better scores this
year. However, Scandinavia shows now for the second year worse performance in SSF masking.
Patterns for areas showing on average many consecutive days wrongly classified remain very
stable, with longest periods in Tibetan Plateau and Siberia (but with the latter improved in 2019),
and only short periods in the rest of the world (Figure 15).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 45 of 78
Figure 13: Percent of correctly classified freeze/thaw states compared to GLDAS Noah soil
temperature as well the difference between the reference period (2007-01-01 until 2018-12-31) and the
current validation period (2019-01-01 until 2019-12-31). Green colours indicate performance
improvements in the current period. In the difference map only the statistically significant differences
are shown whereas the violin plots show all data points. The violin plots show the estimated kernel
density plot as well as the median (dashed line) and the lower and upper quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 46 of 78
Figure 14: Maximum number of consecutive days wrongly classified when comparing the SSF
against GLDAS Noah soil temperature as well the difference between the reference period (2007-01-
01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31). Green colours
indicate performance improvements in the current period. In the difference map only the statistically
significant differences are shown whereas the violin plots show all data points. The violin plots show
the estimated kernel density plot as well as the median (dashed line) and the lower and upper
quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 47 of 78
Figure 15: Mean number of consecutive days wrongly classified when comparing the SSF against
GLDAS Noah soil temperature as well the difference between the reference period (2007-01-01 until
2018-12-31) and the current validation period (2019-01-01 until 2019-12-31). Green colours indicate
performance improvements in the current period. In the difference map only the statistically
significant differences are shown whereas the violin plots show all data points. The violin plots show
the estimated kernel density plot as well as the median (dashed line) and the lower and upper
quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 48 of 78
Table 12 summarizes the validation results for the SSF. The overall accuracy of the classification
remains very stable, as again 95% of all classifications can be considered correct. The SSF was
on average wrongly set for the same amount of days as in the 2018 SWI product, as the global
metrics for the consecutive days metric are almost identical (e.g. mean of 7.4 consecutively
wrongly set days; compared to 7.5 in 2018). However, this year a higher amount of outliers is in
the data, as the median value improve slightly (4.3 consecutively wrongly set days; compared to
4.8 in 2018), indicating an improvement apart from the outliers.
Table 12: Mean / median metrics of the SSF validation with GLDAS Noah as well as percentiles of the
absolute values of the differences.
mean median 50th
percentile
70th
percentile
80th
percentile
correct classification [%] 95.5 100.0 1.8 3.1 4.1
incorrect classification [%] 4.5 0.0 1.7 2.8 3.7
Consecutive days wrongly
classified (max)
18.8 13.0 5.2 8.1 10.2
Consecutive days wrongly
classified (min)
4.2 1.0 0.9 1.7 2.5
Consecutive days wrongly
classified (median)
4.3 3.0 2.2 3.6 4.6
Consecutive days wrongly
classified (mean)
7.4 5.4 2.4 3.8 4.8
5.1.2 Comparison with ISMN
5.1.2.1 Results for SWI
The following plots show the results for stations with sufficient valid data in both periods (mainly
located in the US), the reference years and the year 2019. The plots do not show any networks in
the southern hemisphere, only one in Africa, none in South America, Oceania, and Asia. Although
there are (a few) ISMN stations holding data for the validation period, not enough in-situ data that
temporally matches the SWI data was available for 2019 in these regions, unfortunately.
5.1.2.1.1 Pearson’ correlation coefficient
Figure 16 to Figure 18 show the R-values and the significant differences between the current
validation period and the reference period. The R-values for the current validation period are on
average lower than in the reference period, with poorer agreement for stations in the Mississippi-
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 49 of 78
Missouri area in the US, confirming the difference pattern against GLDAS (e.g. in Figure 3). The
degraded performance around the Great Lakes is not seen in the GLDAS analysis.
Apparently, also a number of stations in southern California, Arizona and Nevada show a negative
correlation, especially for 2019, which is also visible in the comparisons with GLDAS over this
area. Again, the ASCAT model appears to not correctly estimate soil moisture in arid conditions.
Overall, the R-values for all ISMN stations are lower in 2019 compared to the reference period,
with biggest differences coming from a significant number of stations with extreme negative
correlation, clearly visible from the violin-plots. All metrics for R show lower values (Table 13) in
this year’s SQE.
Figure 16: Pearson’s correlation coefficient between SWI T=1 and in-situ data, as well the difference
between the reference period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-
01 until 2019-12-31). Green colours indicate performance improvements in the current period. In the
difference map only the statistically significant differences are shown whereas the violin plots show
all data points. The violin plots show the estimated kernel density plot as well as the median (dashed
line) and the lower and upper quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 50 of 78
Figure 17: Pearson’s correlation coefficient between SWI T=20 and in-situ data, as well the difference
between the reference period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-
01 until 2019-12-31). Green colours indicate performance improvements in the current period. In the
difference map only the statistically significant differences are shown whereas the violin plots show
all data points. The violin plots show the estimated kernel density plot as well as the median (dashed
line) and the lower and upper quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 51 of 78
Figure 18: Pearson’s correlation coefficient between SWI T=100 and in-situ data, as well the
difference between the reference period (2007-01-01 until 2018-12-31) and the current validation
period (2019-01-01 until 2019-12-31). Green colours indicate performance improvements in the
current period. In the difference map only the statistically significant differences are shown whereas
the violin plots show all data points. The violin plots show the estimated kernel density plot as well
as the median (dashed line) and the lower and upper quartile (dotted lines).
Table 13: Mean / median correlation coefficient between SWI and in-situ data as well as percentiles of
the absolute values of the significant differences
mean median 50th percentile 70th percentile 80th percentile
SWI T=1 0.34 0.51 0.30 0.45 0.58
SWI T=20 0.34 0.51 0.30 0.45 0.58
SWI T=100 0.36 0.56 0.40 0.57 0.66
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 52 of 78
5.1.2.1.2 Root mean square difference
The RMSD maps in Figure 19 to Figure 21 show that the RMSD are lower in 2019 than in the
multi-year reference period, but higher than in the last SQE’s, revealing here a fluctuating behavior.
Spatial patterns of the differences cannot be deduced, giving the spatially scattered distribution of
negative and positive differences. Also the RMSD statistics for 2019 in Table 14 show lower
agreement compared to last year’s analysis.
Figure 19: RMSD between SWI T=1 and in-situ data, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31).
Green colours indicate performance improvements in the current period. In the difference map only
the statistically significant differences are shown whereas the violin plots show all data points. The
violin plots show the estimated kernel density plot as well as the median (dashed line) and the lower
and upper quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 53 of 78
Figure 20: RMSD between SWI T=20 and in-situ data, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31).
Green colours indicate performance improvements in the current period. In the difference map only
the statistically significant differences are shown whereas the violin plots show all data points. The
violin plots show the estimated kernel density plot as well as the median (dashed line) and the lower
and upper quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 54 of 78
Figure 21: RMSD between SWI T=100 and in-situ data, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31).
Green colours indicate performance improvements in the current period. In the difference map only
the statistically significant differences are shown whereas the violin plots show all data points. The
violin plots show the estimated kernel density plot as well as the median (dashed line) and the lower
and upper quartile (dotted lines).
Table 14: Mean / median RMSD between SWI and in-situ observations as well as percentiles of the
absolute values of the differences. All values in m3/m
3.
mean median 50th percentile 70th percentile 80th percentile
SWI T=1 0.092 0.083 0.026 0.045 0.060
SWI T=20 0.092 0.083 0.026 0.045 0.060
SWI T=100 0.072 0.060 0.026 0.044 0.060
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 55 of 78
5.1.2.1.3 Bias
The bias maps and violin-plots in Figure 22 to Figure 24 show similar difference patterns between
the current and the reference period, but an almost symmetric distributions of the differences (as in
SQE 2018). Interestingly, the overall spread of the violin plots is smaller, and the bias statistics in
Table 15 show decreased values (opposed to higher values between 2018, and smaller values in
2017), suggesting year-to-year fluctuations in the differences between SWI and in-situ data.
Figure 22: Bias between SWI T=1 and in-situ data, as well the difference between the reference period
(2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31). Green
colours indicate performance improvements in the current period. In the difference map only the
statistically significant differences are shown whereas the violin plots show all data points. The
violin plots show the estimated kernel density plot as well as the median (dashed line) and the lower
and upper quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 56 of 78
Figure 23: Bias between SWI T=20 and in-situ data, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31).
Green colours indicate performance improvements in the current period. In the difference map only
the statistically significant differences are shown whereas the violin plots show all data points. The
violin plots show the estimated kernel density plot as well as the median (dashed line) and the lower
and upper quartile (dotted lines).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 57 of 78
Figure 24: Bias between SWI T=100 and in-situ data, as well the difference between the reference
period (2007-01-01 until 2018-12-31) and the current validation period (2019-01-01 until 2019-12-31).
Green colours indicate performance improvements in the current period. In the difference map only
the statistically significant differences are shown whereas the violin plots show all data points. The
violin plots show the estimated kernel density plot as well as the median (dashed line) and the lower
and upper quartile (dotted lines).
Table 15: Mean / median bias between SWI and in-situ observations as well as percentiles of the
absolute values of the differences. All values in m3/m
3.
mean median 50th percentile 70th percentile 80th percentile
SWI T=1 0.018 0.011 0.033 0.051 0.067
SWI T=20 0.018 0.011 0.033 0.051 0.067
SWI T=100 0.018 0.012 0.025 0.043 0.057
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 58 of 78
5.1.2.2 Time Series for selected in-situ stations
Six in-situ stations were selected to be discussed in more detail. They were selected at random,
but care was taken that poor and good results are shown, and that different climate zones are
represented. The six selected stations and their climate zones are listed in Table 16. For all in-situ
data printed in Figure 25 to Figure 30, the first layer (measured at a depth of 0.05 m) is shown. For
these plots, SWI with T=5 is plotted, which was found to be in general most corresponding to that
depth.
Table 16: Station details and climate classifications
Station Name Network latitude / longitude [°] Köppen-Geiger Climate class
CRAB CREEK SNOTEL 44.44 N / 111.99 W Dfb - snow - fully humid - warm
summer
Edinburg-17-NNE USCRN 26.53 N / 98.06 W Cfa - warm temperate - fully
humid - hot summer
Mercury 3 SSW USCRN 36.57 N / 116.02 W Bwk - arid - desert - cold arid
Pine-Nut SCAN 36.57 N / 115.20 W Bwk - arid - desert - cold arid
Bacles RSMN 44.48 N / 23.11 E Cfb - warm temperate - fully
humid - warm summer
Gevenich TERENO 50.99 N / 6.32 E Cfb - warm temperate - fully
humid - warm summer
Table 17 shows Pearson R-values and RMSD for the stations. The stations Mercury 3 SSW and
Pine Nut show strong negative correlations, confirming the insight from last reports that there is a
low agreement of SWI and in-situ data over arid areas. The station Edinburgh-17-NNE has
unchanged (good) performance compared to 2018, while the Crab Creek station has a small drop
in scores. Also, the stations in Gevenich and Bacles, representing temperate climate zones that
are important for many users, show satisfying but decreased results. Most notably in 2019,
although showing drier-than-normal values, a substantial overestimation of the SWI against the in-
situ measurements is apparent, suggesting that the reported dry conditions (see Section 5.2.4)
were not fully captured by the SWI.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 59 of 78
Table 17: Pearson’s correlation coefficient (R) and Root mean square deviation (RMSD) for the
selected in-situ stations. SWI T=1 was compared to in-situ soil moisture measured at a depth of
0.05m.
Station R RMSD [m³/m³]
CRAB CREEK 0.69 0.05
Edinburg-17-NNE 0.75 0.03
Mercury 3 SSW -0.72 0.08
Pine-Nut -0.79 0.09
Bacles 0.73 0.06
Gevenich 0.55 0.08
Figure 25: SWI and in-situ data for SNOTEL-station CRAB CREEK
Figure 26: SWI and in-situ data for USCRN-station Edinburg-17-NNE
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 60 of 78
Figure 27: SWI and in-situ data for USCRN-station Mercury 3 SSW
Figure 28: SWI and in-situ data for SCAN-station Pine Nut
Figure 29: SWI and in-situ data for RSMN-station Bacles
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 61 of 78
Figure 30: SWI and in-situ data for TERENO-station Gevenich
Figure 31: As Figure 30, but for SWI with T-value=100
For the (German) Gevenich station, a time lag to the in-situ data can be seen, may hinting at an
inappropriate attribution of the T-value=5 to the soil depth. When compared to SWI with T=100
(Figure 31), the timing of the peaks and lows of the satellite data agrees much better with the in-
situ observations. This indicates that, at this location, information on local soil properties is
important for the optimal selection of the T-values. It should be noted that there is no general rule
on the attribution of T-values to depth, as it depends strongly on the application and the soil
composition of the area of interest (Paulik et al., 2014).
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 62 of 78
5.2 REGIONAL ANALYSIS
For the regional analysis, we examine on five regions with extraordinary hydrological dynamics in
2019, comprising southern Africa with a focus on Zambia, Australia, and Europe with a focus on
Spain. The extent and magnitude of the different hydrological situations were assessed in
analogous fashion as in previous SQEs by comparing the monthly SWI anomalies with the
Vegetation Health Index (VHI) provided by the Food and Agriculture Organization of the UN (FAO),
and with rainfall estimates anomalies (built from the long term average of the period 1989-2015)
provided by the European Centre for Medium-Range Weather Forecasts (ECMWF).
It should be noted, that these comparisons are only a qualitative look to track if the SWI product
shows similar patterns to the vegetation health and rainfall estimates maps. While they are
commonly able to illustrate connections between SWI, soil hydrology, and vegetation status, the
comparisons have three main caveats. First, the reference period for the SWI anomaly is only
twelve years, which is relatively short for computing meaningful anomalies. Second, the
comparison datasets are observations of vegetation, which while strongly dependent on soil
moisture, are of course not the same variable and are impacted by other parameters. We see for
instance, that the link between VHI and SWI seems to be stronger and much more direct over hot
climates (as in southern Africa), than over temperate mid-latitude areas (as over Europe). Third,
strong vegetation anomalies will have an effect on the observed soil moisture since the retrieval
model assumes the same vegetation correction for each year. However, we are confident that
monthly SWI anomalies help to understand if the SWI data is capable of capturing hydrological
events, examining not only the temporal evolution, but also their spatial dynamics.
5.2.1 Situation in southern Africa in 2019
The southern part of Africa is still suffering from prolonged dry spells, but also from heavy rainfalls
resulting in displacement and crop loss, which conclude to food insecurity and increasing food
prices. FAO, the International Fund for Agricultural Development (IFAD), and the World Food
Program (WFP) reported that over 45 million people across the Southern African Development
Community were suffering from food insecurity by the end of 2019. Late rains, extended dry
periods and two major cyclones resulted in food insecurity, increasing food prices and driving up
aid needs (NF1, NF2). Dry spells especially in the first half of 2019 can be clearly seen in Figure
32, with a peak of negative SWI anomalies in March. In contrast, October to December 2019 was
one of the wettest seasons since 1981 for northern Tanzania, northern Madagascar and other
regions (NF3), represented as areas with high positive SWI anomalies. For the regional analysis,
we focus in the next subsection on the condition in heavily affected Zambia.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 63 of 78
Figure 32: Monthly SWI anomalies over southern Africa in 2019.
5.2.2 Zambia
As reported by the Regional Bureau for Southern Africa of WFP, drought and prolonged dry spells
have left 2.3 million people severely food insecure (NF4). Figure 33 indicates that the dry weather
led to highly negative SWI anomalies, especially in the south-east part of Zambia for the beginning
of the year. Strong agreement is found in the comparison with the vegetation health index (VHI),
with biggest impact in March and April, accounting for some ability of the vegetation to resist
moisture shortage at the onset of the drought in January and February. On the contrary, from June
to October, SWI anomalies do not represent the marked differences as seen in the VHI maps. In
spite of the drought conditions, floods were reported n the north-eastern part for December (NF5),
which can be related to positive SWI anomalies in this section of Zambia.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 64 of 78
Figure 33: Monthly SWI anomalies (top) and Vegetation Health Index (VHI) data (bottom) over Zambia
in 2019.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 65 of 78
5.2.3 Australia
Severe drought conditions developed in Australia during 2019, resulting a catastrophic bushfire
season, which has left 33 people dead and burned an area of land the size of South Korea (NF6),
colloquially known as the Black Summer. In contradiction to the general hydrological regime,
February and late March brought record-breaking rainfall and disastrous flooding in northern
Queensland (NF7).
2019 was Australia's warmest year on record, with the annual national mean temperature 1.52 °C
above average. Rainfall below average for most of the country as well as significant heat waves in
January and in December resulted in the driest year on record (NF8). The comparison of SWI
anomalies and vegetation health index (VHI), presented in Figure 34, show strong agreement for
drought and flooding events in the eastern half of the continent, and a somewhat lower agreement
of spatial anomaly pattern in the western half. In the northern and eastern part of Australia, high
negative SWI anomalies are detected throughout the whole year.
Clearly distinguishable are the wet anomalies from February to June in Queensland (in north-east
Australia), which are in good accordance with the reported heavy rain falls. In the affected regions,
the soil appeared to remain saturated until winter season.
The overall negative anomalies early and late in 2019, as well as the mixed situation in the middle
of the year, can also be seen in the areal average plot for mainland Australia in Figure 6.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 66 of 78
Figure 34: Monthly SWI anomalies (top) and Vegetation Health Index (VHI) data (bottom) over
Australia in 2019.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 67 of 78
5.2.4 Europe
Also Europe were hit by a dry spell, experienced another extraordinary hot summer in 2019
following already record-breaking hot years. The monthly bulletins produced by the Joint Research
Centre (JRC) give an overview on weather events for crop monitoring in Europe. By the end of the
year, large parts of Turkey, Ukraine and the southern part of the Iberian Peninsula experienced a
lack of rainfall, which affects sowing whereas, in the northern part of the Iberian Peninsula, France,
Italy and regionally in central Europe, a wetter-than-usual weather was recorded (NF9).
As reported by the NASA Earth Observatory (NF10) and EUMETSAT's LSA SAF (NF11), the
summer of 2019 was again extraordinary hot in Europe. In July, at least seven countries broke
temperature records in a torrid heat wave. Data provided by the Copernicus Climate Change
Service showed that it has become the second warmest calendar year on record with a
temperature 0.59 °C above the 1981-2010 average (NF12). Accordingly, SWI anomalies show
large-scale negative patterns over large parts of Europe from January to October, which can be
seen in Figure 35. In this period, countries neighboring the Mediterranean show negative or neutral
values for most months. Central and eastern Europe experienced alternating conditions, but with
largest deviations from average in March, April, and July. As a marked example, Hungary suffered
from severe drought conditions in the beginning of the year resulting in increasing fruit and
vegetable prices and harvest losses for several crops (NF13). The reported drought is supported
by negative SWI anomalies in this area, peaking in March. However, except for Germany, Poland
and France, SWI anomalies only confirm partly (with only small magnitude) with the reported hot
temperatures during the summer months. Probably, this is due to already below long term average
SWI values for the previous years, and the different reference timeline for building the anomalies
(starting in 1981 vs. 2007). From October on, a pressure was relieved for the continent, with
wetter-than-normal conditions starting in Western Europe, and the evolving in southern Europe and
Spain (see also section 5.2.5) from November on.
The exceptional dry conditions can be related to the time series plots of the European in-situ
stations in above chapter. The Gevenich station in Germany featured in 2019 significantly lower
in-situ soil moisture measurements than in the previous years (Figure 30), but not in the SWI data
with T=5, and only little with T=100. This station is located in western Germany, where also the
SWI anomaly maps do not show strong anomalies. Obviously, the dry spring period could not be
observed well there. This might be linked to known wet biases of the ASCAT SSM input data
appearing occasionally during vegetation onset, due to a misinterpretation of vegetation dynamics.
Romania appears overall less affected by the dry summer in the anomaly plots in Figure 35, which
relates also with rather normal soil moisture levels for the station Bacles as plotted in Figure 29,
but also with higher values in the SWI than in the in-situ data.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 68 of 78
Figure 35: Monthly SWI anomalies over Europe in 2019.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 69 of 78
5.2.5 Spain
In the first half of 2019, Spain suffered from severe drought conditions on wildfires and crop
production deficits (NF14). The European Forest Fire Information System (EFFIS) reported that
four of the five major fire events in Europe of 2019 were located in Spain. For this situation, the
SWI (T=1) and precipitation anomaly maps (collected in Figure 36) show a long period of low
precipitation, as well as corresponding negative SWI anomalies in the beginning of the year. While
the intermediate relief in April is reflected by the SWI data, the magnitude of May and June rainfall
deficits is not visible in the SWI data.
In November and December 2019, severe storms bringing heavy wind and rain caused widespread
damage and floods (NF15), with a gradient from north-west to south-east. These weather
conditions are well represented in positive SWI anomalies with corresponding precipitation patterns
for that period. Additionally, October shows good agreement between precipitation and SWI
anomalies with lower values in the south and higher values in the north of Spain.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 70 of 78
Figure 36: Monthly SWI anomalies (top) and ECMWF rainfall anomalies (bottom, provided by FAO (via
the GIEWS) over Spain in 2019.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 71 of 78
6 CONCLUSIONS
6.1 GLOBAL ANALYSIS
The SWI and SSF were compared to the output of a land surface model as well as in-situ data.
The model was GLDAS-Noah V2.1, whereas the in-situ data was taken from suitable networks of
the ISMN. The SWI was compared to different soil moisture layers of these reference datasets
(Table 8), whereas the SSF was assessed using the model’s first soil temperature layer. Overall,
no or only little deviations between SWI data from the reference period and the year 2019 were
found, showing overall a weak performance over tropical rainforests, subarctic areas, and deserts,
and a good performance over Europe, subtropical Africa, India, central and eastern Asia, non-arid
Australia and the (non-polar) Americas.
The compliance matrix in Table 18 shows the compliance with the (optimal) GCOS requirements.
For GLDAS, between 6% and 31% of the grid points meet the requirement for the reference
period, whereas between 16% and 58% of grid points meet the requirement for the current
validation period (i.e. 2019), which is slight decrease compared to the analysis for the antecedent
year. The comparisons against ISMN show identical compliance for the lower T-values, but again
slightly poorer results for T=100 than in 2018. As a note, results for the T=100 (as in Table 18) are
relative to the smaller T-values better because there is less variation in deeper soil layers making it
easier to meet the threshold.
Table 18: Compliance matrix for the comparison with GLDAS Noah and ISMN stations. Percentages
of grid points that fulfill the GCOS requirement of 0.04 m³/m³ for the reference period (2007-07-01
until 2018-12-31) (left) / for the current validation period (2019-01-01 until 2019-12-31) (right).
Variable GLDAS ISMN
[%] REF / CUR REF / CUR
SWI T=1 9 / 29 4 / 13
SWI T=20 6 / 16 4 / 13
SWI T=100 31 / 58 12 / 30
Table 19 and Table 20 show the compliance concerning Threshold, Target and GCOS/optimal
levels from analysis against GLDAS and ISMN. For the SWI, the results are again very good and
stable, as results from previous evaluations (Table 2) are at the same level for the Threshold and
Target.
Against GLDAS, also the performance of the SSF was evaluated, where 85% of the pixels meet
the threshold for median of consecutive days wrongly classified in 2019, but almost no pixels meet
the optimum. Compared to 2018 (Table 2) and 2017, these results are slightly improved for
threshold and target, but similar for optimum, and thus are in alignment with evaluations from
previous years.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 72 of 78
Also the metrics against ISMN, collected by Table 20, show values at the same level as in the
previous years, although reporting a slight decrease in compliance scores against 2018 (Table 3),
but almost identical as in 2017.
Table 19: Compliance matrix for the comparison with GLDAS Noah. Percentages of grid points that
fulfill each requirement. Left the reference period (2007-01-01 until 2018-12-31) / right the current
validation period (2019-01-01 until 2019-12-31).
Variable Threshold Target GCOS/optimal
SWI 0.20 m3/m3
SSF 7 days
SWI 0.10 m3/m3
SSF 3 days
SWI 0.04 m3/m3
SSF 1 day
[%] REF / CUR REF / CUR REF / CUR
SWI T=1 100 / 100 80 / 88 9 / 29
SWI T=20 100 / 100 75 / 81 6 / 16
SWI T=100 100 / 100 90 / 96 31 / 58
SSF consecutive days
wrongly classified (max) 13 / 27 6 / 7 1 / 2
SSF consecutive days
wrongly classified (median) 74 / 85 20 / 54 1 / 8
Table 20: Compliance matrix for the comparison with ISMN in-situ data. Percentages of grid points
that fulfill each requirement. Left the reference period (2007-01-01 until 2018-12-31) / right the current
validation period (2019-01-01 until 2019-12-31).
Variable Threshold Target GCOS/optimal
SWI 0.20 m3/m3 SWI 0.10 m3/m3 SWI 0.04 m3/m3
[%] REF / CUR REF / CUR REF / CUR
SWI T=1 96 / 95 56 / 64 4 / 13
SWI T=20 96 / 95 56 / 64 4 / 13
SWI T=100 97 / 97 68 / 76 12 / 30
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 73 of 78
6.2 REGIONAL ANALYSIS
The regional analyses were carried out over southern Africa, Australia, and Europe, comparing
monthly SWI anomalies with the Vegetation Health Index (VHI) and rainfall anomalies. Drought
conditions over central section of subtropical southern Africa threatening the local population from
January to May are captured by the SWI anomalies. Also the outstanding wet conditions over
Tanzania in the fourth quarter of 2019 are reflected by the SWI. The focus on Zambia showed a
good skill of the SWI to map the drought and its severity in early 2019. From June to October, it is
captured only with little magnitude, though. The record-hitting drought and bushfire series of 2019
in Australia are well captured by the SWI anomalies, as well as the wet intermission in
Queensland. These dynamics are also visible the VHI comparison data, in the quarterly areal
means for mainland Australia (see Figure 6). Europe was hit by an exceptional hot spring and
summer in 2019, and the SWI dynamics over Germany, Poland, and France show dry anomalies
accordingly. However, for the other countries of the continent, the SWI only confirm partly with the
reported hot temperatures during the summer months. Probably, this is due to already below long
term average SWI values for the previous years, and the limited timeline for building the anomalies
(starting in 2007). The very dry spring season for Hungary is well reflected, though. Another focus
on Spain shows a comparison between SWI- and rainfall-anomalies. The dynamics are well
aligned, with drier-than-normal conditions in the first quarter of 2019 and a wet intermission phase
in April, recorded in both datasets. The drier conditions from May to July show less or sparsely
reflection in SWI. However, the increasingly wet conditions from October on (with severe storms
and heavy rain falls in November and December 2019), with a movement from north-east Spain to
the south-west are in good accordance again.
In summary, the SWI anomalies show a good performance in indicating geographic patterns of
outstanding hydrological events, but some dynamics (e.g. the prolonged heat wave in Europe) are
not captured with the same magnitude as reported in other studies. It is not clear whether this due
to biased satellite observations, an insufficient anomaly reference baseline, or due to the different
impacts on temperature, vegetation, rainfall and soil moisture variables, which naturally can
happen with different magnitudes.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 74 of 78
7 RECOMMENDATIONS
The overall performance of the SWI and SSF products is consistent with the previous years’
scientific evaluation experiments. No inconsistencies in the SWI were detected in the current SQE
exercise. The detected differences between the reference period and the current validation period
were not consistently in one direction, indicating that they represent year-to-year fluctuations rather
than a systematic difference. The SSF flagging over Siberia, appeared in the previous analysis for
2018 with a turn to a positive trend is supported by results for 2019.
The regional analyses offer interesting insights in medium- and large-scaled hydrological events
and should be continued in coming SQE exercises, as they not only document the skill of the SWI
product to reflect droughts and rainfall anomalies, but also help to improve the understanding of
spatio-temporal dynamics of such processes.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 75 of 78
8 REFERENCES
8.1 SCIENTIFIC LITERATURE
ALBERGEL, C., DORIGO, W., REICHLE, R. H., BALSAMO, G., DE ROSNAY, P., MUÑOZ-SABATER, J., ISAKSEN, L., DE JEU, R. & WAGNER, W. 2013. Skill and Global Trend Analysis of Soil Moisture from Reanalyses and Microwave Remote Sensing. Journal of Hydrometeorology, 14, 1259-1277.
BELL, J. E., PALECKI, M. A., BAKER, C. B., COLLINS, W. G., LAWRIMORE, J. H., LEEPER, R. D., HALL, M. E., KOCHENDORFER, J., MEYERS, T. P., WILSON, T. & DIAMOND, H. J. 2013. U.S. Climate Reference Network Soil Moisture and Temperature Observations. Journal of Hydrometeorology, 14, 977-988.
BROCCA, L., HASENAUER, S., LACAVA, T., MELONE, F., MORAMARCO, T., WAGNER, W., DORIGO, W., MATGEN, P., MARTÍNEZ-FERNÁNDEZ, J., LLORENS, P., LATRON, J., MARTIN, C. & BITTELLI, M. 2011. Soil moisture estimation through ASCAT and AMSR-E sensors: An intercomparison and validation study across Europe. Remote Sensing of Environment, 115, 3390-3408.
CHEN, Y., YANG, K., QIN, J., ZHAO, L., TANG, W. & HAN, M. 2013. Evaluation of AMSR-E retrievals and GLDAS simulations against observations of a soil moisture network on the central Tibetan Plateau. Journal of Geophysical Research: Atmospheres, 118, 4466-4475.
DIAMOND, H. J., KARL, T. R., PALECKI, M. A., BAKER, C. B., BELL, J. E., LEEPER, R. D., EASTERLING, D. R., LAWRIMORE, J. H., MEYERS, T. P. & HELFERT, M. R. 2013. US Climate Reference Network after one decade of operations: Status and assessment. Bulletin of the American Meteorological Society, 94, 485-498.
DORIGO, W. A., SCIPAL, K., PARINUSSA, R. M., LIU, Y. Y., WAGNER, W., DE JEU, R. A. M. & NAEIMI, V. 2010. Error characterisation of global active and passive microwave soil moisture datasets. Hydrol. Earth Syst. Sci., 14, 2605-2616.
DORIGO, W. A., XAVER, A., VREUGDENHIL, M., GRUBER, A., HEGYIOVÁ, A., SANCHIS-DUFAU, A. D., ZAMOJSKI, D., CORDES, C., WAGNER, W. & DRUSCH, M. 2013. Global Automated Quality Control of In-situ Soil Moisture Data from the International Soil Moisture Network. Vadose Zone Journal, 12.
FISHER, R. A. 1915. Frequency Distribution of the Values of the Correlation Coefficient in Samples from an Indefinitely Large Population. Biometrika, 10, 507-521.
KOGAN, F. N. 1995. Application of vegetation index and brightness temperature for drought detection. Advances in Space Research 15.11: 91-100.
KOGAN, F., GITELSON, A., ZAKARIN, E., SPIVAK, L. AND LEBED, L., 2003. AVHRR-based spectral vegetation index for quantitative assessment of vegetation state and productivity. Photogrammetric Engineering & Remote Sensing, 69(8), pp.899-906.
LEAVESLEY, G., DAVID, O., GAREN, D., LEA, J., MARRON, J., PAGANO, T., PERKINS, T. & STROBEL, M. A modeling framework for improved agricultural water supply forecasting. AGU Fall Meeting Abstracts, 2008. 0497.
LIU, Y. Y., PARINUSSA, R. M., DORIGO, W. A., DE JEU, R. A. M., WAGNER, W., VAN DIJK, A. I. J. M., MCCABE, M. F. & EVANS, J. P. 2011. Developing an improved soil moisture dataset by blending passive and active microwave satellite-based retrievals. Hydrol. Earth Syst. Sci., 15, 425-436.
NAEIMI, V., PAULIK, C., BARTSCH, A., WAGNER, W., KIDD, R. A., PARK, S.-E., ELGER, K. & BOIKE, J. 2012. ASCAT Surface State Flag (SSF): Extracting Information on Surface Freeze/Thaw Conditions From Backscatter Data Using an Empirical Threshold-Analysis
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 76 of 78
Algorithm. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 50, 2566-2582.
NASA 2014. SMAP Handbook, Mapping Soil Moisture and Freeze/Thaw from Space. NASA. PAULIK, C., DORIGO, W., WAGNER, W., & KIDD, R., 2014. Validation of the ASCAT Soil Water
Index using in situ data from the International Soil Moisture Network. International Journal of Applied Earth Observation and Geoinformation, 30, 1-8
RODELL, M., HOUSER, P. R., JAMBOR, U., GOTTSCHALCK, J., MITCHELL, K., MENG, C. J., ARSENAULT, K., COSGROVE, B., RADAKOVICH, J., BOSILOVICH, M., ENTIN*, J. K., WALKER, J. P., LOHMANN, D. & TOLL, D. 2004. The Global Land Data Assimilation System. Bulletin of the American Meteorological Society, 85, 381-394.
ROJAS, OSCAR, YANYUN LI, AND RENATO CUMANI. Understanding the Drought Impact of El Niño on the Global Agricultural Areas: An Assessment Using FAO's Agricultural Stress Index (ASI). 2014.
SMITH, A. B., J. P. WALKER, A. W. WESTERN, R. I. YOUNG, K. M. ELLETT, R. C. PIPUNIC, R. B. GRAYSON, L. SIRIWARDENA, F. H. S. CHIEW, AND H. RICHTER, 2012. The Murrumbidgee soil moisture monitoring network data set, Water Resour. Res., 48, W07701,
YOUNGWOOK, K., KIMBALL, J. S., MCDONALD, K. C. & GLASSY, J. 2011. Developing a Global Data Record of Daily Landscape Freeze/Thaw Status Using Satellite Passive Microwave Remote Sensing. Geoscience and Remote Sensing, IEEE Transactions on, 49, 949-960.
ZACHARIAS, S., BOGENA, H., SAMANIEGO, L., MAUDER, M., FUß, R., PÜTZ, T., FRENZEL, M., SCHWANK, M., BAESSLER, C., BUTTERBACH-BAHL, K., BENS, O., BORG, E., BRAUER, A., DIETRICH, P., HAJNSEK, I., HELLE, G., KIESE, R., KUNSTMANN, H., KLOTZ, S., MUNCH, J. C., PAPEN, H., PRIESACK, E., SCHMID, H. P., STEINBRECHER, R., ROSENBAUM, U., TEUTSCH, G. & VEREECKEN, H. 2011. A Network of Terrestrial Environmental Observatories in Germany All rights reserved. No part of this periodical may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. gsvadzone, 10, 955-973.
ZREDA, M., DESILETS, D., FERRÉ, T. P. A. & SCOTT, R. L. 2008. Measuring soil moisture content non-invasively at intermediate spatial scale using cosmic-ray neutrons. Geophysical Research Letters, 35, L21402.
8.2 NEWS FOOTAGE
NF1 Drought in Africa leaves 45 million in need across 14 countries.
https://www.thenewhumanitarian.org/analysis/2019/06/10/drought-africa-2019-45-million-in-need
Accessed: 2020-03-17.
NF2 As climate shocks intensify, un food agencies urge more support for southern Africa's
hungry people.
https://reliefweb.int/report/angola/climate-shocks-intensify-un-food-agencies-urge-more-support-southern-africa-s-hungry
Accessed: 2020-03-17.
NF3 WFP southern Africa seasonal update - October 2019 - February 2020.
https://reliefweb.int/report/world/wfp-southern-africa-seasonal-update-october-2019-february-2020
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 77 of 78
Accessed: 2020-03-17.
NF4 Southern Africa: Drought - 2018-2020.
https://reliefweb.int/disaster/dr-2018-000429-zwe
Accessed: 2020-03-17.
NF5 2019: Natural disasters claim more than 1200 lives across east and southern Africa.
https://reliefweb.int/report/world/2019-natural-disasters-claim-more-1200-lives-across-east-and-southern-africa
Accessed: 2020-03-17.
NF6 Australia fires were far worse than any prediction.
https://www.bbc.com/news/science-environment-51590080
Accessed: 2020-03-17.
NF7 Queensland floods: the water journey to Kati Thanda-Lake Eyre.
http://media.bom.gov.au/social/blog/2059/queensland-floods-the-water-journey-to-kati-thanda-lake-eyre/
Accessed: 2020-03-17.
NF8 Annual climate statement 2019.
http://www.bom.gov.au/climate/current/annual/aus/
Accessed: 2020-03-17.
NF9 JRC MARS Bulletin vol 27 no 11.
https://ec.europa.eu/jrc/sites/jrcsh/files/jrc-mars-bulletin-vol27-no11.pdf
Accessed: 2020-03-17.
NF10 A second scorching heatwave in Europe.
https://earthobservatory.nasa.gov/images/145377/a-second-scorching-heatwave-in-europe?src=nha
Accessed: 2020-03-12.
NF11 Heat wave strikes Europe.
https://landsaf.ipma.pt/en/newsmedia/news-show-cases-2/heatwave-strikes-europe-1/
Accessed: 2020-03-17.
NF12 Surface air temperature for December 2019.
https://climate.copernicus.eu/surface-air-temperature-december-2019.
Accessed: 2020-03-17.
NF13 Hungary especially endangered by drought, WWF says.
https://bbj.hu/energy-environment/hungary-especially-endangered-by-drought-wwf-says_169953
Accessed: 2020-03-19.
Copernicus Global Land Operations – Lot 1 Date Issued: 25.03.2020 Issue: I1.00
Document-No. CGLOPS1_SQE2019-SWIV3 © C-GLOPS Lot1 consortium
Issue: I1.00 Date: 25.03.2020 Page: 78 of 78
NF14 Spain battles biggest wildfires in 20 years as heat wave grips Europe.
https://www.theguardian.com/world/2019/jun/27/hundreds-of-firefighters-tackle-blaze-in-north-east-spain
Accessed: 2020-03-17.
NF15 Nine dead as extreme weather hits Spain, Portugal and France.
https://newseu.cgtn.com/news/2019-12-23/Eight-dead-as-extreme-weather-hits-Spain-Portugal-and-France-
MDOladmGmA/index.html
Accessed: 2020-03-17.