xiwu zhan umbc-gest/nasa-gsfc, code 974.1, greenbelt, md paul houser
Post on 10-Jan-2016
43 Views
Preview:
DESCRIPTION
TRANSCRIPT
AGU 2004 Fall Meeting, April 21, 2023 Slide 1
Jointly Retrieving Surface Soil Moisture from Active and Passive Microwave Observations using
Cubist Data-Mining
Xiwu ZhanUMBC-GEST/NASA-GSFC, Code 974.1, Greenbelt, MD
Paul HouserNASA-GSFC HSB, Code 974, Greenbelt, MD
Jeffrey WalkerUniversity of Melbourne, Victoria, Australia
AGU 2004 Fall Meeting, April 21, 2023 Slide 2
OBJECTIVES
There will be high resolution (up to 1km) radar backscatter observations of land surface soil moisture from NASA ESSP Hydros mission. Radiation transfer models are usually inversed to retrieve soil moisture value. Can data-mining provide an alternative?
There are many global high resolution satellite data sets of land surface parameters that are related to soil moisture. Can we use them to derive soil moisture when radar data are not available?
What are the accuracies of these alternative soil moisture retrieval methods?
AGU 2004 Fall Meeting, April 21, 2023 Slide 3
METHODOLOGY
Use the 1km geophysical and biophysical data fields and microwave emission and backscatter models (MEBM) from the Observation System Simulation Experiment (OSSE in Crow et al, 2004, Zhan et al, 2005) of NASA ESSP Hydros Mission;
Inverse the simulated radiometer and radar observations with the MEBM for soil moisture retrievals;
Use the Cubist data-mining tool to generate Cubist models and use the models to obtain fine resolution soil moisture retrievals
Use the update equation of the Extended Kalman Filter (EKF) to combine course resolution and fine resolution soil moisture estimations for an optimal soil moisture retrieval data product;
Compute the RMSEs of soil moisture retrievals of the three methods (INV, Cubist, & EKF) against the original soil moisture data fields.
AGU 2004 Fall Meeting, April 21, 2023 Slide 4
Hydros: Hydrosphere States Mission
Spinning 6m dish
A NASA Earth System Science Pathfinder mission;Surface soil moisture w/ 4%vol. accuracy and Freeze/Thaw state transitions;Revisit time: Global 3 days, boreal area 2 daysL-band (1.41GHz) Radiometer sensing 40km brightness temp. with H & V polarization;L-band (1.26GHz) Radar measuring 1-3km backscatters with hh, vv, hv polarization;Soil moisture products: 3km radar retrievals, 40km radiometer retrievals, 10km radar and radiometer combined retrievals and 5km 4DDA results.
AGU 2004 Fall Meeting, April 21, 2023 Slide 5
Hydros OSSE: Data Layers
36 km TBh, TBv data from Hydros radiometer
simulator
9 km soil moisture retrieval
product
3 km hh, vv, hv data from Hydros radar simulator
1 km soil moisture data
from nature run
1 36km pixel
9 9km pixels
144 3km pixels
1296 1km pixels
AGU 2004 Fall Meeting, April 21, 2023 Slide 6
“Truth” Validation
RadiometerInversion
3/9/36km SM
Retrieval Error
1 km Nature Run(Input: LC, ST, NDVI, Rainfall, Met data)
3/9km
Hydros Instrument Simulator
1km SM
36km Tb
Aggregateto
3/9/36km
1km SM
3/9/36kmSM
3/9km SM
EKF Algorithm
36km SM Tb & Errors
Radiometer TbForward Model
Radar Forward Model
EKF Algorithm
Innovations
Calculate optimized SM
Error Models
Based on redAnd white
noise
3/9km
3/9/36km SM
36km Radiometer
Forward Model
Tb Iteration
36km SM
36km Tb
Observations
From Hydrosinstrumentsimulator
Background
From radio-meter
inversion
1km Tsoil 1km Tskin
36km
SM
Data Flow for Using EKF to Retrieve SM from Tb & Observations
RadarInversion
3km Radar
Forward Model
Iteration
1km Radiometer TbForward Model
Aggregateto 36km
White Noise
1km Radar Forward Model
Aggregateto 3/9km
Red Noise
White Noise
AGU 2004 Fall Meeting, April 21, 2023 Slide 7
“Truth” Validation
RadiometerInversion
3/9/36km SM
Retrieval Error
1 km Nature Run(Input: LC, ST, NDVI, Rainfall, Met data)
3/9km
Hydros Instrument Simulator
1km SM
36km Tb
Aggregateto
3/9/36km
1km SM
3/9/36kmSM
1km SM
EKF Algorithm
36km SM 1km SMErrors
EKF Algorithm
Innovations
Calculate optimized SM
Error Models
Based on redAnd white
noise
1km ndvi, Ts/
1/3/9/36km SM
36km Radiometer
Forward Model
Tb Iteration
36km SM
36km Tb
Observations
From Cubist model
Background
From radio-meter
inversion
1km Tsoil 1km Tskin
36km
SM Cubist
Model
1 km CubistModels
ndvi, Ts/
Observ.
1km SM
1km Radiometer TbForward Model
Aggregateto 36km
White Noise
1km Radar Forward Model
Aggregateto 3/9km
Red Noise
White Noise
Data Flow for Using EKF to Retrieve SM from Tb & other Observations
AGU 2004 Fall Meeting, April 21, 2023 Slide 8
Radar observational are not handily available for retrieving soil moisture before the launch of Hydros in 2010: spatial coverage, revisit time;
Radar radiation transfer models are not as mature as radiometer models for inversing soil moisture;
Why Alternative for Radar Model?
T*
NDVI*Low Soil Moisturehigh
Soil Moisture
NDVIo
NDVIs
To Ts
Visible/Infrared observations such as NDVI, LST and albedo from MODIS, Landsat and future VIIRS on NPOESS are available everyday at high spatial resolutions;
The “Universal Triangle” relationships between soil moisture and the visible/infrared observations have been documented in literature for many years.
AGU 2004 Fall Meeting, April 21, 2023 Slide 9
Cubist is used to build regression tree model of the relationships between soil moisture and its related land surface parameters such radar backscatter, or, NDVI, surface temperature and albedo;
Regression tree is similar to the decision tree classifier in that it recursively splits training samples into subsets, two at each split;
Instead of assigning class labels to the subsets, it develops a linear regression model for each of them;
Each splitting is made such that the combined residual error of the models for the two subsets is substantially lower than the residual error of the single best linear model for the samples in the two subsets, and that the combined residual error of the split is the minimum of all possible splits
Cubist: a Data-mining Computer Tool
AGU 2004 Fall Meeting, April 21, 2023 Slide 10
Noise in data: Sigma - .5dB, Tb – 1K, ndvi – 10%, Ts-.5K, Roughness-5%, VWC-10%
Cubist Model Compared with Radar Model
0.00
0.05
0.10
0.15
0.20
145 150 155 160 165 170 175 180
Day of Year
Cubist Sigmas
Sigma Inversion
Cubist Sigmas vs Sigma Inversion
Tb Inversion
Low noise data
For low noise data, Cubist model of radar backscatters may reduce the RMSEs of radar model inversions by about 1-2 %v/v;
AGU 2004 Fall Meeting, April 21, 2023 Slide 11
Noise in data: Sigma - 1dB, Tb – 1K, ndvi – 20%, Ts-1K, Roughness-10%, VWC-20%
Cubist Model Compared with Radar Model
0.00
0.05
0.10
0.15
0.20
145 150 155 160 165 170 175 180
Day of Year
Cubist Sigmas
Sigma Inversion
Cubist Sigmas vs Sigma Inversion
Tb Inversion
High noise data
For high noise data, Cubist model of radar backscatters could reduce the RMSEs of radar model inversions by about 3-4 %v/v;
AGU 2004 Fall Meeting, April 21, 2023 Slide 12
Cubist Model Applicability/Stability
0.000
0.020
0.040
0.060
0.080
0.100
0.120
145 150 155 160 165 170 175 180
Day of Year
Day 166 Model
Daily Cubist Model
Cubist Sigma Model Stability
Day 146 Model
High noise data
Low noise dataDay 146 H Model
Day 166 L Model
Cubist model of radar backscatters using same day or other day training data results very similar accuracy;
Cubist model based on low noise data produces almost the same accuracy as based on high noise data, and the opposite is true too.
AGU 2004 Fall Meeting, April 21, 2023 Slide 13
Cubist Model Using Visible/IR Data
0.00
0.05
0.10
0.15
0.20
145 150 155 160 165 170 175 180
Day of Year
Cubist ndvi & Ts
Sigma Inversion
Tb Inversion
Low noise dataCubist ndvi & Ts vs Sigma Inversion
If the data noises are low, RMSEs of soil moisture retrievals from Cubist model using only visible/IR observations may be 3-5%v/v higher than radar backscatter model inversions.
AGU 2004 Fall Meeting, April 21, 2023 Slide 14
Cubist Model Using Visible/IR Data
0.00
0.05
0.10
0.15
0.20
145 150 155 160 165 170 175 180
Day of Year
Cubist ndvi & Ts
Sigma Inversion
Cubist ndvi & Ts vs Sigma Inversion
Tb Inversion
High noise data
RMSEs of soil moisture retrievals from Cubist model using only visible/IR observations may be 1-3%v/v higher than radar backscatter model inversions based on the high noise data. Radar observations are apparently more reliable than visible/IR obs as long as a radar model is known.
AGU 2004 Fall Meeting, April 21, 2023 Slide 15
][)](([ fbba xXhZKXX
Extended Kalman Filter for SM Retrieval
RHPH
PHK
T
T
x
XhH
b
)(
1434144,1,1,1, ... x
ThvhvvvhhBvBh TTZ
1434)(...)()()()()()( 144,1,1,1, x
TchvchvcvvchhcBhcBh xxxxxTxTXh
Xa – soil moisture retrievalXb – background SMK – Kalman gainZ – observationsh(X) – obs functionH – obs operatorP – bg error covarianceR – obs error covariance
Kalman filter is a statistical data assimilation technique that calculates an optimal observation correction term to the background value based on the relative magnitude of the error covariances of the observations and the background.
AGU 2004 Fall Meeting, April 21, 2023 Slide 16
0.00
0.05
0.10
0.15
0.20
145 150 155 160 165 170 175 180
Day of Year
Cubist Sigmas
Sigma Inversion
EKF Combining Cubist and Tb Inversion
Tb Inversion
Low noise data
EKF Tb + Cubist
EKF Application Result - 1
EKF retrievals using Cubist model of 1km radar sigmas and 36km Tb inversion are marginally better than Cubist model estimates when the error covariance difference between Tb inversion and Cubist model is large;
AGU 2004 Fall Meeting, April 21, 2023 Slide 17
0.00
0.05
0.10
0.15
0.20
145 150 155 160 165 170 175 180
Day of Year
Cubist Sigmas
Sigma InversionTb Inversion
High noise data
EKF Tb + Cubist
EKF Combining Cubist and Tb Inversion
EKF Application Result - 2
When the error covariance difference between Tb inversion and Cubist model are smaller, the advantage of the EKF retrievals is larger (2-4% less RMSE);
AGU 2004 Fall Meeting, April 21, 2023 Slide 18
0.00
0.05
0.10
0.15
0.20
145 150 155 160 165 170 175 180
Day of Year
Cubist ndvi & Ts
Sigma Inversion
EKF Combining Cubist and Tb Inversion
Tb Inversion
Low noise data
EKF Tb + Cubist
EKF Application Result - 3
EKF retrievals using Cubist model of 1km visible/IR obs and 36km Tb inversion are better than both Cubist model estimates and Tb inversion (2-3% less RMSE). But the Cubist model does not produce retrievals as good as using radar backscatter (4-5% larger RMSE).
AGU 2004 Fall Meeting, April 21, 2023 Slide 19
0.00
0.05
0.10
0.15
0.20
145 150 155 160 165 170 175 180
Day of Year
Cubist ndvi & Ts
Sigma Inversion
EKF Combining Cubist and Tb Inversion
Tb Inversion
High noise data
EKF Tb + Cubist
EKF Application Result - 4
EKF retrievals using Cubist model of 1km visible/IR obs and 36km Tb inversion are marginally better than both Cubist model estimates and Tb inversion for high noise data. Cubist model of ndvi and Ts is not as good as radar backscatter models (1-3% higher RMSE).
AGU 2004 Fall Meeting, April 21, 2023 Slide 20
Error Distribution of SM Retrievals
36km Tb Inversion 1km Cubist Sigma Model
1km Radar Sigma Inversion 1km EKF Tb Inv + Cubist Sigma
9.1%RMSE in %v/v
3.3%
3.2%5.3%
Low noise data, Day 155
AGU 2004 Fall Meeting, April 21, 2023 Slide 21
Error Distribution of SM Retrievals
36km Tb Inversion 1km Cubist Sigma Model
1km Radar Sigma Inversion 1km EKF Tb Inv + Cubist Sigma
9.1%RMSE in %v/v
7.1%
5.9%10.9%
High noise data, Day 155
AGU 2004 Fall Meeting, April 21, 2023 Slide 22
SUMMARYWhen radar backscatter (sigma) observations are available, a Cubist model of sigmas could be used to retrieve soil moisture with better accuracy (1-4% less RMSE) than a radar backscatter model.
The same set of equations of the Cubist model based on one set of training data may be applicable to other sets of data. Thus a Cubist model could be an alternative to a radar backscatter model based on the data we used.
EKF Data Assimilation method can combine high resolution and low resolution soil moisture estimations and improve retrieval accuracy.
Based on the NDVI and Ts used currently, a Cubist model of NDVI and Ts is not as reliable as a Cubist model of radar sigma data. The difference of RMSE could be as high as 1-5%.
However, the radar sigma inversions were obtained with the same radar backscatter model used to generate the radar backscatter data with noises added while the impacts of NDVI and Ts in the radar backscatter model may be as significant as in reality. Thus further investigations using real high resolution soil moisture, and visible/IP observational data are still needed.
AGU 2004 Fall Meeting, April 21, 2023 Slide 23
Combining Optical/IR RS and MW RS forHigh Resolution Soil Moisture
NDVI,LST,A or Sigmas – SM Relationships
AMSR-E/CMIS SMOS, Hydros,
MW Observations
MODIS/VIIRSTM, SPOT, Hydros
Observations of NDVI, LST, A or Sigmas
Course Rez (20-50km)
Soil Moisture Retrievals
Soil Moisture Truth Data from Airplane/Ground
Observations
High Rez (30m-3km)
Soil Moisture Retrievals
EKF Data Assimilation Algorithms
Cubist: Data Mining Tools
top related