application of hdf/hdf-eos data to atmospheric and climate sciences at university of illinois
TRANSCRIPT
Guangyu Zhao Prof. Larry Di Girolamo
Department of Atmospheric Sciences University of Illinois at Urbana-Champaign
Experience with HDF/HDF-EOS data1. HDF-EOS Datasets MODIS
ASTER
One orbit of MISR level-1 radiance product ~= 600 (Mb) x 1 + 200 (Mb) x 8 ~= 2.2 Gb
Level-1 data
Level-2 data
Level-3 data (only monthly)
One orbit of MISR level-2 cloud and aerosol products~= 1 Gb
As of today, over 56300 orbits of data have been collected. In total ~= 200 Tb
11-year ~= 100 Gb
Experience with HDF/HDF-EOS data2. HDF data processing and visualization
ASTER
•Format Conversion•HDF ↔ Binary •HDF → Netcdf
•Re-projection •Data fusion
•Visualization •Image display and manipulation •Stereoscopic display
Experience with HDF/HDF-EOS data3. Computer languages
ASTER
•C/C++, Fortran (…<-shell scripts), Python•Matlab
4. Computer support from JPL
•HDFscan, MISR toolbox…
Example : winter global map of cloud coverproducing
Data ordering from DAAC
3 month Level-1B data [2.5 Tb]
Applying cloud mask algorithm
Leve-2 cloud mask product [75 Gb]
Level-3 cloud cover [20 Mb]
Example : winter global map of cloud covervalidating
Data ordering from DAAC
3 month Level-1B data [2.5 Tb]
Applying cloud mask algorithm
Leve-2 cloud mask product [75 Gb]
Level-3 cloud cover [20 Mb]
MISR MODIS
Computational Prospection
•Data I/O SpeedAbout 30% of computation time is spent on I/O…
•File size> 50% of content not used
•VisualizationSynchronizing multiply fields from multiply HDF filesSimple manipulation (+, -, x, G(L)T_mask,…) Fast, Fast, Fast!!!