Satellite Data Provided by the NASA Ocean Biology Processing Group

(OBPG)

Deep-Water Chlorophyll Imagescommon-bin 12-day composite, Winter 2002

0.01-1 mg/m^3

SeaWiFS R5

MODIS/Aqua R1

OBPG Satellite Data

- MODIS Aqua (2002-present)

- MODIS Terra (2000-present)

- SeaWiFS (1997-present)

- OCTS (1996-1997)

- CZCS (1978-1986)

HDF File Format

• Hierarchical Data Format (HDF) is a data file format

• OBPG data files stored as HDF (L1A, L1B, L2, L3)

• HDF can be thought of as a “container” to store a variety of data types and meta data in a single file

Example HDF file

HDF-EOS

• OBPG files stored in HDF-EOS format:– specialized form of HDF– created by NASA in 1993– standard format for all EOS data

• SeaDAS is compatible with OBPG HDF-EOS

MODIS Data Levels

Level 0

Level 1A

Level 1B

Level 2

Level 3 binned

Level 3 mapped

Data Levels & Flow• Level 0

– raw digital counts– native binary format

• Level 1A – raw digital counts– HDF formatted

• Level 1B

– calibrated reflectances

– converted telemetry

• Level 2

– geolocated geophysical products for each pixel

• Ancillary data– wind speed– surface pressure – total ozone– Reynolds SST

• GEO – geolocation– radiant path geometry

• ATT & EPH – spacecraft attitude– spacecraft position

Data Levels & Flow• Level 0

– raw digital counts– native binary format

• Level 1A – raw digital counts– HDF formatted

• Level 1B

– calibrated reflectances

– converted telemetry

• Level 2

– geolocated geophysical products for each pixel

• Ancillary data– wind speed– surface pressure – total ozone– Reynolds SST

• GEO – geolocation– radiant path geometry

• ATT & EPH – spacecraft attitude– spacecraft position

• Level 1A Subset – reduced to standard

ocean bands only

Level 1A (HDF)

– raw radiance counts– Onboard calibration

data– navigation data– spacecraft telemetry– instrument telemetry

A2006027191500.L1A_LAC

Level 1B (HDF)

– calibrated, geolocated at-aperture radiancesi.e. Top of Atmosphere (TOA) radiances

– quality flags– error estimates– calibration data

A2006027191500.L1B_LACA2006027191500.L1B_HKMA2006027191500.L1B_QKM

Level 2 (HDF)

– geophysical values for each pixel

– products include chlorophyll-a, SST, nLw’s, K490, flags

– geolocation data

A2006027191500.L2_LACA2006027191500.L2_LAC_SSTA2006027191500.L2_LAC_SST4

Chlorophyll-a SST

Level 2 Flags

Level 2 Standard MODIS Ocean Products

• Ocean Color (day only)– Normalized water-leaving radiances - nLw() – Chlorophyll - Ca

– Diffuse attenuation - Kd(490)– Aerosol type and concentration

• Optical thickness - a

• Ångström exponent• Atmospheric epsilon

– Processing flags• Cloud, land, glint, atmfail, atmwarn, chlfail, chlwarn, etc.

• Ocean Temperature– Long-wave SST (11-12 m) (day and night)– Short-wave SST (3.9 - 4.0 m) (night only)– SST quality level (0-4)

Level 2 Non-standard Ocean Products

• Ocean Color– Alternate Ca and Kd algorithms– Chlorophyll fluorescence, FLH– Particulate inorganic carbon, Calcite– Inherent optical properties (various bio-optical models)

• absorption (total, phaeophytin, dissolved matter)• backscatter (total, particulate)

– Photosynthetically active radiation, iPAR, PAR (coming)– Euphotic depth (Zeu, Zsd)– Subsurface PAR at 1st optical depth, Kd(PAR)– Intermediate atmospheric correction products

• Ocean Temperature– Brightness temperatures

After Level-2…

Level-2

L3 Binning

Level 3 Binned data products (HDF)

– data composites– binned Level 2 data (spatially and temporally)– arithmetic means with statistics for each bin– daily, 8-day, monthly, yearly periods– 0.5, 1, 2, 4, 9, 36 km2 resolutions– stored as one file or a main file with subordinates– useful for scientific research

A20071822007212.L3b_MO.main

Sinusoidal Equal-Area Projection

Level 3 SMI (mapped) (HDF)

– Mapped L3 binned product– Data is scaled to 16-bit values– Equidistant cylindrical projection

A20071822007212.L3m_MO_CHLO_4

Ocean coverage over time for binned files

Ancillary data products(HDF and other formats)

– NRT data and refined ancillary data

– used in various stages of processing:

• Geolocation (attitude and ephemeris)• Calibration (coefficient look-up tables)• atmospheric correction (MET/OZONE/SST)

Band Number

Wavelength (nm)

Band Width

(nm)

Spatial Resolution

(m)

SNR at Ltyp

Ltyp mW cm-2

m-1 sr-1

Lmax mW cm-2

m-1 sr-1 8 412 15 1000 1773 7.84 26.9 9 443 10 1000 2253 6.99 19.0

10 488 10 1000 2270 5.38 14.0 11 531 10 1000 2183 3.87 11.1 12 551 10 1000 2200 3.50 8.8 13 667 10 1000 1962 1.47 4.2 14 678 10 1000 2175 1.38 4.2 15 748 10 1000 1371 0.889 3.5 16 869 15 1000 1112 0.460 2.5

Operational MODIS Ocean Band Suite

Band Number

Wavelength (nm)

Band Width

(nm)

Spatial Resolution

(m)

NEdT

22 3959 60 1000 0.07 23 4050 60 1000 0.07 31 11000 60 1000 0.05 32 12000 60 1000 0.05

VIS/NIR

Ocean Color

Thermal

SST

Notes

Ingest queuetable

L0Ingest

process

L1A-L1B(MOD_PR02)

L1B-L2(MSl12)

MODISA L2table

Archive - DistribServer

L3BIN(l2bin)

MODISA L3-bintable

Archive - DistribServer

MODISA L3-maptable

Archive - DistribServer

Operational MODIS-Aqua Data Flow

Software process

Hardware system

Database table

Ancillary input data

SensorCAL

Sensorattribs

Atmcorr

BrowserCGI /httpd

Ocean ColorWeb Server

UserCommunity

MySQLDB

Ozone

L3MAP(smigen)

MET

MET, Ozone, and OISST data

are dynamically selected for each L1A granule

Product meta data are

populated from production DB

April 21, 2023

OISST

Level-0Ingest

MODISA L0table

Archive - DistribServer

ATT/EPHIngest

process MODISA attephtable

Archive - DistribServer

L0-L1A(MOD_PR01)

Geo-Location(MOD_PR03)

MODISA L1table

Archive - DistribServer

Full-resolution day- and nighttime via SEN (~60 GB per

day)

NOAA RealtimeSystem

NASA EDOSSystem

ODPS Data Processing System Current Components

Processing Cluster 34 processing nodes 1.5 TB

Ingest Servers2 SeaSpace groundstations

5 storage nodes6 TB

Distribution Servers (ftp)6 storage nodes

7.2 TBDistribution Servers (web)1 large

server – 3 TB 58 storage nodes

69.6 TB

Testing Cluster13 test nodes

2 TB

Network Support Systems

Database Server1 large server

876 GB

Backup Servers1 large server

876 GB+2 TB tape5 storage nodes

6 TB

Extreme NetworksBlack Diamond 6816

Gigabit Ethernet switch

Development Servers1 processing node

2 storage nodes2.4 TB

User Desktops

Cal/Val & QC Systems

Mission Operations Systems

Data Processing and Storage

• database controlled, distributed processing and storage system

• many Linux-based, processing nodes (34 dual 3GHz Xeon)

• 200X processing for MODIS/Aqua ocean color

• 4000X processing through-put for SeaWiFS ocean color

• 9-month OCTS mission can be reprocessed over lunch

• all data online for direct access (100TB RAID-5 array)

• capacity for global mission testing prior to reprocessing, allowing opportunity for community participation in reprocessing decisions

• direct and near real-time user access to data archives

processing node storage nodedatabase server

gigabit ethernet

Supermicro 6024 rack mount server

2 Intel Xeon CPUs 3.06 Ghz

8 1 GB DDR2 RAM modules

2 Intel Gigabit Ethernet network interfaces

1 Seagate 80 GB EIDE disk (system)

5 Maxtor 73.4 GB Atlas 10K IV Ultra320 SCSI disks (processing)

1 CD-ROM (low profile)

1 Supermicro X5DP8-G2 motherboard with built-in Ultra320 SCSI controller

1 Supermicro 6024H 2U rack mount server casewith dual power supplies

Operating system: Linux

Ocean Biology Data Processing System - Processing Node

NAS 1.2 TB rack mount server

1 Intel Pentium-4 CPU 3.06 Ghz

2 512 MB DDR2 RAM modules

1 Gigabit Ethernet network interface

1 Seagate 80 GB EIDE disk (system)

8 Maxtor 250 GB MaxLine Plus II EIDE disks(RAID 5, 6 data + 2 hot spares)

1 CD-ROM (low profile)

1 Asus P4C800-E Motherboard

1 3ware Escalade RAID controller for 8 drives

1 Comix Computer RM-3U9SCA (IDE) ATX rackmount chassis 3 U, 9 hot swappabledrive bays

Operating system: Linux

Ocean Biology Data Processing System - Storage Node

SunFire V880 rack mounted

4-8 UltraSPARC-III+ processors

8-16 GB RAM

6-12 73 GB 10K RPM FibreChannel disks

2 Gigabit Ethernet nework interfaces

1-5 Fast Ethernet network interfaces

1 External FibreChannel adapter

Operating system: Solaris

Ocean Biology Data Processing System - Large Server

Extreme Networks Black Diamond 6816

768 Gbps total switching capacityRoute/filter/forward 192 million pps

160 GigE ports in current configuration

All modules are hot swappable

4 MSM management modules

2 G8Ti 8 port Gigabit Ethernet modules

6 G24T3 24 port Gigabit Ethernet modules

4 Hot-swappable power supplies

Ocean Biology Data Processing System - Network Switch