UNCLASSIFIED

Environmental Data ServerEnvironmental Data Server (EDS) (EDS)

support of USCG’s SAROPSsupport of USCG’s SAROPS(Search And Rescue(Search And Rescue

Optimal Planning System) Optimal Planning System)Art Allen

USCG Office of Search and Rescue

Eoin Howlett Applied Science Associates

UNCLASSIFIED

USCG Area of Responsibility

UNCLASSIFIED

EDS Data Set Display, ArcGIS

Example of an aggregated NetCDF file (ADCIRC and NCOM) returned by a request to the EDS from an ArcGIS client.

ADCIRC – works better for coastal zonesNCOM – works better for deep water

UNCLASSIFIED

EDS and Connected Systems Overview

Data Request Interface (DRI) – EDS Web ServicesData Request Interface (DRI) – EDS Web Services

Data Request Services (DRS)

(EDS Request Processing Service Array)

Data Request Services (DRS)

(EDS Request Processing Service Array)

Data StoreData Store

Data Acquisition and Normalization Services

(DAN)(EDS Catalog Server)

Data Acquisition and Normalization Services

(DAN)(EDS Catalog Server)

External Data Sources

Environmental Data Server

SAROPSSAROPS PORTALPORTAL WMS CLIENTWMS CLIENT OTHEROTHER

UNCLASSIFIED

DispatchService

DispatchService

AcquisitionScheduler

Service

AcquisitionScheduler

Service

Processor,NOGAPS

Processor,NOGAPS

Processor,GFS

Processor,GFS

Processor,FTP

Processor,FTP

DAN

Qu

eues

Qu

eues

DATA ACQUISITION AND NORMALIZATION SERVICE ARRAY (high-level)

Processor,NDFD

Processor,NDFD

Processor,HYCOM

Processor,HYCOM

Processor,OPENDAPProcessor,OPENDAP

QCService

QCService

NotificationManagerService

NotificationManagerService

PORTAL SERVICESPORTAL

SERVICES

QueueQueue

Processor,NAM

Processor,NAM

Processor,GLERL

Processor,GLERL

UNCLASSIFIED

DispatchService

DispatchService

NotificationManagerService

NotificationManagerService

Processor,ADCIRC

Processor,ADCIRC

Processor,AGG

Processor,AGG

DeliveryManagerService

DeliveryManagerService

Qu

eues

Qu

eues

Data Request Interface (DRI)

(EDS Web Services)

WMSWMS

WFSWFS

ASYNCH (1.1)

ASYNCH (1.1)

Qu

eues

Qu

eues

Processor,STATIC

Processor,STATIC

DR

JMBLJMBL

DATA REQUEST SERVICE ARRAY (high-level)

SYNCH (1.0)

SYNCH (1.0)

PORTAL SERVICESPORTAL

SERVICES Processor,TIDAL

Processor,TIDAL

FUTUREFUTURE

Processor,RIVERS

Processor,RIVERS

UNCLASSIFIED

SOURCE NAME

COVERAGE & TYPE SOURCE FORMAT

EDS CONVERSION

DAN PROCESSOR

DATA REQ PROCESSOR

CODAR STPS East coast currents NetCDF None OPENDAP AGG

GFS Global winds Grib2 NetCDF GFS AGG

GLERL Great Lakes winds & currents Ascii NetCDF GLERL AGG

HYCOM Atlantic currents Grib2 NetCDF HYCOM AGG

HFRADAR East coast currents NetCDF None OPENDAP AGG

NAM US winds Grib2 NetCDF NAM AGG

NCOM Global currents, sst, el, salinity NetCDF None FTP AGG

NDBC_ALL US wind observations Ascii NetCDF FTP AGG

NDFD PR, AK, GU, HI, US winds Grib2 NetCDF NDFD AGG

NOGAPS Global winds Grib1 NetCDF NOGAPS (CAGIPS)

AGG

NOS Great lakes, Galveston bay, St John’s river, Chesapeake bay, Port of NY winds & currents

NetCDF None FTP AGG

SLDMBs Global buoy deployments Ascii None FTP HTTP

TIDAL SE Alaska, P Wm Sound, Columbia river, SF, JdF Strait, Narr bay currents

ASA None N/A CIR

STATIC Florida, Mariano currents NetCDF None N/A FNST

ADCIRC US East and West coast currents USACOE None N/A ADCIRC

RIVERS Major US rivers SQL+Ascii SQL FTP RIVERS

EDS DATA

UNCLASSIFIED

Service Arrays: Scalable

• Multiple Windows Services provide for a scalable system. Services may be started on several servers, spreading processing across servers and providing redundancy.

• Windows Services are started with Server StartUp. No login necessary. Windows Services are configured to ‘start-as’ a user that will have necessary EDS system privileges.

• Service array allows for plug-in of new services on-the-fly. • Workflow and configuration are data-driven.• Message queue usage is light and quick. Order XML submitted to

any queue is consumed as soon as processors are able to do so (any appreciable delay would be solved by starting additional services). Systems Admins (SA) may elect to apply redundant queues with no impact to the EDS. Queues are monitored to determine array performance with threshold excesses reported to SAs.

• Orders will be entered into a database and tracked through their life cycles. Allowing for metrics reports and future order warm-starts.

UNCLASSIFIED

Service Arrays: Expandable

• Basic structure includes SQL data driven dispatch of orders, throughout order life cycle, to queues monitored by specific services.

• Delivery management is data driven and dependent upon order origination.

• The service array design provides for modular processing, ultimately allowing OSC the option to create its own Processing Services. The Service Array Architecture is expandable without, necessarily, future reliance on ASA.

• An Automated Validation service exists to test data immediately after acquisition via the EDS’ own request services. Validation orders are processed with a unique ‘client’ ID, allowing SAs to differentiate between operational and validation data orders. Validation order setup is completely data-driven, allowing SAs to specify AOIs for any of the non-tidal/static data sources.

UNCLASSIFIED

EDS Portal

DANS DRS DataStore MAP Help

Queue Stats

Database view of source status

Notifications

OrderOrder PurgePurge FilterFilterEditEdit

Queue Stats

Database view of Client Orders In Process

Notifications

FilterFilter

DANS DRS DataStore MAP Help DANS DRS DataStore MAP Help

DANS DRS DataStore MAP Help

File System View

Port

al S

ervi

cePo

rtal

Ser

vice

Map view of available sources

QA Notifications

EDSDATABASE

UNCLASSIFIED

• Thin-client front-end supported by Portal Service• User login required• Use of Portal Service promotes greater security, removing direct access to

EDS, Database, Queue and FileSystem from the Portal itself• DAN (Catalog Server) view

– Dealer-like visual of source order status– Dealer-like utilities (Edit, Order, Purge)– View of outstanding orders on queues– View of any error, QA notifications

• DRS (EDS Orders) view– View of any client orders in-process (source, AOI, interval, order time)– View of outstanding orders on queues (provides view of service array)– View of any error, cancellation notifications

• Data Store view– Simple File System view– QA notification history may go here

• Map view– Thin-client map view of any data source via WMS

EDS PORTAL

UNCLASSIFIED

.

EDS Wind Products

• US Navy NOGAPS (global winds)

• NOAA NCEP GFS (Global Forecast System)

• NOAA NCEP NAM (North America Mesoscale)

• NOAA NDFD land and coastal winds

• NOAA GLERL & NOS Great Lakes winds

• NOAA NOS PORTS winds

• NOAA NDBC buoys & CMAN stations

UNCLASSIFIED

.

NOAA NCEP NAM (10-meter Winds)

Tropical Storm Beryl 20 July 2006

UNCLASSIFIED

.

EDS Current Products

• US Navy NAVO (via NCDDC) NCOM

• NOAA NCEP North Atlantic HYCOM

• NOAA GLERL/NOS Great Lakes currents

• ADCIRC East and West

• ASA tidal models

• Mariano Seasonal Ship drift

• Florida Current Data File

• MARCOORA HF radar & STPS

• USCG SLDMB’s

UNCLASSIFIED

MARCOORAHF Radar

UNCLASSIFIED

MARCOORAHF Radar