introduction and detailed system overview: science data … · introduction and detailed system...
TRANSCRIPT
625-EMD-002, Rev. 02
Introduction and Detailed System Overview: Science Data Processing Internal Training
Introduction and Detailed System Overview: Introduction and Detailed System Overview: Science Data Processing Internal TrainingScience Data Processing Internal Training
July 2006July 2006
EMD SDP Internal TrainingEMD SDP Internal Training
INSE
RTIO
NRETRIEVAL
PRODUCTION
SubscribeSearch& Order
Store
ExternalData Provider User
Deliver
Generate Plan
Ingest
User
625-EMD-002, Rev. 02
3625-EMD-002, Rev. 02
Objectives
• Overall objective: Describe ECS structure and function for Science Data Processing (SDP)– Identify subsystems and Computer Software Configuration
Items (CSCIs)– Specify major components and functions/processes of CSCIs– Describe role of CSCIs/functions/processes in the context of
ECS operational scenarios- ASTER-specific functions (e.g., DAR, expedited data support)- Producing and distributing data products (including media)- Updating QA metadata- On-demand processing- User registration- MODIS data access
4625-EMD-002, Rev. 02
What This Lesson Is (and Is Not)• Is
– Brief illustration of ECS high-level structure– Introduction to subsystems that make up ECS at a site– Examination of each subsystem and its Computer Software
Configuration Items (CSCIs), with components- Introduction of all system elements and brief description of functions- Background for subsequent scenario-based presentation of system
functional flows– Detailed look at system functioning in the context of operational
scenarios• Is Not
– Full description of overall ECS structure and function– Description of specific individual ECS entities (e.g., SMC)– Software development lesson– Complete description of interfaces and event sequences– Operations training
5625-EMD-002, Rev. 02
Program Overview
NASAScience Mission Directorate
U.S. Global Change Research Program
Earth Observing System
Terra (AM)Aqua (PM)AURA(CHEM)
Other
Earth Probes &Focused Missions
UARSTOPEXTRMMTOMSOther
EDOSEMSnECS
Science Satellites EOSDISEOS:- Designed to collectEarth Science Data
- 3 Components: spacesystem, ground system,research system.
EOSDIS
460 GB/DayRaw Data
28 GB/DayRaw Data
EOSDIS:- Provides the ground system for
collection and analysis of science data
Science Data User
1600 GB/DayProcessed Data
Science Data User:- Science Researchers
Worldwide Global Change Research Program
6625-EMD-002, Rev. 02
EOSDIS Principal Components
White Sands
S/C DataAcquisition
TDRSS
Complex
Polar Stations
ECSDataCommunications
EOSMissionSupportNetwork(EMSn)
KEYECSEDOSData Comm
S/COperations
EOS Spacecraft
USERS
USERS
USERS
USERS
Distribution/Access
DataCommunications
EOS Science
NetworkSupport
(ESSn)
EDOS
EOS Dataand
OperationsSystem (EDOS)
EOS Real-time Processing
System (ERPS)
ECS Mission OperationsSystem (EMOS)
Science Data ProcessingSegment (SDPS)
Communications andSystem Management Segment
(CSMS)
Instrument Support Toolkits
EOSDIS
Flight Dynamics
7625-EMD-002, Rev. 02
EOSDIS Data Flow
Data from other Mission Data Systems(TRMM, ICESat, SAGE III, . . .)
ECS MissionOperations
System(EMOS)
EOS
Miss
ion
Supp
ort N
etwo
rk (E
MSn)EDOS
FlightDynamics
System
Totally suppliedby ECSContains ECScomponents
ScienceComputingFacilities
ScienceCommunity
CooperatingData Centers
InstrumentSupport Toolkit
PI/TL Facility
EOS Science Support Network (ESSn)
InternationalPartner
Ground Systems, Operations Centers
Data ScienceInvestigator-
led ProcessingSystems (SIPS)
ECS DAACsGES ASDC NSIDC LP.DAAC
ScienceData
Processing
Non-ECS DAACsASF SEDAC ORNL
Communications andInternetworking Infrastructure
Non-ECS DAACsGES ASDCNSIDC LP.DAAC
PO.DAAC
InternationalPartner
Data Centers
8625-EMD-002, Rev. 02
ECS SDP Context
Insertand
Retrieve
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Archive
Communications (CSS)and Internetworking (ISS)
EMOS
ExternalData Sourcesand Providers
ScienceSoftwareand Data
IngestSubsystem
(INS) Ingest
Data
PUSH
InventorySearch
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULL
External/InternalUsers
Acquire
User Registration, OrderStatus, User Profile
Client (CLS)External/InternalUsers
System MgmtSubsystem
MSS
Hard MediaOrders
ProductDistributionSystem (PDS)
AcquireRequests
Data/Services
ProductRequests
ProductOrders
Order ManagerSubsystem
(OMS)ElectronicProducts
Data
Hard MediaProducts
Searchand
Download
Data Pool(DPL)
DATAPOOL
Searchand
Access
Data MgmtSubsystem
(DMS)
Meta-data
Data
Spatial Subscrip-tion Server
(SSS)
9625-EMD-002, Rev. 02
Subsystems and Functions
Science Data Processing Segment (SDPS)• Data Server Subsystem (DSS)
– Data storage and management: archive science data (with related insert, search and retrieve functions), archive management, data resource staging
• Product Distribution System (PDS)– Service for hard media orders, in conjunction with DSS and
Order Manager Server (OMS)• Ingest Subsystem (INS)
– Interface with external data providers and transfer data into ECS (with related staging functions and operator interfaces)
• Spatial Subscription Server (SSS)– Creation and management of subscriptions for data
distribution/notification and for Data Pool insert
10625-EMD-002, Rev. 02
Subsystems and Functions (Cont.)
SDPS (Cont.)• Data Pool (DPL)
– Provides on-line access for browsing and FTP download of selected granules, metadata, and browse data
• Client Subsystem (CLS)– Provides interfaces and access for external users
• Data Management Subsystem (DMS)– Enables cross-site data search and retrieval; gateway for
interface of ECS with EOS Data Gateway Web Client (Version 0 IMS) protocol
• Order Manager Subsystem (OMS)– Manages orders from EDG and other sources, distributing them
to appropriate ECS services (SDSRV, PDS)
11625-EMD-002, Rev. 02
Subsystems and Functions (Cont.)
SDPS (Cont.)
• Planning Subsystem (PLS)– Long- and short-term planning of science data processing, and
management of production resources• Data Processing Subsystem (DPS)
– Dispatches and monitors execution of science software
12625-EMD-002, Rev. 02
Subsystems and Functions (Cont.)
Communications and System Management Segment (CSMS)• System Management Subsystem (MSS)
– System maintenance, management, and administration (includes trouble ticketing, baseline and configuration management, fault and performance monitoring, and user account management and order management)
• Communications Subsystem (CSS)– General system infrastructure functions (includes network
communications, libraries to standardize software mechanisms, application error handling, subscription service, interfaces to e-mail, file transfer and file copy)
• Internetworking Subsystem (ISS)– Networking hardware devices and embedded softwareNOTE: The ISS is part of the ECS infrastructure and is not
addressed in detail in this course.
13625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: DSS
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
14625-EMD-002, Rev. 02
Subsystems and CSCIs: DSS (Cont.)
• Data Server Subsystem (DSS)– Provides capabilities to store, search,
retrieve, and distribute earth scienceand related data
– Client-server information transfer is by commands and requests
– Generates Universal References to identify ECS entities- Granule UR: represents a granule in the data server (e.g., as
follows)UR:10:DsShESDTUR:UR:15:DsShSciServerUR:13:[GSF:DSSDSRV]:16:SC:MOD10_L2:1411
- Server UR: represents a specific running data server application (e.g., )
– Interfaces with virtually all ECS subsystems and components– Uses several COTS tools: RogueWave tools and libraries,
Sybase relational database, Spatial Query Server
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
DsShSciServerUR
A granule is thesmallest piece ofdata that isindependentlymanaged by thesystem, i.e.,represented by arecord in the inventory.
15625-EMD-002, Rev. 02
Subsystems and CSCIs: DSS (Cont.)
• Science Data Server (SDSRV) CSCI– Provides the ECS with a catalog of Earth
Science Data holdings, and the Earth ScienceData Type (ESDT) services that operate on the data
– Manages and provides user access to data collections through its catalog of metadata and mechanisms to acquire data from the archive
– Seven major components- Science Data Server - services requests for storage, search,
retrieval, and manipulation of science data- HDF EOS Server - provides science data subsetting- Science Data Server GUI - provides operator interface- Granule Deletion Tool - provides a command-line interface for
deleting granules- Science Data Server Command Line Interface (SCLI) - provides
interface to the S4PM processing system and the Product Distribution System (PDS)
SDSRVInven-toryDB
16625-EMD-002, Rev. 02
Subsystems and CSCIs: DSS (Cont.)
• Science Data Server (SDSRV) CSCI (Cont.)– Seven major components (Cont.)
- Sybase ASE Server - manages catalog (metadata)- SQS Server - manages catalog (specialized spatial searches)
17625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
18625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
Metadata/Spatial Metadata storage request;Query
Metadata/Spatial Metadata retrieval
19625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
Metadata/storage request;Query
Metadata/retrieval
20625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
Request On-Demand
Granule Deletion
21625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
System status requests and controls; ESDT install/update requests
Error and system status information
22625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
Distribution Request;
Push Data Instructions;
Acquire Subsetted Data
23625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
Subsetting request;
Reformatting request
Subsetting status
24625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
Staging disk creation; archive requests
25625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
Inventory search
requests
ESDT installation
information; replace
ESDT installation
information
26625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
Granule value
parameter requests;
submit user a
cquire
(EcPlOdMgr)
Granule value
parameters (metadata
attributes)
27625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
MCF requests;
granule
acquire/deletio
n
requests; data
insert requests;
metadata update
requests
Metadata configuration
files (MCFs); product
history
28625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
Event notifica-fication; URs of inserted granules; register/replace
events; request distribution media options
Acquire requests based on subscriptions; request info for Expedited Data Set Notification (EDN); search; media options
29625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
Requests for metadata config-
uration files and validation of
metadata; data insertion requests
Metadata configura-tion files; metadata verification
30625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
User Profile
Request User Profile
31625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
Non-Synergy IV: Product
Requests for non-media
distribution
32625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
Non-Synergy IV: Product Requests for data to fill media orders
33625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
Non-Synergy IV: Query forgranule size
Non-Synergy IV: Granule size
34625-EMD-002, Rev. 02
EcDsScienceDataServer
EcDsSdSrvGui
ASE (Sybase)
ECS InventoryDatabase
EcDsHdfEosServer
Spatial QueryServer (SQS)
EcDsBulkDelete.pl
EcDsSCLI
Subsystems and CSCIs: DSS SDSRV Architecture and Interfaces
Request metadata
Metadata for conversion and storage in Data Pool
35625-EMD-002, Rev. 02
Subsystems and CSCIs: DSS (Cont.)
• Storage Management (STMGT) CSCI– Stores, manages, and retrieves data files on behalf of other
science data processing components– Six major components
- Archive Server - provides GUI and access to stored data- Cache/Staging Manager - Cache Manager server and Staging Disk
server manage data files that have been retrieved from the archive and placed into a cache area on staging disk
- Media Server Process - schedules access to shared peripheral resources (FTP, secure copy) and devices for Ingest (8mm, DTF-2)
- Pull Monitor - links to Cache Manager to manage files in the user pull area, making them eligible for deletion as they are retrieved by users or as their time-out periods expire
- Request Manager - routes requests from clients to servers- Data Base - contains data tables for STMGT devices, cache
management, event and log management, requests, and related functions
STMGT
36625-EMD-002, Rev. 02
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
37625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
Send all requests, for routing
Ingest Function
38625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
Ingest Function
Create_Allocate;
Destroy_Deallocate
39625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
Ingest Function
Allocate device; Mount
tape; Ingest to; Unmount
tape; Deallocate device;
Create_Allocate; Destroy_Deallocate
40625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
Transfer files
Ingest Function
Implement secure file transfers
41625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
Ingest Function
Device management and staging disk attachment information
Staging disk
attachment
information
Staging disk management information
42625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
Ingest Function
Request Data Insert
43625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
Ingest Function
Copy granule; copy browse
44625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
Ingest Function
Staging disk
attachment informa-
tion; Get volume
group/server/path
mapping
Volume group/server/ path mapping used to store data for the specified ESDT
45625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
Ingest Function
Status monitoring and control
46625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
Send all requests, for routing
DistributionStagingFunction
47625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionStagingFunction
Create_Allocate;Destroy_Detach;Link files
48625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionStagingFunction
Retrieve data;
Create_Attach/Allocate;
Destroy_Detach/Deallocate;
claim ownership; link file
49625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionStagingFunction
Create_Attach/Allocate;
Destroy_Detach/Deallocate;
link file
50625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionStagingFunction
Get volume group/server/path mapping
Volume group/server/path mapping
Retrieve data via
volume
group/server/path
51625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionStagingFunction
Copy data into cache
Copy file
52625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionStagingFunction
Release lien; manage disk information
53625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionStagingFunction
Update
cache
information
54625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionStagingFunction
Status monitoring and control
55625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionTransferFunction
Send all requests, for routing
56625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionTransferFunction
Request capacity; FTP pull/push;
create_allocate/attach; claim
ownership; destroy_deallocate;
link file
57625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionTransferFunction
Create_allocate/attach;
destroy_deallocate; link file
58625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionTransferFunction
Capacity
Attach to staging disk;
claim ownership; request capacity
59625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionTransferFunction
FTP pull; FTP push
60625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionTransferFunction
MkDir; FTP into cache; link file
61625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionTransferFunction
FTP push
directly to user
62625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionTransferFunction
Attach to staging disk; detach from staging disk
63625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionTransferFunction
Modify information for disk/pull area
64625-EMD-002, Rev. 02
ASE (Sybase) STMGT/DDISTDatabase
AMASS
DiskIndexFile
StagingDataList
Device InfoRequest List
EcDsStArchiveServer EcDsStCacheManagerServer
EcDsSt8MMServerEcDsStDTFServer
EcDsStFtpServer
EcDsStPullMonitorServer
EcDsStmgtGui
EcDsStStagingDiskServer
FTP Daemon
ResourceList
EcDsStRequestManagerServerEcDsStCopyServer
Subsystems and CSCIs: DSS STMGT Architecture and Interfaces
DistributionTransferFunction
Status monitoring and control
65625-EMD-002, Rev. 02
Subsystems and CSCIs: DSS (Cont.)
• Data Distribution (DDIST) CSCI– Monitors and controls processing of requests for internal and
external electronic distributions; distributions on physical media (8mm tape, CD-ROM, DVD, Digital Linear Tape) are handled as Product Distribution System (PDS) requests via FTPPush onto a PDS working directory, from which PDS reads the data for copy to hard media
– Sends e-mail notifications– Supports distribution of externally subsetted products– Five major components
- Data Distribution Server - provides control and coordination for data distribution through request processing
- Data Distribution GUI - allows operations staff to initiate, track, and manipulate distribution requests
- Data Base - contains the request list; updates and provides the request configuration
DDIST
Collection
66625-EMD-002, Rev. 02
Subsystems and CSCIs: DSS (Cont.)
• Data Distribution (DDIST) CSCI– Five major components (Cont.)
- External Product Dispatcher (EPD) - receives products from external subsetter and transfers them via DCLI to DDIST
- DDIST Command Line Interface (DCLI) - submits distribution requests for distribution of externally subsetted products
67625-EMD-002, Rev. 02
EcDsDdistGui
ASE (Sybase)
STMGT/DDISTDatabase
Ingest Local Disk
EXTERNAL
SUBSETTE
R
EcDsDdDCLISubsetterLocation(e.g., DAAC)
EPD Server(epdserver)
EcDsDistributionServer
Subsystems and CSCIs: DSS DDIST Architecture and Interfaces
68625-EMD-002, Rev. 02
EcDsDdistGui
ASE (Sybase)
STMGT/DDISTDatabase
Ingest Local Disk
EXTERNAL
SUBSETTE
R
EcDsDdDCLISubsetterLocation(e.g., DAAC)
EPD Server(epdserver)
EcDsDistributionServer
Subsystems and CSCIs: DSS DDIST Architecture and Interfaces
Distribution request; push data instructions; Acquire
subsetted data
69625-EMD-002, Rev. 02
EcDsDdistGui
ASE (Sybase)
STMGT/DDISTDatabase
Ingest Local Disk
EXTERNAL
SUBSETTE
R
EcDsDdDCLISubsetterLocation(e.g., DAAC)
EPD Server(epdserver)
EcDsDistributionServer
Subsystems and CSCIs: DSS DDIST Architecture and Interfaces
Create_Att; Create_All; Destroy_Dea; Link files; Retrieve data; Claim staging area; Get capacity (of a media type); FTP Push/Pull
70625-EMD-002, Rev. 02
EcDsDdistGui
ASE (Sybase)
STMGT/DDISTDatabase
Ingest Local Disk
EXTERNAL
SUBSETTE
R
EcDsDdDCLISubsetterLocation(e.g., DAAC)
EPD Server(epdserver)
EcDsDistributionServer
Subsystems and CSCIs: DSS DDIST Architecture and Interfaces
Product Delivery Record (via FTPpull);
subsetting output data
Return status (PAN or PDRD) *
*PAN Product Acceptance NotificationPDRD Product Delivery Record Discrepancy
71625-EMD-002, Rev. 02
EcDsDdistGui
ASE (Sybase)
STMGT/DDISTDatabase
Ingest Local Disk
EXTERNAL
SUBSETTE
R
EcDsDdDCLISubsetterLocation(e.g., DAAC)
EPD Server(epdserver)
EcDsDistributionServer
Subsystems and CSCIs: DSS DDIST Architecture and Interfaces
Distribution Request
72625-EMD-002, Rev. 02
EcDsDdistGui
ASE (Sybase)
STMGT/DDISTDatabase
Ingest Local Disk
EXTERNAL
SUBSETTE
R
EcDsDdDCLISubsetterLocation(e.g., DAAC)
EPD Server(epdserver)
EcDsDistributionServer
Subsystems and CSCIs: DSS DDIST Architecture and Interfaces
Request management
services (e.g., user
profile, order/request
information)
User profile
information;
order/request
information
73625-EMD-002, Rev. 02
EcDsDdistGui
ASE (Sybase)
STMGT/DDISTDatabase
Ingest Local Disk
EXTERNAL
SUBSETTE
R
EcDsDdDCLISubsetterLocation(e.g., DAAC)
EPD Server(epdserver)
EcDsDistributionServer
Subsystems and CSCIs: DSS DDIST Architecture and Interfaces
Request communication services; Distribution Notification/Information (via Email to user, external subsetter, or EcInEmailGWServer)
Configuration parameters; server location information
74625-EMD-002, Rev. 02
EcDsDdistGui
ASE (Sybase)
STMGT/DDISTDatabase
Ingest Local Disk
EXTERNAL
SUBSETTE
R
EcDsDdDCLISubsetterLocation(e.g., DAAC)
EPD Server(epdserver)
EcDsDistributionServer
Subsystems and CSCIs: DSS DDIST Architecture and Interfaces
Push Data (from another mode) to Ingest Local Disk (via staging in STMGT and then FTP service)
75625-EMD-002, Rev. 02
EcDsDdistGui
ASE (Sybase)
STMGT/DDISTDatabase
Ingest Local Disk
EXTERNAL
SUBSETTE
R
EcDsDdDCLISubsetterLocation(e.g., DAAC)
EPD Server(epdserver)
EcDsDistributionServer
Subsystems and CSCIs: DSS DDIST Architecture and Interfaces
Status and error
information
Cancel, Resume, or Suspend request; Set Priority; Configuration control information (e.g.,distribution polling rate)
76625-EMD-002, Rev. 02
EcDsDdistGui
ASE (Sybase)
STMGT/DDISTDatabase
Ingest Local Disk
EXTERNAL
SUBSETTE
R
EcDsDdDCLISubsetterLocation(e.g., DAAC)
EPD Server(epdserver)
EcDsDistributionServer
Subsystems and CSCIs: DSS DDIST Architecture and Interfaces
Set Data Distribution parameters (queues, thresholds, suspend, and resume)
77625-EMD-002, Rev. 02
EcDsDdistGui
ASE (Sybase)
STMGT/DDISTDatabase
Ingest Local Disk
EXTERNAL
SUBSETTE
R
EcDsDdDCLISubsetterLocation(e.g., DAAC)
EPD Server(epdserver)
EcDsDistributionServer
Subsystems and CSCIs: DSS DDIST Architecture and Interfaces
Data Distribution
parameters (queues,
thresholds, suspend
and resume status)
78625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: DSS (Cont.)
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager ProcessEcDsStCacheManagerServer
EcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionTool
EcDsBulkDelete.pl
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
79625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: INS
IngestSubsystem
(INS) Ingest
80625-EMD-002, Rev. 02
Subsystems and CSCIs: INS
• Ingest Subsystem (INS)– Transfer of data into ECS (SDPS repositories) in accordance
with approved ICDs– Supports varied data formats and structures– Ingest Client: A set of ingest software configured for
requirements of a specific situation– Ingest clients perform data preprocessing, such as format
conversion, metadata extraction, and metadata validation on incoming data
– Data staged to one of two areas- Level 0 (L0) data from ongoing missions, and EDOS ancillary data,
staged to INS working storage area- Non-L0 data (e.g., non-EDOS ancillary data, L1A-L4 data) staged
directly to DSS working storage area – Uses several COTS tools: RogueWave class libraries, Sybase
relational database, CCS Middleware Client
IngestSubsystem Ingest
81625-EMD-002, Rev. 02
Subsystems and CSCIs: INS (Cont.)
• Ingest (INGST) CSCI– Gets data by various methods and transfers the data into ECS
- Polling: transfer of data from predetermined network locations which Ingest periodically checks for new data- With Delivery Record- Without Delivery Record
- Media: reading data from physical media; uses GUI- Cross-Mode Ingest: E-mail distribution notification used to create
a Delivery Record File for Polling with Delivery Record– Stores and manages request information– Provides for data preprocessing and insertion
82625-EMD-002, Rev. 02
Subsystems and CSCIs: INS (Cont.)
• Ingest (INGST) CSCI (Cont.)– Six major components
- Polling Ingest Client Interface - creates polling request, detects new files in a specified external location, creates and submits ingest request
- Media Ingest Interface - provides operators ability to perform ingest from physical media
- Cross-Mode Ingest Interface - provides an E-mail gateway server to receive E-mail distribution notifications and store them as files in a location for polling with delivery record
- Ingest Request Manager - manages ingest request traffic and processing
- Ingest Granule Server - provides services for required preprocessing of data and subsequent insertion into Data Server
- Ingest Database - stores and provides access to Ingest Subsystem internal data (e.g., Request Status, History Logs)
83625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
84625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Store datato beIngested
85625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Detect and
read data
86625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Send request
87625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
*PAN Product Acceptance NotificationPDRD Product Delivery Record Discrepancy
Return status
(PAN or PDRD) *
88625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Clean up directory (Ingest local disk only)
89625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Monitor and control Ingest functions
90625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Monitor and
control Tape
Reader functions
91625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Control
peripherals
Create staging disk;control peripherals
92625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Store datato beIngested
93625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Granule processing/cancellation requests
Completion notification
94625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Allocate and transfer files to Staging Disk; Get MCF; Validate metadata; Data granule insert; Data search
95625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Update Ingest
tunable parameters; Data for
reports
Record and retrieve request information
96625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Update file
and granule
information
97625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Query and obtain data
98625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Push data (fromanothermode)
99625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Send Distribution
Notification
100625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Detect data;transfer PDR
101625-EMD-002, Rev. 02
INSINGST
EcInGUI
ASE(Sybase)
ExternalDataProviders
EcInPolling
EcInReqMgr
IngestDatabase
DAAC IngestTechnician
EcInEmailGWServer
EcInTapeReaderGUI
EcInGran
LocalDisk
Subsystems and CSCIs: INS (Cont.)Architecture and Interfaces
Request
Communications
Support
102625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: INS (Cont.)
IngestSubsystem
(INS) IngestIngest(INGST) CSCI
Polling IngestClient Interface
ProcessEcInPolling
MediaIngest Interface
ProcessEcInGUI
EcInTapeReaderGUI
IngestRequest Manager
ProcessEcInReqMgr
IngestGranule Server
ProcessEcInGran
IngestDatabase
Sybase Server (COTS)
Cross-ModeIngest Interface
EcInEmailGWServer
103625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: SSS
Spatial Subscrip-tion Server
(SSS)
104625-EMD-002, Rev. 02
Subsystems and CSCIs: SSS
• Spatial Subscription Server (SSS)– Creating, viewing, updating Subscriptions (specification of an
action and an event that initiates the action)- Actions: Notification, Distribution, Data Pool Insert- Events: Granule Insert, Granule Deletion, Metadata Update
– Creating, viewing, deleting Bundling Orders (specification of distribution packages and criteria for package completion)- Minimum bundle size- Minimum granule count- Maximum bundle age- Bundling order information stored in Order Manager database
– Subscription processing triggered by appearance of events in Science Data Server database- Identify all subscriptions to the specified event- Process the actions defined in the subscriptions
Spatial Subscrip-tion Server
(SSS)
105625-EMD-002, Rev. 02
Subsystems and CSCIs: SSS (Cont.)
• Spatial Subscription Server (SSS) (Cont.)– Uses several COTS tools: Netscape Navigator, Sun ONE Web
Server, Sybase ASE• Spatial Subscription Server (NBSRV) CSCI (a.k.a. NSBRV
CSCI)– Provides a Graphical User Interface (GUI) and a set of drivers
for implementing subscription functions– Six major components
- Spatial Subscription Server database - repository for all data created expressly for use by the NBSRV
- Subscription GUI - tool for entering, modifying, or deleting subscriptions and bundling orders
- Event Queue Monitor - multiple instances of a Perl script that monitors the event queue for new arrivals
- Action Queue Monitor - multiple instances of a Perl script that monitors the action queue for new arrivals
106625-EMD-002, Rev. 02
Subsystems and CSCIs: SSS (Cont.)
• Spatial Subscription Server (NBSRV) CSCI (a.k.a. NSBRV CSCI) (Cont.)– Six major components (Cont.)
- Recovery Driver - Monitors logs for stalled events or actions; re-enqueues stalled events/actions
- Deletion Driver - Works off a deletion queue to purge the database of outdated information (e.g., completed events/actions)
107625-EMD-002, Rev. 02
Subsystems and CSCIs: SSS (Cont.) NBSRV Architecture and Interfaces
ASE(Sybase)
SSSDatabase
SSSNBSRV
EcNbSubscribedEventDriver.pl
WebBrowser EcNbGUISun ONE
WebServer
EcNbDeleteRequestDriver.pl
EcNbRecoverDriver.plEcNbActionDriver.pl
DAACOperator
108625-EMD-002, Rev. 02
Subsystems and CSCIs: SSS (Cont.) NBSRV Architecture and Interfaces
ASE(Sybase)
SSSDatabase
SSSNBSRV
EcNbSubscribedEventDriver.pl
WebBrowser EcNbGUISun ONE
WebServer
EcNbDeleteRequestDriver.pl
EcNbRecoverDriver.plEcNbActionDriver.pl
DAACOperator
List/view subscribable events, subscriptions, actions, statistics, bundling orders
Create/modify/delete subscriptions
for Data Pool insert o
r distribution/
notification; create/modify/delete
bundling orders
109625-EMD-002, Rev. 02
Subsystems and CSCIs: SSS (Cont.) NBSRV Architecture and Interfaces
ASE(Sybase)
SSSDatabase
SSSNBSRV
EcNbSubscribedEventDriver.pl
WebBrowser EcNbGUISun ONE
WebServer
EcNbDeleteRequestDriver.pl
EcNbRecoverDriver.plEcNbActionDriver.pl
DAACOperator
Create (bundling) order
Order ID (for bundling order)
110625-EMD-002, Rev. 02
ASE(Sybase)
SSSDatabase
SSSNBSRV
EcNbSubscribedEventDriver.pl
WebBrowser EcNbGUISun ONE
WebServer
EcNbDeleteRequestDriver.pl
EcNbRecoverDriver.plEcNbActionDriver.pl
DAACOperator
Subsystems and CSCIs: SSS (Cont.) NBSRV Architecture and Interfaces
Retrieve data
Update data
111625-EMD-002, Rev. 02
ASE(Sybase)
SSSDatabase
SSSNBSRV
EcNbSubscribedEventDriver.pl
WebBrowser EcNbGUISun ONE
WebServer
EcNbDeleteRequestDriver.pl
EcNbRecoverDriver.plEcNbActionDriver.pl
DAACOperator
Subsystems and CSCIs: SSS (Cont.) NBSRV Architecture and Interfaces
Bundling order information
112625-EMD-002, Rev. 02
ASE(Sybase)
SSSDatabase
SSSNBSRV
EcNbSubscribedEventDriver.pl
WebBrowser EcNbGUISun ONE
WebServer
EcNbDeleteRequestDriver.pl
EcNbRecoverDriver.plEcNbActionDriver.pl
DAACOperator
Subsystems and CSCIs: SSS (Cont.) NBSRV Architecture and Interfaces
Event for subscribed event queue in SSS Database (based on activation of trigger upon insert of qualified granule)
113625-EMD-002, Rev. 02
ASE(Sybase)
SSSDatabase
SSSNBSRV
EcNbSubscribedEventDriver.pl
WebBrowser EcNbGUISun ONE
WebServer
EcNbDeleteRequestDriver.pl
EcNbRecoverDriver.plEcNbActionDriver.pl
DAACOperator
Subsystems and CSCIs: SSS (Cont.) NBSRV Architecture and Interfaces
Monitor event queue and
detect event;
Get metadata for granule that triggered event
metadata
(from SDSRV database);
identify subscriptions for
which a granule qualifies
114625-EMD-002, Rev. 02
ASE(Sybase)
SSSDatabase
SSSNBSRV
EcNbSubscribedEventDriver.pl
WebBrowser EcNbGUISun ONE
WebServer
EcNbDeleteRequestDriver.pl
EcNbRecoverDriver.plEcNbActionDriver.pl
DAACOperator
Subsystems and CSCIs: SSS (Cont.) NBSRV Architecture and Interfaces
Data Pool Insert action; browse granule (direct write to Data Pool database)
115625-EMD-002, Rev. 02
ASE(Sybase)
SSSDatabase
SSSNBSRV
EcNbSubscribedEventDriver.pl
WebBrowser EcNbGUISun ONE
WebServer
EcNbDeleteRequestDriver.pl
EcNbRecoverDriver.plEcNbActionDriver.pl
DAACOperator
Subsystems and CSCIs: SSS (Cont.) NBSRV Architecture and Interfaces
Monitor action queue and
detect event; get action(s)
for subscription
116625-EMD-002, Rev. 02
ASE(Sybase)
SSSDatabase
SSSNBSRV
EcNbSubscribedEventDriver.pl
WebBrowser EcNbGUISun ONE
WebServer
EcNbDeleteRequestDriver.pl
EcNbRecoverDriver.plEcNbActionDriver.pl
DAACOperator
Subsystems and CSCIs: SSS (Cont.) NBSRV Architecture and Interfaces
Create (non-bundling) order
Order ID and request ID (for non-bundling order)Note: The request ID is used for an immediate distribution action.
117625-EMD-002, Rev. 02
ASE(Sybase)
SSSDatabase
SSSNBSRV
EcNbSubscribedEventDriver.pl
WebBrowser EcNbGUISun ONE
WebServer
EcNbDeleteRequestDriver.pl
EcNbRecoverDriver.plEcNbActionDriver.pl
DAACOperator
Subsystems and CSCIs: SSS (Cont.) NBSRV Architecture and Interfaces
Acquire request
118625-EMD-002, Rev. 02
ASE(Sybase)
SSSDatabase
SSSNBSRV
EcNbSubscribedEventDriver.pl
WebBrowser EcNbGUISun ONE
WebServer
EcNbDeleteRequestDriver.pl
EcNbRecoverDriver.plEcNbActionDriver.pl
DAACOperator
Subsystems and CSCIs: SSS (Cont.) NBSRV Architecture and Interfaces
Email text file
119625-EMD-002, Rev. 02
ASE(Sybase)
SSSDatabase
SSSNBSRV
EcNbSubscribedEventDriver.pl
WebBrowser EcNbGUISun ONE
WebServer
EcNbDeleteRequestDriver.pl
EcNbRecoverDriver.plEcNbActionDriver.pl
DAACOperator
Subsystems and CSCIs: SSS (Cont.) NBSRV Architecture and Interfaces
Read deletion queue; clean
up database tables to purge
outdated information (e.g.,
completed events/actions)
120625-EMD-002, Rev. 02
ASE(Sybase)
SSSDatabase
SSSNBSRV
EcNbSubscribedEventDriver.pl
WebBrowser EcNbGUISun ONE
WebServer
EcNbDeleteRequestDriver.pl
EcNbRecoverDriver.plEcNbActionDriver.pl
DAACOperator
Subsystems and CSCIs: SSS (Cont.) NBSRV Architecture and Interfaces
Monitor logs for
stalled processing
of events/actions;
re-enqueue stalled
events/actions
121625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: SSS (Cont.)
Spatial Subscrip-tion Server
(SSS)
Spatial Subscription Server(NBSRV) CSCI
Action DriverEcNbActionDriver.pl
NBSRV GUIEcNbGUI
Subscribed EventDriver
EcNbSubscribedEventDriver.pl
Deletion DriverEcNbDeleteRequestDriver.pl
SSS (NBSRV)Database
Sybase Server (COTS)
Recovery DriverEcNbRecoverDriver.pl
122625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: DPL
Data Pool(DPL)
DATAPOOL
123625-EMD-002, Rev. 02
Subsystems and CSCIs: DPL
• Data Pool (DPL)– An on-line repository of selected granules with associated
metadata and, if available, browse granules– Accessible through a web browser– Accessible through FTP– Data downloadable via FTP– Provides easy-to-use drill-down web user interface– Included in EDG data search results– Populated by subscriptions for Data Pool insert– Stages data for OMS FTP distributions– Uses several COTS tools: Netscape Navigator, Sun ONE Web
Server, Apache Web Server, wuftp (COTS FTP Server), Sybase ASE
Data Pool(DPL)
DATAPOOL
124625-EMD-002, Rev. 02
Subsystems and CSCIs: DPL (Cont.)
• Data Pool Management (DPMGT) CSCI– Provides on-line cache for access to selected ECS data,
metadata, and browse granules– Permits user search and FTP download through a web interface– Permits user browsing and download during an FTP session– Provides an Insert Utility for insert of data and metadata– Serves as staging cache for OMS FTP distributions (push and
pull)– Manages archive drive resources (with OMS help)
125625-EMD-002, Rev. 02
Subsystems and CSCIs: DPL (Cont.)
• Data Pool Management (DPMGT) CSCI (Cont.)– Four major components
- Data Pool Insert Utility - consists of four subcomponents- Action Driver schedules insert actions- Insert Utility requests copy to the Data Pool and updates the
inventory- Quick Server, a C++ executable, performs the copy from AMASS- EcDlM2XT, a java executable, translates granule metadata into
XML format- Data Pool Web Access - a web-based GUI that provides easy drill-
down search and FTP download for the user- wuftp - COTS FTP server for user-interactive FTP sessions- Data Pool Database - Sybase database stores Data Pool inventory
and configuration information
126625-EMD-002, Rev. 02
ASE(Sybase)
DPL Database
DPLDPMGT
Data Pool Insert(EcDlInsertUtility,EcDlActionDriver,EcDlQuickServer,EcDlM2XTApp)
DPMNT
wuftp(COTS FTP Server)
Data PoolData PoolDisksDisks
ApacheWeb Server
WebBrowser
FTP Client
Firewall
EcDlWebAccess
Science User
Subsystems and CSCIs: DPL (Cont.)DPMGT Architecture and Interfaces
127625-EMD-002, Rev. 02
ASE(Sybase)
DPL Database
DPLDPMGT
Data Pool Insert(EcDlInsertUtility,EcDlActionDriver,EcDlQuickServer,EcDlM2XTApp)
DPMNT
wuftp(COTS FTP Server)
Data PoolData PoolDisksDisks
ApacheWeb Server
WebBrowser
FTP Client
Firewall
EcDlWebAccess
Science User
Subsystems and CSCIs: DPL (Cont.)DPMGT Architecture and Interfaces
Place an insert on the
Data Pool Insert Action
Queue
Submit granule staging request
128625-EMD-002, Rev. 02
ASE(Sybase)
DPL Database
DPLDPMGT
Data Pool Insert(EcDlInsertUtility,EcDlActionDriver,EcDlQuickServer,EcDlM2XTApp)
DPMNT
wuftp(COTS FTP Server)
Data PoolData PoolDisksDisks
ApacheWeb Server
WebBrowser
FTP Client
Firewall
EcDlWebAccess
Science User
Subsystems and CSCIs: DPL (Cont.)DPMGT Architecture and Interfaces
Query/ update database tables
129625-EMD-002, Rev. 02
ASE(Sybase)
DPL Database
DPLDPMGT
Data Pool Insert(EcDlInsertUtility,EcDlActionDriver,EcDlQuickServer,EcDlM2XTApp)
DPMNT
wuftp(COTS FTP Server)
Data PoolData PoolDisksDisks
ApacheWeb Server
WebBrowser
FTP Client
Firewall
EcDlWebAccess
Science User
Subsystems and CSCIs: DPL (Cont.)DPMGT Architecture and Interfaces
Copy granule,
insert browse;
convert and
store metadata
Request data
insert; request
browse insert;
request metadata
130625-EMD-002, Rev. 02
ASE(Sybase)
DPL Database
DPLDPMGT
Data Pool Insert(EcDlInsertUtility,EcDlActionDriver,EcDlQuickServer,EcDlM2XTApp)
DPMNT
wuftp(COTS FTP Server)
Data PoolData PoolDisksDisks
ApacheWeb Server
WebBrowser
FTP Client
Firewall
EcDlWebAccess
Science User
Subsystems and CSCIs: DPL (Cont.)DPMGT Architecture and Interfaces
Update Data Pool
Inventory
131625-EMD-002, Rev. 02
ASE(Sybase)
DPL Database
DPLDPMGT
Data Pool Insert(EcDlInsertUtility,EcDlActionDriver,EcDlQuickServer,EcDlM2XTApp)
DPMNT
wuftp(COTS FTP Server)
Data PoolData PoolDisksDisks
ApacheWeb Server
WebBrowser
FTP Client
Firewall
EcDlWebAccess
Science User
Subsystems and CSCIs: DPL (Cont.)DPMGT Architecture and Interfaces
User search of
Data Pool to locate
file(s) of interest
132625-EMD-002, Rev. 02
ASE(Sybase)
DPL Database
DPLDPMGT
Data Pool Insert(EcDlInsertUtility,EcDlActionDriver,EcDlQuickServer,EcDlM2XTApp)
DPMNT
wuftp(COTS FTP Server)
Data PoolData PoolDisksDisks
ApacheWeb Server
WebBrowser
FTP Client
Firewall
EcDlWebAccess
Science User
Subsystems and CSCIs: DPL (Cont.)DPMGT Architecture and Interfaces
nfs read for
FTP download
133625-EMD-002, Rev. 02
ASE(Sybase)
DPL Database
DPLDPMGT
Data Pool Insert(EcDlInsertUtility,EcDlActionDriver,EcDlQuickServer,EcDlM2XTApp)
DPMNT
wuftp(COTS FTP Server)
Data PoolData PoolDisksDisks
ApacheWeb Server
WebBrowser
FTP Client
Firewall
EcDlWebAccess
Science User
Subsystems and CSCIs: DPL (Cont.)DPMGT Architecture and Interfaces
Submit hard
media order
134625-EMD-002, Rev. 02
ASE(Sybase)
DPL Database
DPLDPMGT
Data Pool Insert(EcDlInsertUtility,EcDlActionDriver,EcDlQuickServer,EcDlM2XTApp)
DPMNT
wuftp(COTS FTP Server)
Data PoolData PoolDisksDisks
ApacheWeb Server
WebBrowser
FTP Client
Firewall
EcDlWebAccess
Science User
Subsystems and CSCIs: DPL (Cont.)DPMGT Architecture and Interfaces
User ftp
search and
download of
Data Pool
file(s) of
interest
135625-EMD-002, Rev. 02
ASE(Sybase)
DPL Database
DPLDPMGT
Data Pool Insert(EcDlInsertUtility,EcDlActionDriver,EcDlQuickServer,EcDlM2XTApp)
DPMNT
wuftp(COTS FTP Server)
Data PoolData PoolDisksDisks
ApacheWeb Server
WebBrowser
FTP Client
Firewall
EcDlWebAccess
Science User
Subsystems and CSCIs: DPL (Cont.)DPMGT Architecture and Interfaces
FTP search
and download
using web
browser
136625-EMD-002, Rev. 02
ASE(Sybase)
DPL Database
DPLDPMGT
Data Pool Insert(EcDlInsertUtility,EcDlActionDriver,EcDlQuickServer,EcDlM2XTApp)
DPMNT
wuftp(COTS FTP Server)
Data PoolData PoolDisksDisks
ApacheWeb Server
WebBrowser
FTP Client
Firewall
EcDlWebAccess
Science User
Subsystems and CSCIs: DPL (Cont.)DPMGT Architecture and Interfaces
Request metadata
URLs, science data
URLs, browse
URLs, and
expiration dates
Return metadata
URLs, science data
URLs, browse
URLs, and
expiration dates
137625-EMD-002, Rev. 02
ASE(Sybase)
DPL Database
DPLDPMGT
Data Pool Insert(EcDlInsertUtility,EcDlActionDriver,EcDlQuickServer,EcDlM2XTApp)
DPMNT
wuftp(COTS FTP Server)
Data PoolData PoolDisksDisks
ApacheWeb Server
WebBrowser
FTP Client
Firewall
EcDlWebAccess
Science User
Subsystems and CSCIs: DPL (Cont.)DPMGT Architecture and Interfaces
Copy data
138625-EMD-002, Rev. 02
ASE(Sybase)
DPL Database
DPLDPMGT
Data Pool Insert(EcDlInsertUtility,EcDlActionDriver,EcDlQuickServer,EcDlM2XTApp)
DPMNT
wuftp(COTS FTP Server)
Data PoolData PoolDisksDisks
ApacheWeb Server
WebBrowser
FTP Client
Firewall
EcDlWebAccess
Science User
Subsystems and CSCIs: DPL (Cont.)DPMGT Architecture and Interfaces
Data Pool Maintenance and Monitoring
139625-EMD-002, Rev. 02
Subsystems and CSCIs: DPL (Cont.)
• Data Pool Maintenance (DPMNT) CSCI– Provides a maintenance GUI that allows operators to monitor
and control Data Pool insert activity and control the Data Pool configuration
– Provides utilities and scripts for Data Pool maintenance– Nine major components
- Data Pool Maintenance (DPM) GUI - a Perl web-based GUI for Data Pool monitoring and control
- Update Granule Expiration Utility - a Perl utility that allows updating the expiration date and retention priority for granules in the Data Pool
- Data Pool Cleanup - a Perl utility that removes expired granules from the Data Pool and database (normally run as a cron job)
- Data Pool Access Statistics Utility (DPASU) - Perl utilities that extract access statistics from logs and roll up access information for storage in the Data Pool database
140625-EMD-002, Rev. 02
Subsystems and CSCIs: DPL (Cont.)
• Data Pool Maintenance (DPMNT) CSCI (Cont.)– Nine major components (Cont.)
- Data Pool Archive/Delete/Restore Access Statistics - shell scripts to manage access statistics
- Data Pool Most Recent Insert Utility - a Perl utility that creates files containing information concerning granules recently inserted into the Data Pool
- Data Pool Collection Remapping Utility - a Perl utility that allows DAAC operations staff to remap all data in a Data Pool collection directory from one higher-level collection group directory to another
- Data Pool Density Map Utility - a Perl utility that calculates spatial density map information about Data Pool collections and stores the information in the Data Pool database
- Data Pool Statistics Table Population Utility - a Perl utility that populates Data Pool database tables which maintain counts of granules by collection and collection group for use by the Web Access drill down GUI
141625-EMD-002, Rev. 02DPL
DPMNT
DPMGT
WebBrowser
DAAC Operator
Data Pool Maintenance GUI
EcDlCleanupDataPool.pl
EcDlUpdateGranule.pl
EcDlRollupWebLogs.plEcDlRollupFwFtpLogs.pl
EcDlRemap.pl
Sun ONE Web Server
EcDlDensityMapUtility.pl
EcDlMostRecentInsert.pl
File of RecentGranules
EcDlPopulateStatTables.pl
DlDbArchiveAccessStatDlDbDeleteAccessStatDlDbRestoreAccessStat
Subsystems and CSCIs: DPL (Cont.)DPMNT Architecture and Interfaces
142625-EMD-002, Rev. 02DPL
DPMNT
DPMGT
WebBrowser
DAAC Operator
Data Pool Maintenance GUI
EcDlCleanupDataPool.pl
EcDlUpdateGranule.pl
EcDlRollupWebLogs.plEcDlRollupFwFtpLogs.pl
EcDlRemap.pl
Sun ONE Web Server
EcDlDensityMapUtility.pl
EcDlMostRecentInsert.pl
File of RecentGranules
EcDlPopulateStatTables.pl
DlDbArchiveAccessStatDlDbDeleteAccessStatDlDbRestoreAccessStat
Subsystems and CSCIs: DPL (Cont.)DPMNT Architecture and Interfaces
Control Data Pool processes
(e.g., Suspend/Resume/
Cancel Data Pool Insert
actions, configure defaults,
add/modify collections and
collection groups)Query/
update
database
tables
View/monitor Data Pool processes
143625-EMD-002, Rev. 02DPL
DPMNT
DPMGT
WebBrowser
DAAC Operator
Data Pool Maintenance GUI
EcDlCleanupDataPool.pl
EcDlUpdateGranule.pl
EcDlRollupWebLogs.plEcDlRollupFwFtpLogs.pl
EcDlRemap.pl
Sun ONE Web Server
EcDlDensityMapUtility.pl
EcDlMostRecentInsert.pl
File of RecentGranules
EcDlPopulateStatTables.pl
DlDbArchiveAccessStatDlDbDeleteAccessStatDlDbRestoreAccessStat
Subsystems and CSCIs: DPL (Cont.)DPMNT Architecture and Interfaces
Extend retention/modify priority for science data in the Data Pool
144625-EMD-002, Rev. 02DPL
DPMNT
DPMGT
WebBrowser
DAAC Operator
Data Pool Maintenance GUI
EcDlCleanupDataPool.pl
EcDlUpdateGranule.pl
EcDlRollupWebLogs.plEcDlRollupFwFtpLogs.pl
EcDlRemap.pl
Sun ONE Web Server
EcDlDensityMapUtility.pl
EcDlMostRecentInsert.pl
File of RecentGranules
EcDlPopulateStatTables.pl
DlDbArchiveAccessStatDlDbDeleteAccessStatDlDbRestoreAccessStat
Subsystems and CSCIs: DPL (Cont.)DPMNT Architecture and Interfaces
Invoke cleanup manually (normally it
runs as a cron job)Query database to qualify granules for deletion; delete
145625-EMD-002, Rev. 02DPL
DPMNT
DPMGT
WebBrowser
DAAC Operator
Data Pool Maintenance GUI
EcDlCleanupDataPool.pl
EcDlUpdateGranule.pl
EcDlRollupWebLogs.plEcDlRollupFwFtpLogs.pl
EcDlRemap.pl
Sun ONE Web Server
EcDlDensityMapUtility.pl
EcDlMostRecentInsert.pl
File of RecentGranules
EcDlPopulateStatTables.pl
DlDbArchiveAccessStatDlDbDeleteAccessStatDlDbRestoreAccessStat
Subsystems and CSCIs: DPL (Cont.)DPMNT Architecture and Interfaces
Extract Data Pool access statistics from web logs and FTP logs; store in database
146625-EMD-002, Rev. 02DPL
DPMNT
DPMGT
WebBrowser
DAAC Operator
Data Pool Maintenance GUI
EcDlCleanupDataPool.pl
EcDlUpdateGranule.pl
EcDlRollupWebLogs.plEcDlRollupFwFtpLogs.pl
EcDlRemap.pl
Sun ONE Web Server
EcDlDensityMapUtility.pl
EcDlMostRecentInsert.pl
File of RecentGranules
EcDlPopulateStatTables.pl
DlDbArchiveAccessStatDlDbDeleteAccessStatDlDbRestoreAccessStat
Subsystems and CSCIs: DPL (Cont.)DPMNT Architecture and Interfaces
Archive/delete/restore information in the database tables populated by the Data Pool Access Statistics Utility (Rollup)
147625-EMD-002, Rev. 02DPL
DPMNT
DPMGT
WebBrowser
DAAC Operator
Data Pool Maintenance GUI
EcDlCleanupDataPool.pl
EcDlUpdateGranule.pl
EcDlRollupWebLogs.plEcDlRollupFwFtpLogs.pl
EcDlRemap.pl
Sun ONE Web Server
EcDlDensityMapUtility.pl
EcDlMostRecentInsert.pl
File of RecentGranules
EcDlPopulateStatTables.pl
DlDbArchiveAccessStatDlDbDeleteAccessStatDlDbRestoreAccessStat
Subsystems and CSCIs: DPL (Cont.)DPMNT Architecture and Interfaces
Remap data from one higher-level collection groupto another
148625-EMD-002, Rev. 02DPL
DPMNT
DPMGT
WebBrowser
DAAC Operator
Data Pool Maintenance GUI
EcDlCleanupDataPool.pl
EcDlUpdateGranule.pl
EcDlRollupWebLogs.plEcDlRollupFwFtpLogs.pl
EcDlRemap.pl
Sun ONE Web Server
EcDlDensityMapUtility.pl
EcDlMostRecentInsert.pl
File of RecentGranules
EcDlPopulateStatTables.pl
DlDbArchiveAccessStatDlDbDeleteAccessStatDlDbRestoreAccessStat
Subsystems and CSCIs: DPL (Cont.)DPMNT Architecture and Interfaces
Create files containing information about recently inserted granules
149625-EMD-002, Rev. 02DPL
DPMNT
DPMGT
WebBrowser
DAAC Operator
Data Pool Maintenance GUI
EcDlCleanupDataPool.pl
EcDlUpdateGranule.pl
EcDlRollupWebLogs.plEcDlRollupFwFtpLogs.pl
EcDlRemap.pl
Sun ONE Web Server
EcDlDensityMapUtility.pl
EcDlMostRecentInsert.pl
File of RecentGranules
EcDlPopulateStatTables.pl
DlDbArchiveAccessStatDlDbDeleteAccessStatDlDbRestoreAccessStat
Subsystems and CSCIs: DPL (Cont.)DPMNT Architecture and Interfaces
Calculate spatial
density map
information about
collections; store
in the database
150625-EMD-002, Rev. 02DPL
DPMNT
DPMGT
WebBrowser
DAAC Operator
Data Pool Maintenance GUI
EcDlCleanupDataPool.pl
EcDlUpdateGranule.pl
EcDlRollupWebLogs.plEcDlRollupFwFtpLogs.pl
EcDlRemap.pl
Sun ONE Web Server
EcDlDensityMapUtility.pl
EcDlMostRecentInsert.pl
File of RecentGranules
EcDlPopulateStatTables.pl
DlDbArchiveAccessStatDlDbDeleteAccessStatDlDbRestoreAccessStat
Subsystems and CSCIs: DPL (Cont.)DPMNT Architecture and Interfaces
Populate database
tables that
maintain counts of
granules by
collection/group
151625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: DPL (Cont.)
Data Pool(DPL)
DATAPOOL
Data Pool Management(DPMGT) CSCI
COTS FTPServer
wuftp
Data PoolWeb Access GUI
EcDlWebAccess
Data PoolDatabase
Sybase Server (COTS)
Data PoolInsert Utility
EcDlInsertUtilityEcDlActionDriverEcDlQuickServer
EcDlM2XTApp
Data Pool Maintenance(DPMNT) CSCI
Data PoolUpdate Granule
ExpirationEcDlUpdateGranule.pl
DPM GUIEcDlDpmHomepage.pl
Data Pool AccessStatistics UtilityEcDlRollupWebLogs.plEcDlRollupFtpLogs.pl
Data PoolArchive Access
Statistics ScriptsDlDbArchiveAccessStatDlDbDeleteAccessStat
DlDbRestoreAccessStat
Data Pool CleanupEcDlCleanupDataPool.pl
Data PoolMost RecentInsert Utility
EcDlMostRecentInsert.pl
Data PoolCollection
Remapping UtilityEcDlRemap.pl
Data PoolDensity Map
UtilityEcDlDensityMapUtility.pl
Data PoolStatistics Table
Population UtilityEcDlPopulateStatTables.pl
152625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: CLS
Client (CLS)External/InternalUsers
153625-EMD-002, Rev. 02
Subsystems and CSCIs: CLS
• Client Subsystem (CLS)– User access to ECS services for ASTER
- Permits Data Acquisition Request to task ASTER instrument- Supports request of ASTER On-demand Products
– Provides user authentication and User Profile information to the Search and Order tool- Search and retrieval of data are performed by the EOS Data
Gateway (Version 0 Web Client)– Includes applications programs accessible via user interfaces
- EOSView- ASTER Data Acquisition Request (DAR) Tool- On-Demand Form Request Manager (ODFRM)
– Uses several COTS tools: Netscape Navigator, Netscape Enterprise Server, XVT (widget set and development tool for EOSView), and Interactive Data Language (IDL) (used in EOSView visualization features)
Client
154625-EMD-002, Rev. 02
Subsystems and CSCIs: CLS (Cont.)
• Workbench (WKBCH) CSCI– Includes a set of application programs that implement
functions of the CLS science user interface– Release 7 Workbench includes two tools
- EOSView (X/Motif-based)- ASTER DAR Tool (Java/HTML-based)
155625-EMD-002, Rev. 02
Science/DAAC User
CLS
WKBCH
EOSView
EcClWbJdt
HDFFile
WebBrowser
ODFRM CSCI
Subsystems and CSCIs: CLS (Cont.)WKBCH Architecture and Interfaces
156625-EMD-002, Rev. 02
Science/DAAC User
CLS
WKBCH
EOSView
EcClWbJdt
HDFFile
WebBrowser
ODFRM CSCI
Subsystems and CSCIs: CLS (Cont.)WKBCH Architecture and Interfaces
Read/Display
HDF File
Enter
HDF File
Name
157625-EMD-002, Rev. 02
Science/DAAC User
CLS
WKBCH
EOSView
EcClWbJdt
HDFFile
WebBrowser
ODFRM CSCI
Subsystems and CSCIs: CLS (Cont.)WKBCH Architecture and Interfaces
Invoke browser; Enter
User ID information;
submit service requests;
enter DAR parameters
158625-EMD-002, Rev. 02
Science/DAAC User
CLS
WKBCH
EOSView
EcClWbJdt
HDFFile
WebBrowser
ODFRM CSCI
Subsystems and CSCIs: CLS (Cont.)WKBCH Architecture and Interfaces
Send User ID
information; Submit
service requests,
Send DAR parameters
159625-EMD-002, Rev. 02
Science/DAAC User
CLS
WKBCH
EOSView
EcClWbJdt
HDFFile
WebBrowser
ODFRM CSCI
Subsystems and CSCIs: CLS (Cont.)WKBCH Architecture and Interfaces
User Profile Information
Request User Profile
160625-EMD-002, Rev. 02
Science/DAAC User
CLS
WKBCH
EOSView
EcClWbJdt
HDFFile
WebBrowser
ODFRM CSCI
Subsystems and CSCIs: CLS (Cont.)WKBCH Architecture and Interfaces
Session ID; DAR ID; Results of ASTER DAR search
Submit/Modify Data Acquisition Request (DAR) or submit request to search ASTER DAR database [to EcGwDARServer]; Subscription Request [to EcSbSubServer]; User Information [through CCS Middleware Support]; allvia EcCsMojoGateway
161625-EMD-002, Rev. 02
Science/DAAC User
CLS
WKBCH
EOSView
EcClWbJdt
HDFFile
WebBrowser
ODFRM CSCI
Subsystems and CSCIs: CLS (Cont.)WKBCH Architecture and Interfaces
Submit call to attach aData Processing Request(DPR) to a DAR
162625-EMD-002, Rev. 02
Subsystems and CSCIs: CLS (Cont.)
• Desktop (DESKT) CSCI– Provides a gateway server for communication with MSS User
Registration Server to support seamless user registration through the EOS Data Gateway (EDG) web client and to obtain or update user profile information
– User Profile Gateway - provides user profile information to the EDG for ECS users- User authentication- Submit/Update user information in profile
163625-EMD-002, Rev. 02
Subsystems and CSCIs: CLS (Cont.)DESKT Architecture and Interfaces
Science/DAAC User
CLSDESKT
WebBrowser
EcClDtUserProfileGatewayUserProfileFile(profile.odl)
(Designcontrolled underseparate contract)
V0 IMS
SMC Location
164625-EMD-002, Rev. 02
Subsystems and CSCIs: CLS (Cont.)DESKT Architecture and Interfaces
Science/DAAC User
CLSDESKT
WebBrowser
EcClDtUserProfileGatewayUserProfileFile(profile.odl)
(Designcontrolled underseparate contract)
V0 IMS
SMC Location
Send User Registration request; Get/Update User Profile; Send password change; Send Search & Order data parameters
Invoke browser; Submit
User Registration request;
Request/Update User Profile;
Request password change;
Enter Search & Order data
parameters
165625-EMD-002, Rev. 02
Subsystems and CSCIs: CLS (Cont.)DESKT Architecture and Interfaces
Science/DAAC User
CLSDESKT
WebBrowser
EcClDtUserProfileGatewayUserProfileFile(profile.odl)
(Designcontrolled underseparate contract)
V0 IMS
SMC LocationUser Profile
Information
Submit/Update
User Profile/Pass-
word information
166625-EMD-002, Rev. 02
Subsystems and CSCIs: CLS (Cont.)DESKT Architecture and Interfaces
Science/DAAC User
CLSDESKT
WebBrowser
EcClDtUserProfileGatewayUserProfileFile(profile.odl)
(Designcontrolled underseparate contract)
V0 IMS
SMC LocationSearch Inventory; Request Product
Results ofSearch
167625-EMD-002, Rev. 02
Subsystems and CSCIs: CLS (Cont.)
• On-Demand Form Request Manager (ODFRM) CSCI– Supports attachment of a Data Processing Request (DPR) to an
ASTER Data Acquisition Request (a required capability that is not used)
– ODFRM HTML pages and Common Gateway Interface (CGI) programs
– Creation of an on-demand processing request and its submission to the Planning Subsystem (PLS) is now done through the EOS Data Gateway (EDG) tool
– ASTER on-demand products- ASTER L1B*- ASTER DEM (Digital Elevation Model)*- ASTER higher-level products (AST_04, AST_05, AST_06V,
AST_06T, AST_06S, AST_07S, AST_07V, AST_09T, AST_09V, AST_09S, AST_08)
* Note: Requires special privilege (in User Profile) to use ODFRM to order this product
168625-EMD-002, Rev. 02
CLS
ODFRM
ODFRM
ECS Users
WebBrowser
Sun ONEWeb Server
EcClOdRequest(CGI Program)
WKBCH CSCI
Subsystems and CSCIs: CLS (Cont.)ODFRM Architecture and Interfaces
169625-EMD-002, Rev. 02
CLS
ODFRM
ODFRM
ECS Users
WebBrowser
Sun ONEWeb Server
EcClOdRequest(CGI Program)
WKBCH CSCI
Subsystems and CSCIs: CLS (Cont.)ODFRM Architecture and Interfaces
Request user profile info;
create On-Demand
Product (ODP) Requests
Invoke browser; request
user profile info; enter
on-demand products info
Submit on-demand request using an
attached DPR
170625-EMD-002, Rev. 02
CLS
ODFRM
ODFRM
ECS Users
WebBrowser
Sun ONEWeb Server
EcClOdRequest(CGI Program)
WKBCH CSCI
Subsystems and CSCIs: CLS (Cont.)ODFRM Architecture and Interfaces
Pass validated ODP request; request user profile
Submit on-demand product requests; submit standing on-demand ordersRequest user
profile; request order/request tracking
Request media
options
171625-EMD-002, Rev. 02
CLS
ODFRM
ODFRM
ECS Users
WebBrowser
Sun ONEWeb Server
EcClOdRequest(CGI Program)
WKBCH CSCI
Subsystems and CSCIs: CLS (Cont.)ODFRM Architecture and Interfaces
Return media options
Return user
profile; return
order status
Display user profile;
display media options;
display order status
172625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: CLS (Cont.)
Desktop(DESKT) CSCI
User ProfileGateway
EcClDtUserProfileGateway
Workbench(WKBCH) CSCI
HDF ProductViewing Tool
EOSView
ASTER DARTool
EcClWbJdt
On-Demand FormRequest Manager
(ODFRM) CSCI
HTML UserInterface
ODFRM
CGIProgramsEcClOdRequest
Client (CLS)External/InternalUsers
173625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: DMS
Data MgmtSubsystem
(DMS)
174625-EMD-002, Rev. 02
Subsystems and CSCIs: DMS
• Data Management Subsystem (DMS)– Provides one-way catalog interoperability between ECS and the
V0 Information Management System (IMS)– Supplies gateway processes to translate requests between V0
protocol and ECS– Maintains a Data Dictionary that stores ECS data collection
information (i.e., collection metadata, attributes, valid keywords) and mappings between this information and V0 to permit translation of requests between the systems
– Uses several COTS tools: RogueWave class libraries, Builder Xcessory (GUI builder tool), and Sybase ASE Server (for Data Dictionary database search and update)
Data MgmtSubsystem
175625-EMD-002, Rev. 02
Subsystems and CSCIs: DMS(Cont.)
• Data Dictionary (DDICT) CSCI– Manages definitions of data collections including metadata,
data domains (valid values), and data location– Stored in a relational Database Management System (DBMS)– Three major components
- Data Dictionary Server - provides DDICT client processes the ability to perform data searches, inserts, updates, or deletes to the DDICT database
- Data Dictionary Maintenance Tool - provides a GUI to insert, update, or delete schema information held in the DDICT database,and allows operations staff to modify database attributes (e.g.,valids, mapping)
- Data Dictionary ASE Server - COTS database server
176625-EMD-002, Rev. 02
DMS
DDICT
EcDmDdMaintenanceTool
ASE(Sybase)
EcDmDictServiceDatabase
EcDmDictServer
V0 GTWAY CSCI
V0 IMS
Subsystems and CSCIs: DMS (Cont.)DDICT Architecture and Interfaces
177625-EMD-002, Rev. 02
DMS
DDICT
EcDmDdMaintenanceTool
ASE(Sybase)
EcDmDictServiceDatabase
EcDmDictServer
V0 GTWAY CSCI
V0 IMS
Subsystems and CSCIs: DMS (Cont.)DDICT Architecture and Interfaces
Request attribute mappings
Attribute mappings
178625-EMD-002, Rev. 02
DMS
DDICT
EcDmDdMaintenanceTool
ASE(Sybase)
EcDmDictServiceDatabase
EcDmDictServer
V0 GTWAY CSCI
V0 IMS
Subsystems and CSCIs: DMS (Cont.)DDICT Architecture and Interfaces
Retrieved table data
Update tables
179625-EMD-002, Rev. 02
DMS
DDICT
EcDmDdMaintenanceTool
ASE(Sybase)
EcDmDictServiceDatabase
EcDmDictServer
V0 GTWAY CSCI
V0 IMS
Subsystems and CSCIs: DMS (Cont.)DDICT Architecture and Interfaces
Retrieve DDICT data
Update DDICT data
180625-EMD-002, Rev. 02
DMS
DDICT
EcDmDdMaintenanceTool
ASE(Sybase)
EcDmDictServiceDatabase
EcDmDictServer
V0 GTWAY CSCI
V0 IMS
Subsystems and CSCIs: DMS (Cont.)DDICT Architecture and Interfaces
Export valids
181625-EMD-002, Rev. 02
DMS
DDICT
EcDmDdMaintenanceTool
ASE(Sybase)
EcDmDictServiceDatabase
EcDmDictServer
V0 GTWAY CSCI
V0 IMS
Subsystems and CSCIs: DMS (Cont.)DDICT Architecture and Interfaces
Send/ Replace ESDT installation information
182625-EMD-002, Rev. 02
Subsystems and CSCIs: DMS (Cont.)DDICT Architecture and Interfaces
DMS
DDICT
EcDmDdMaintenanceTool
ASE(Sybase)
EcDmDictServiceDatabase
EcDmDictServer
V0 GTWAY CSCI
V0 IMS
Request communications support
Validate subscription qualifiers
183625-EMD-002, Rev. 02
Subsystems and CSCIs: DMS (Cont.)DDICT Architecture and Interfaces
DMS
DDICT
EcDmDdMaintenanceTool
ASE(Sybase)
EcDmDictServiceDatabase
EcDmDictServer
V0 GTWAY CSCI
V0 IMS
Request Management Services
184625-EMD-002, Rev. 02
Subsystems and CSCIs: DMS (Cont.)
• Version Zero Gateway (V0 GTWAY) CSCI– Provides one-way interoperability with the V0 Information
Management System (IMS) for inventory searches, browse requests, product orders, and price estimate requests; search results include links to URLs for Data Pool products
– At the LP DAAC, transmits ASTER product requests to the V0 IMS to allow billing by the billing and accounting system- Distributed Ordering, Reporting, Researching, and Accounting
Network (DORRAN)– Queries between V0 IMS and the ECS V0 GTWAY use the
Object Description Language (ODL) format– One component
- V0 to ECS Gateway Server - allows use of the EOS Data Gateway Web Client to search and request data and services defined within ECS
185625-EMD-002, Rev. 02
Subsystems and CSCIs: DMS (Cont.)V0 GTWAY Architecture & Interfaces
DMS
V0 GTWAY
DDICT CSCI
EcDmV0ToEcsGateway
V0 IMS
EXTERNAL
SUBSETTE
R
SubsetterLocation(e.g., DAAC)
O
186625-EMD-002, Rev. 02
DMS
V0 GTWAY
DDICT CSCI
EcDmV0ToEcsGateway
V0 IMS
EXTERNAL
SUBSETTE
R
SubsetterLocation(e.g., DAAC)
O
Subsystems and CSCIs: DMS (Cont.)V0 GTWAY Architecture & Interfaces
Request Attribute Mappings
Attribute Mappings
187625-EMD-002, Rev. 02
DMS
V0 GTWAY
DDICT CSCI
EcDmV0ToEcsGateway
V0 IMS
EXTERNAL
SUBSETTE
R
SubsetterLocation(e.g., DAAC)
O
Subsystems and CSCIs: DMS (Cont.)V0 GTWAY Architecture & Interfaces
Search inventory, request Browse data, product Acquire requests, request price estimate, request subsetted product, request on-demand product
Return product metadata, Browse data, price estimate, product request status, and URLs and expiration dates for Data Pool data, metadata, and browse
188625-EMD-002, Rev. 02
DMS
V0 GTWAY
DDICT CSCI
EcDmV0ToEcsGateway
V0 IMS
EXTERNAL
SUBSETTE
R
SubsetterLocation(e.g., DAAC)
O
Subsystems and CSCIs: DMS (Cont.)V0 GTWAY Architecture & Interfaces
Status of User Billing
Informa-tion
Product
Requests
requiring billin
g
and accounting
189625-EMD-002, Rev. 02
DMS
V0 GTWAY
DDICT CSCI
EcDmV0ToEcsGateway
V0 IMS
EXTERNAL
SUBSETTE
R
SubsetterLocation(e.g., DAAC)
O
Subsystems and CSCIs: DMS (Cont.)V0 GTWAY Architecture & Interfaces
Request input product for subsetting; request order status update Product request
acknowledgement; order status acknowledgement
190625-EMD-002, Rev. 02
DMS
V0 GTWAY
DDICT CSCI
EcDmV0ToEcsGateway
V0 IMS
EXTERNAL
SUBSETTE
R
SubsetterLocation(e.g., DAAC)
O
Subsystems and CSCIs: DMS (Cont.)V0 GTWAY Architecture & Interfaces
Search Inventory,
Request Browse
Data; Product
Acquire requests
for input data for
subsetting
191625-EMD-002, Rev. 02
DMS
V0 GTWAY
DDICT CSCI
EcDmV0ToEcsGateway
V0 IMS
EXTERNAL
SUBSETTE
R
SubsetterLocation(e.g., DAAC)
O
Subsystems and CSCIs: DMS (Cont.)V0 GTWAY Architecture & Interfaces
Return request
status
Request On-Demand
Product
192625-EMD-002, Rev. 02
DMS
V0 GTWAY
DDICT CSCI
EcDmV0ToEcsGateway
V0 IMS
EXTERNAL
SUBSETTE
R
SubsetterLocation(e.g., DAAC)
O
Subsystems and CSCIs: DMS (Cont.)V0 GTWAY Architecture & Interfaces
Request metadata
URLs, science data
URLs, browse URLs, and
expiration datesReturn metadata
URLs, science data
URLs, browse
URLs, and expiration dates
193625-EMD-002, Rev. 02
DMS
V0 GTWAY
DDICT CSCI
EcDmV0ToEcsGateway
V0 IMS
EXTERNAL
SUBSETTE
R
SubsetterLocation(e.g., DAAC)
O
Subsystems and CSCIs: DMS (Cont.)V0 GTWAY Architecture & Interfaces
Product Acquire Requests
194625-EMD-002, Rev. 02
DMS
V0 GTWAY
DDICT CSCI
EcDmV0ToEcsGateway
V0 IMS
EXTERNAL
SUBSETTE
R
SubsetterLocation(e.g., DAAC)
O
Subsystems and CSCIs: DMS (Cont.)V0 GTWAY Architecture & Interfaces
Request
communications
support
195625-EMD-002, Rev. 02
DMS
V0 GTWAY
DDICT CSCI
EcDmV0ToEcsGateway
V0 IMS
EXTERNAL
SUBSETTE
R
SubsetterLocation(e.g., DAAC)
O
Subsystems and CSCIs: DMS (Cont.)V0 GTWAY Architecture & Interfaces
Request Management Services
(e.g., order/request tracking
update, User Profile request)
196625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: DMS (Cont.)
Data Dictionary(DDICT) CSCI
Data DictionaryMaintenance Tool
EcDmMaintTool
Data DictionaryASE Server
Sybase Server (COTS)
Data DictionaryServer
EcDmDictService
Version Zero Gateway(V0 GTWAY) CSCI
V0 to ECSGateway ServerEcDmV0ToECSGateway
Data MgmtSubsystem
(DMS)
197625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: OMS
Order ManagerSubsystem
(OMS)
198625-EMD-002, Rev. 02
Subsystems and CSCIs: OMS
• Order Manager Subsystem (OMS)– Manages orders arriving via the following routes:
- DMS V0 Gateway (i.e., from EDG, ECHO, and GDS users)- Spatial Subscription Server (NBSRV)- CSS Machine-to-Machine Gateway- SCLI- Data conversion (HEG) requests or media orders from the Data Pool
Web Access GUI– Provides an Order Manager Server that receives the data
distribution orders and forwards them to the appropriate ECS services- Orders for electronic distribution either filled from the Data Pool or
sent to SDSRV (as appropriate)- Orders for media distribution sent to PDS
– Provides a Graphical User Interface (GUI) for monitoring and controlling Order Manager operations
Order ManagerSubsystem
(OMS)
199625-EMD-002, Rev. 02
Subsystems and CSCIs: OMS
• Order Manager Subsystem (OMS) (Cont.)– Uses several COTS tools: RogueWave class libraries, Sybase
Software Developer’s Kit (formerly Open Client), Sybase ASE Server
Order ManagerSubsystem
(OMS)
200625-EMD-002, Rev. 02
Subsystems and CSCIs: OMS (Cont.)
• Order Manager Server (OMSRV) CSCI– Receives Product Distribution Requests from the various
sources– Immediately stores request information in a relational Database
Management System (DBMS)– Validates the requests for correctness (e.g., request size, media
capacity, accessibility, validity of UR)– Queues HEG requests and dispatches individual line items to
HEG services, which subset the individual line items– If the media type or ESDTs of a request are configured for
Synergy III processing mode or the order manager server is configured for Synergy III mode, dispatches each validated request to the appropriate ECS order-fulfillment service; e.g., PDS or SDSRV, depending on whether the request is for physical media or electronic distribution
201625-EMD-002, Rev. 02
Subsystems and CSCIs: OMS (Cont.)
• Order Manager Server (OMSRV) CSCI (Cont.)– Otherwise, stages the order to Data Pool storage (and creates
links from staged files to the FtpPull directory in the Data Pool storage if the distribution type is FtpPull), distributes the order to the appropriate service (i.e., Production Module or OMS Ftp Driver) depending on whether distribution type is media or FtpPush, then sends a Distribution Notice to the end user when the order is considered shipped
– Fills valid electronic distribution requests from DPL or submitsvalid requests to SDSRV or PDS (as appropriate)
– Generates Operator Interventions for invalid requests– Two major components
- Order Manager Server - interacts with the Order Manager Database, Order Manager Ftp Push Server, Data Pool, Product Distribution System, and Science Data Server
- Order Manager ASE Server - COTS database server
202625-EMD-002, Rev. 02
Subsystems and CSCIs: OMS (Cont.)
• Order Manager GUI (OMGUI) CSCI– Based on web standards– Communicates directly with the Order Manager Service
database (not through a server)– Permits monitoring and control of the OMSRV (e.g., view
operator alerts, view status of queues, suspend and resume queues, view and modify configuration parameters, monitor statistics)
– Monitor open Operator Interventions for invalid requests; manage interventions; view closed interventions; view, cancel, suspend, resume, or resubmit distribution requests; view order information and user profile data; view, update, or cancel bundling orders
– One major component- Order Manager GUI - interacts with the Order Manager Database
203625-EMD-002, Rev. 02
Subsystems and CSCIs: OMS (Cont.)
• Production Module CSCI– Processes physical media distribution (PMD) requests
(Synergy V feature)– One major component
- EcOmPdMediaProduction - processes product distribution requests on physical media
204625-EMD-002, Rev. 02
Subsystems and CSCIs: OMS (Cont.)OMS Architecture and Interface
OMS
OMSRVASE(Sybase)
OMS Database
EcOmOrderManager
OMGUI
EcOmGui
Production ModuleEcOmPdMediaProduction
205625-EMD-002, Rev. 02
OMS
OMSRVASE(Sybase)
OMS Database
EcOmOrderManager
OMGUI
EcOmGui
Production ModuleEcOmPdMediaProduction
Subsystems and CSCIs: OMS (Cont.)OMS Architecture and Interface
Insert Product Distribution Request (EcNbActionDriver.pl)
Insert Product Distribution Request (EcCsMtMGateway)
206625-EMD-002, Rev. 02
OMS
OMSRVASE(Sybase)
OMS Database
EcOmOrderManager
OMGUI
EcOmGui
Production ModuleEcOmPdMediaProduction
Subsystems and CSCIs: OMS (Cont.)OMS Architecture and Interface
Granule staging request
Insert Hard Media Distribution Request or HEG Request (EcDlWebAccess)
Insert Product Distribution Request (EcDmV0ToEcsGateway)
207625-EMD-002, Rev. 02
OMS
OMSRVASE(Sybase)
OMS Database
EcOmOrderManager
OMGUI
EcOmGui
Production ModuleEcOmPdMediaProduction
Subsystems and CSCIs: OMS (Cont.)OMS Architecture and Interface
Create/update Product Distribution Request; create Operator Intervention; create Alert; submit granule staging request
Retrieve Distribution Request; retrieve configuration information; retrieve e-mail information
208625-EMD-002, Rev. 02
OMS
OMSRVASE(Sybase)
OMS Database
EcOmOrderManager
OMGUI
EcOmGui
Production ModuleEcOmPdMediaProduction
Subsystems and CSCIs: OMS (Cont.)OMS Architecture and Interface
Order information; user profile information
209625-EMD-002, Rev. 02
OMS
OMSRVASE(Sybase)
OMS Database
EcOmOrderManager
OMGUI
EcOmGui
Production ModuleEcOmPdMediaProduction
Subsystems and CSCIs: OMS (Cont.)OMS Architecture and Interface
Retrieve OM Server information, Inter-vention Requests, and PMD actions
Update OM Server configuration data; update/respond to Intervention Requests; control PMD operations
210625-EMD-002, Rev. 02
OMS
OMSRVASE(Sybase)
OMS Database
EcOmOrderManager
OMGUI
EcOmGui
Production ModuleEcOmPdMediaProduction
Subsystems and CSCIs: OMS (Cont.)OMS Architecture and Interface
Pre-Synergy IV: Submit electronic distribution request
Synergy IV/V: FtpPush request
211625-EMD-002, Rev. 02
OMS
OMSRVASE(Sybase)
OMS Database
EcOmOrderManager
OMGUI
EcOmGui
Production ModuleEcOmPdMediaProduction
Subsystems and CSCIs: OMS (Cont.)OMS Architecture and Interface
Pre-Syn
ergy V
:
Dispatc
h har
d
media
distrib
ution
reque
st
Synerg
y V:
Dispatc
h phys
ical
media
distrib
ution
reque
st
Retrieve PMD request informa-tion; update PMD request information
212625-EMD-002, Rev. 02
OMS
OMSRVASE(Sybase)
OMS Database
EcOmOrderManager
OMGUI
EcOmGui
Production ModuleEcOmPdMediaProduction
Subsystems and CSCIs: OMS (Cont.)OMS Architecture and Interface
Submit HEG request
213625-EMD-002, Rev. 02
OMS
OMSRVASE(Sybase)
OMS Database
EcOmOrderManager
OMGUI
EcOmGui
Production ModuleEcOmPdMediaProduction
Subsystems and CSCIs: OMS (Cont.)OMS Architecture and Interface
Request commun-ications support (e.g., send e-mail)
214625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: OMS (Cont.)
Order Manager GUI(OMGUI) CSCI
Order Manager GUIEcOmGui
Order Manager Server(OMSRV) CSCI
Order ManagerServer
EcOmOrderManager
Order ManagerASE Server
Sybase Server (COTS)
Order ManagerSubsystem
(OMS)
Production Module CSCI
ProductionModule
EcOmPdMediaProduction
215625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
No more PDS Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: PLS
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
216625-EMD-002, Rev. 02
Subsystems and CSCIs: PLS
• Planning Subsystem (PLS)– Allows operations staff to define
data processing tasks to be performed at a site– Generates efficient plans for scheduling defined data
processing and reprocessing tasks according to production rules that define how a Product Generation Executive (PGE) is to run
– Coordinates production with the Data Server and Data Processing subsystems to achieve a highly automated production system
– Interfaces with the Algorithm Integration and Test Tools CSCI within DPS for information on PGEs
– Permits entry of Production Requests and generates resulting Data Processing Requests (DPRs)
– Uses a set of Raytheon-provided COTS libraries as a basis for its scheduling components (Resource Planning Workbench and Production Planning Workbench)
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
217625-EMD-002, Rev. 02
Subsystems and CSCIs: PLS (Cont.)
• Production Planning (PLANG) CSCI– Eight major components
- Data Store - handles insertion of data for planning and processing activities into the PDPS shared database
- Resource Planning Workbench - GUIs for preparing a site resource schedule [Resource Editor (EcPlRpRe), Scheduling Interface (EcPlRpSi), Timeline (EcPlRpTl)]
- Production Request Editor - GUI for submitting production requests that describe the data products to be produced; uses PGE descriptions to generate the DPRs necessary to meet the requests(EcPlPREditor_IF)
- Production Planning Workbench - GUIs and a script for preparing a site production schedule [Workbench (EcPlWb), Timeline (EcPlTl) and Chain Determination (EcPlDetermineChain.pl)]
- On-Demand Production Request Manager - receives requests for data from the scientist via the EDG web page, generates the necessary Production Request, submits it for processing, and distributes the data to the scientist (EcPlOdMgr)
218625-EMD-002, Rev. 02
Subsystems and CSCIs: PLS (Cont.)
• Production Planning (PLANG) CSCI (Cont.)– Eight major components (Cont.)
- Production Request Generator - command-line interface for creating production requests (EcPlPRGenerator)
- Subscription Manager - server to manage receipt of subscription notifications from the Data Server via SBSRV (EcPlSubMgr)
- Production Strategies GUI - used to create a set of planning priorities to be applied to each DPR in a plan (EcPlProdStrat)
219625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
220625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Request Management Support
221625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Submit On-Demand Product request; Submit Standing On-Demand orders (from V0 Gateway)
222625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Tracking elements (e.g., order ID,request ID)
Request On-Demand tracking elements; request standing order tracking elements
223625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Search inventory; submit Acquire
224625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Subscribe
225625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Store/Update PRs
Retrieve PRs
226625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Submit DPR
227625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Search inventory; Query for metadata attributes
228625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Subscribe
229625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Create/Store PRs;
Create/Store DPRs;
Delete PRs;
Delete DPRs
Retrieve PRs;
Retrieve DPRs
230625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Search inventory; Query for metadata attributes
231625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Subscribe
232625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Create/Store PRs;Create/Store DPRs;
233625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Submit DPR
234625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Retrieve productionstrategiesCreate/update production strategies
235625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Retrieve plans
Store/Update plans
236625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
DPR scheduling information
237625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Submit/cancel DPR; Submit ground event
Update chaininfoRequest chain
determination
238625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Subscriptionnotification
239625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Retrieve information relating jobs to subscriptions
Update with data availability notification
240625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Metadata attribute
query (to determine
which jobs can use
an available
granule as input)
241625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Release DPR
242625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Delete granules (for cancelled DPR)
243625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Store/Update resource definitions
Retrieve resource definitions
244625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Define/Update ground events
Retrieve ground events
245625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Cancel ground event
246625-EMD-002, Rev. 02
PLSPLANG
EcPlSubMgr
EcPlPREditor_IF
EcPlRpRe
EcPlRpSi
EcPlTl
EcPlProdStratEcPlRpTl
PDPSDatabase
ASE(Sybase)
EcPlWbEcPlOdMgr
EcPlPRGenerator
EcPlDetermineChain.pl
Subsystems and CSCIs: PLS (Cont.)PLANG Architecture and Interfaces
Resource reservation schedule
247625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
Subsystems and CSCIs: PLS (Cont.)
Production Planning(PLANG) CSCI
DataStore
Sybase Database (COTS)
Resource PlanningWorkbench
EcPlRpReEcPlRpSiEcPlRpTl
ProductionRequest Editor
EcPlPrEditor_IF
ProductionPlanning Workbench
EcPlWbEcPlTl
EcPlDetermineChain.pl
On-DemandProduction RequestManager (ODPRM)
EcPlOdMgr
SubscriptionManagerEcPlSubMgr
ProductionStrategies GUI
EcPlProdStrat
ProductionRequest Generator
EcPlPRGenerator
248625-EMD-002, Rev. 02
Subsystems and CSCIs: DPS
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
249625-EMD-002, Rev. 02
Subsystems and CSCIs: DPS
• Data Processing Subsystem (DPS)– Manages allocation and recovery of computer resources (e.g.,
CPU, disk space) used in processing science data– Manages, queues, and executes DPRs– Supports execution of science algorithms through the Science
Data Processing (SDP) Toolkit– Supports preliminary processing of ancillary data sets– Provides an Algorithm Integration and Test (AIT) environment
for the introduction of science software– Provides a Quality Assessment (QA) environment for updating
the quality flags in metadata for data products
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
250625-EMD-002, Rev. 02
Subsystems and CSCIs: DPS (Cont.)
• Data Processing Subsystem (DPS) (Cont.)– Uses COTS tools
- AutoSys: a job scheduling software application to automate operations in a distributed UNIX environment
- Job Management Web Interfaces: provide interfaces to monitor and manage the job schedule being processed in AutoSys
- Sybase: ASE server
251625-EMD-002, Rev. 02
Subsystems and CSCIs: DPS (Cont.)
• Processing (PRONG) CSCI– Provides services required to manage and monitor the Science
Data Processing environment, which executes Science Software items (PGEs) and produces data products
– Nine major components- Job Management - handles flow of information to the COTS
products; also creates and starts Ground Event jobs- Execution Management - initiates execution of PGEs and performs
final activities subsequent to execution of PGEs; handles flow of science data to and from science processing resources (through adata management software library, DpPrDM); also provides status of On-demand Processing requests
- PGE Management - controls and monitors execution of PGEs and the growth of the output products (EcDpPrRunPGE); measures and reports resource use to AutoSys (EcDpPrRusage)
252625-EMD-002, Rev. 02
Subsystems and CSCIs: DPS (Cont.)
• Processing (PRONG) CSCI (Cont.)– Nine major components (Cont.)
- Deletion Server - notifies Science Data Server to remove interim granules that are no longer needed
- Quality Assurance Monitor - supports visualizing science data products and updating QA metadata
- Data Preprocessing - manages preprocessing of ancillary data used as inputs to a PGE
- AutoSys - provides the job scheduling engine (COTS)- Data Store - handles insertion of data for planning and processing
activities into the PDPS shared database- Ground Event Process - initiated by Job Management upon receipt
of a ground event request; sets a computer resource to an off-line state, making it unavailable for PGEs during the request
253625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
254625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Submit requests to create, release, cancel DPR jobs (and ground events)
255625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Create, release, cancel job box; create/cancel ground event job
256625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
InitiateExecution
257625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Initiate execu-tion of a job
258625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Monitorstatus
259625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Display Job States
260625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Request/Allocate disk space; update PGE information, request CPU and RAM allocation
261625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Request MCF;
insert granules;
request PGE tar
file; insert product
history tar files;
insert Failed PGE
tar file
Granule acquires; PGE tar file;
MCF
262625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Retrieve data,
granule information
Update data,
granule information
263625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Updatemetadata (QA flag)
Retrieve metadata; Retrieve Product History
264625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Request (interim) granule deletion
265625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Delete granules (associated with cancelled DPR)
266625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Send On-Demand Failure Message
267625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Update Request
Status
268625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Request
Communications
Support
269625-EMD-002, Rev. 02
DPSPRONG
PDPSDatabase
ASE(Sybase)
EcDpPrJobMgmtEcDpPrJobMgmtClient
EcDpPrDeletion
AutoSys
EcDpPrEM EcDpPrQaMonitorGUI
EcDpPrRunPGEEcDpPrRusage
PGEs (e.g., EcDpPrAm1EdosEphAttDPREP_PGE,EcDpPrAm1FddAttitudeDPREP_PGE,EcDpPrAm1FddEphemerisDPREP_PGE)
EcDpPrViewJobStates
EcDpPrGE
User
Subsystems and CSCIs: DPS (Cont.)PRONG Architecture and Interfaces
Management Services
270625-EMD-002, Rev. 02
Subsystems and CSCIs: DPS (Cont.)
• Algorithm Integration and Test Tools (AITTL) CSCI– Provides a set of tools used for testing and integration of new
science software, new versions of science software, and user methods into the Science Data Processing operational environment
– Combines custom-developed code with COTS software– Tools are accessed from a centralized application called the
Science Software Integration and Test (SSIT) Manager
271625-EMD-002, Rev. 02
Subsystems and CSCIs: DPS (Cont.)
• Algorithm Integration and Test Tools (AITTL) CSCI (Cont.)– Six major components
- Science Software Archive Package (SSAP) GUI - allows for the creation, update, and deletion of SSAPs
- SSIT Manager - GUI for SSIT activities; provides menus to launch other SSIT applications and a checklist to mark completion of SSIT functions
- Define PGE - a group of applications to specify a PGE in the PDPS database
- View/Compare Tools - a group of applications for viewing and comparing data files
- Check Software Tools - a group of applications that check the source code for PGEs and their process control files (PCFs) for errors or prohibited functions
- Insert/Acquire Tools - a group of applications that provide mechanisms to insert and acquire data items from Data Server
272625-EMD-002, Rev. 02DPSAITTL
ASE (Sybase)PDPSDatabase
EcDpAtSSAPGui
PGE/ESDTODLFiles
Define
PGE
View/Compare
EcDpAtBinDiffGui
EcDpAtMgrXdiff
EcDpAtCheckHdfFile
EcDpAtRegisterPGE
EcDpAtCheckODL
EcDpAtCreateODLTemplate
EcDpAtOpDbGui
EcDpAtMgr(SSIT Manager)
EcDpAtStageDAP
EcDpAtInsertTestFile
EcDpAtInsertExeTarFile
EcDpAtGetMCF
Insert/Acquire
EcDpAtInsertStaticFile
EcDpAtMgrLogDump
EcDpAtCheckPCF
EcDpAtPrologs
EcDpAtBadFuncGui
EcDpAtBadFunc
CheckSoftware
Subsystems and CSCIs: DPS (Cont.)AITTL Architecture and Interfaces
273625-EMD-002, Rev. 02DPSAITTL
ASE (Sybase)PDPSDatabase
EcDpAtSSAPGui
PGE/ESDTODLFiles
Define
PGE
View/Compare
EcDpAtBinDiffGui
EcDpAtMgrXdiff
EcDpAtCheckHdfFile
EcDpAtRegisterPGE
EcDpAtCheckODL
EcDpAtCreateODLTemplate
EcDpAtOpDbGui
EcDpAtMgr(SSIT Manager)
EcDpAtStageDAP
EcDpAtInsertTestFile
EcDpAtInsertExeTarFile
EcDpAtGetMCF
Insert/Acquire
EcDpAtInsertStaticFile
EcDpAtMgrLogDump
EcDpAtCheckPCF
EcDpAtPrologs
EcDpAtBadFuncGui
EcDpAtBadFunc
CheckSoftware
Subsystems and CSCIs: DPS (Cont.)AITTL Architecture and Interfaces
Initiate Execution
274625-EMD-002, Rev. 02DPSAITTL
ASE (Sybase)PDPSDatabase
EcDpAtSSAPGui
PGE/ESDTODLFiles
Define
PGE
View/Compare
EcDpAtBinDiffGui
EcDpAtMgrXdiff
EcDpAtCheckHdfFile
EcDpAtRegisterPGE
EcDpAtCheckODL
EcDpAtCreateODLTemplate
EcDpAtOpDbGui
EcDpAtMgr(SSIT Manager)
EcDpAtStageDAP
EcDpAtInsertTestFile
EcDpAtInsertExeTarFile
EcDpAtGetMCF
Insert/Acquire
EcDpAtInsertStaticFile
EcDpAtMgrLogDump
EcDpAtCheckPCF
EcDpAtPrologs
EcDpAtBadFuncGui
EcDpAtBadFunc
CheckSoftware
Subsystems and CSCIs: DPS (Cont.)AITTL Architecture and Interfaces
Request/insert SSAP information
SSAP information
275625-EMD-002, Rev. 02DPSAITTL
ASE (Sybase)PDPSDatabase
EcDpAtSSAPGui
PGE/ESDTODLFiles
Define
PGE
View/Compare
EcDpAtBinDiffGui
EcDpAtMgrXdiff
EcDpAtCheckHdfFile
EcDpAtRegisterPGE
EcDpAtCheckODL
EcDpAtCreateODLTemplate
EcDpAtOpDbGui
EcDpAtMgr(SSIT Manager)
EcDpAtStageDAP
EcDpAtInsertTestFile
EcDpAtInsertExeTarFile
EcDpAtGetMCF
Insert/Acquire
EcDpAtInsertStaticFile
EcDpAtMgrLogDump
EcDpAtCheckPCF
EcDpAtPrologs
EcDpAtBadFuncGui
EcDpAtBadFunc
CheckSoftware
Subsystems and CSCIs: DPS (Cont.)AITTL Architecture and Interfaces
Request PGE &
data insertion;
Request MCF;
Request DAPs;
Request product
276625-EMD-002, Rev. 02DPSAITTL
ASE (Sybase)PDPSDatabase
EcDpAtSSAPGui
PGE/ESDTODLFiles
Define
PGE
View/Compare
EcDpAtBinDiffGui
EcDpAtMgrXdiff
EcDpAtCheckHdfFile
EcDpAtRegisterPGE
EcDpAtCheckODL
EcDpAtCreateODLTemplate
EcDpAtOpDbGui
EcDpAtMgr(SSIT Manager)
EcDpAtStageDAP
EcDpAtInsertTestFile
EcDpAtInsertExeTarFile
EcDpAtGetMCF
Insert/Acquire
EcDpAtInsertStaticFile
EcDpAtMgrLogDump
EcDpAtCheckPCF
EcDpAtPrologs
EcDpAtBadFuncGui
EcDpAtBadFunc
CheckSoftware
Subsystems and CSCIs: DPS (Cont.)AITTL Architecture and Interfaces
Insert/Update
Granules and
PGE Information
277625-EMD-002, Rev. 02DPSAITTL
ASE (Sybase)PDPSDatabase
EcDpAtSSAPGui
PGE/ESDTODLFiles
Define
PGE
View/Compare
EcDpAtBinDiffGui
EcDpAtMgrXdiff
EcDpAtCheckHdfFile
EcDpAtRegisterPGE
EcDpAtCheckODL
EcDpAtCreateODLTemplate
EcDpAtOpDbGui
EcDpAtMgr(SSIT Manager)
EcDpAtStageDAP
EcDpAtInsertTestFile
EcDpAtInsertExeTarFile
EcDpAtGetMCF
Insert/Acquire
EcDpAtInsertStaticFile
EcDpAtMgrLogDump
EcDpAtCheckPCF
EcDpAtPrologs
EcDpAtBadFuncGui
EcDpAtBadFunc
CheckSoftware
Subsystems and CSCIs: DPS (Cont.)AITTL Architecture and Interfaces
Request
ODL File
Receive
PGE Data
278625-EMD-002, Rev. 02DPSAITTL
ASE (Sybase)PDPSDatabase
EcDpAtSSAPGui
PGE/ESDTODLFiles
Define
PGE
View/Compare
EcDpAtBinDiffGui
EcDpAtMgrXdiff
EcDpAtCheckHdfFile
EcDpAtRegisterPGE
EcDpAtCheckODL
EcDpAtCreateODLTemplate
EcDpAtOpDbGui
EcDpAtMgr(SSIT Manager)
EcDpAtStageDAP
EcDpAtInsertTestFile
EcDpAtInsertExeTarFile
EcDpAtGetMCF
Insert/Acquire
EcDpAtInsertStaticFile
EcDpAtMgrLogDump
EcDpAtCheckPCF
EcDpAtPrologs
EcDpAtBadFuncGui
EcDpAtBadFunc
CheckSoftware
Subsystems and CSCIs: DPS (Cont.)AITTL Architecture and Interfaces
Request PGE Insertion or Update
279625-EMD-002, Rev. 02DPSAITTL
ASE (Sybase)PDPSDatabase
EcDpAtSSAPGui
PGE/ESDTODLFiles
Define
PGE
View/Compare
EcDpAtBinDiffGui
EcDpAtMgrXdiff
EcDpAtCheckHdfFile
EcDpAtRegisterPGE
EcDpAtCheckODL
EcDpAtCreateODLTemplate
EcDpAtOpDbGui
EcDpAtMgr(SSIT Manager)
EcDpAtStageDAP
EcDpAtInsertTestFile
EcDpAtInsertExeTarFile
EcDpAtGetMCF
Insert/Acquire
EcDpAtInsertStaticFile
EcDpAtMgrLogDump
EcDpAtCheckPCF
EcDpAtPrologs
EcDpAtBadFuncGui
EcDpAtBadFunc
CheckSoftware
Subsystems and CSCIs: DPS (Cont.)AITTL Architecture and Interfaces
Request Communications
Support
280625-EMD-002, Rev. 02DPSAITTL
ASE (Sybase)PDPSDatabase
EcDpAtSSAPGui
PGE/ESDTODLFiles
Define
PGE
View/Compare
EcDpAtBinDiffGui
EcDpAtMgrXdiff
EcDpAtCheckHdfFile
EcDpAtRegisterPGE
EcDpAtCheckODL
EcDpAtCreateODLTemplate
EcDpAtOpDbGui
EcDpAtMgr(SSIT Manager)
EcDpAtStageDAP
EcDpAtInsertTestFile
EcDpAtInsertExeTarFile
EcDpAtGetMCF
Insert/Acquire
EcDpAtInsertStaticFile
EcDpAtMgrLogDump
EcDpAtCheckPCF
EcDpAtPrologs
EcDpAtBadFuncGui
EcDpAtBadFunc
CheckSoftware
Subsystems and CSCIs: DPS (Cont.)AITTL Architecture and Interfaces
ManagementServices
281625-EMD-002, Rev. 02
Subsystems and CSCIs: DPS (Cont.)
• SDP Toolkit (SDPTK) CSCI– Not described in detail in this course – Provides a set of software tools used to integrate Science
Software into ECS– Provides common functionality (e.g., geolocation) required
across the ECS community– Allows Science Data Processing to support generation of data
products in a heterogeneous computer hardware environment- Facilitates the smooth transition and integration of science
software code into the DAAC by abstracting out science process dependencies on external system architecture
- Provides an interface between science software and the production system environment
- Interface is implemented in both the SCF development environments and DAAC production environments
282625-EMD-002, Rev. 02
Subsystems and CSCIs: DPS (Cont.)
• SDP Toolkit (SDPTK) CSCI (Cont.)– Insulates science software from the SDP software and provides
a development environment that emulates critical SDP functions- Helps ensure code portability as the algorithm is ported from
development hardware, through the DAAC system, and through potential hardware changes as ECS matures
- Provides for limited access and control to system level resources, including processes, shared memory, and I/O capabilities
- Where control of system resources is necessary (e.g.,shared memory allocation), Toolkit provides a set of routines through which the application must obtain those services
- Partitioning and layering of operating system services allows Toolkit to work on behalf of DPS in allocating, deallocating, and making use of system-wide shared resources
283625-EMD-002, Rev. 02
Subsystems and CSCIs: DPS (Cont.)
– SDP Toolkit Tools - Mandatory- File I/O Tools- Error/Status Reporting [Status
Message File (SMF) Tools]- Process Control Tools- Shared Memory Management
Tools- Bit Manipulation Tools- Spacecraft Ephemeris and
Attitude Data Access Tools- Time and Date Conversion
Tools
– SDP Toolkit Tools - Optional - Digital Elevation Model Tools- Ancillary Data Tools- Celestial Body Position Tools- Coordinate System
Conversion Tools- Geo-Coordinate
Transformation Tools- Math and Statistical Support
Tools- Constants and Unit
Conversions- Dynamic Memory Management
Tools- Graphics Support Tools
• SDP Toolkit (SDPTK) CSCI (Cont.)
284625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
Subsystems and CSCIs: DPS (Cont.)
Algorithm Integration and Test Tools(AITTL) CSCI
SSAP GUIEcDpAtSSAPGui
SSIT ManagerEcDpAtMgr
Define PGEEcDpAtCheckODL
EcDpAtCreateODLTemplateEcDpAtRegisterPGE
EcDpAtOpDbGui
View/CompareData ToolsEcDpAtBinDiffGui
EcDpAtCheckHdfFileEcDpAtMgrXdiff
Check SoftwareTools
EcDpAtCheckPCFEcDpAtBadFuncGui
EcDpAtBadFuncEcDpAtPrologs
Insert/AcquireTools
EcDpAtMgrLogDumpEcDpAtInsertStaticFile
EcDpAtInsertExeTarFileEcDpAtInsertTestFile
EcDpAtStageDAPEcDpAtGetMCF
SDP Toolkit(SDPTK) CSCI
(Separatesoftware libraries
for SSI&T)
Processing(PRONG) CSCI
ExecutionManagement
EcDpPrEMDpPrDM (library)
PGEManagement
EcDpPrRunPGEEcDpPrRusage
DeletionServer
EcDpPrDeletion
Job ManagementEcDpPrJobMgmt
EcDpPrJobMgmtClientEcDpPrViewJobStates
AutoSysCOTS Product
QAMonitor
EcDpPrQaMonitorGUI
DataStore
Sybase Database (COTS)
Ground EventProcess
EcDpPrGE
Data PreprocessingEcDpPrAm1EdosEphAttDPREP_PGEEcDpPrAm1FddAttitudeDPREP_PGE
EcDpPrAm1FddEphemerisDPREP_PGEEcDpPrDumpAttitudeDPREP
EcDpPrDumpEphemerisDPREPEcDpPrPm1FddEphemerisDPREP_PGE
EcDpPrPm1AttitudeDPREP_PGEEcDpPrAuraEphemerisDPREP_PGE
EcDpPrAuraAttitudeDPREP_PGE
285625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: MSS
System MgmtSubsystem
MSS
286625-EMD-002, Rev. 02
Subsystems and CSCIs: MSS
• System Management Subsystem (MSS)– Provides the set of tools needed by
maintenance and operations staff to manage ECS operations– Addresses five areas
- Fault Management- Configuration Management- Accountability Management- Performance Management- Security Management
– Installed locally at each DAAC and at System Monitoring and Coordination Center (SMC)
– Uses COTS applications extensively, including Sybase Replication Server
– Includes ECS Assistant, a GUI that runs an extensive array of UNIX scripts for system installation, monitoring, and administration
System MgmtSubsystem
(MSS)
287625-EMD-002, Rev. 02
Subsystems and CSCIs: MSS (Cont.)
• Management CSCI (MCI)– Primarily COTS-based, with some custom software– Provides services for monitoring and coordinating ECS– Network and Enterprise Management Framework component
- Whazzup???- Monitors server status- Monitors host resource usage
- WhatsUp Gold- Network monitoring- Fault detection
– Security Service component- Various freeware or public domain packages- Monitor and evaluate security and report status
288625-EMD-002, Rev. 02
Subsystems and CSCIs: MSS (Cont.)
• Management CSCI (MCI) (Cont.)– Accountability Management Service (AMS) component
- Custom software- Account Management Tool (for User Registration and User Profile
updates)- Order Tracking Tool- Sybase ASE Server/Sybase Replication Server
289625-EMD-002, Rev. 02
Subsystems and CSCIs: MSS (Cont.)
• Management CSCI (MCI) (Cont.)– Trouble Ticketing component
- Custom-configured COTS software: Remedy Action Request System
– Network Backup/Restore component- COTS software: Legato Networker
– ASTER E-Mail Header Handler component- Custom scripts work with COTS e-mail to add a formatted header
to all e-mail exchanges between the ASTER Ground Data System and ECS
290625-EMD-002, Rev. 02
O
MSS
MCI - Accountability
EcMsAcDAACRegUserGUI
ASE(Sybase)
Order TrackingDatabase
User ProfileDatabase
EcMsAcRegUserSrvr
EcMsAcOrderSrvr
EcMsAcOrderGUI
EcMsAcSMCRegUserGUI
SMCSybaseReplicationServer
ASE(Sybase)
UserProfileDatabase
EcMsAcRegUserSrvr
SybaseReplicationServer
(At SMC)
Subsystems and CSCIs: MSS (Cont.)AMS Architecture and Interfaces
291625-EMD-002, Rev. 02
O
MSS
MCI - Accountability
EcMsAcDAACRegUserGUI
ASE(Sybase)
Order TrackingDatabase
User ProfileDatabase
EcMsAcRegUserSrvr
EcMsAcOrderSrvr
EcMsAcOrderGUI
EcMsAcSMCRegUserGUI
SMCSybaseReplicationServer
ASE(Sybase)
UserProfileDatabase
EcMsAcRegUserSrvr
SybaseReplicationServer
(At SMC)
Subsystems and CSCIs: MSS (Cont.)AMS Architecture and Interfaces
Request user registration/User
Profile update UserProfiledata
292625-EMD-002, Rev. 02
O
MSS
MCI - Accountability
EcMsAcDAACRegUserGUI
ASE(Sybase)
Order TrackingDatabase
User ProfileDatabase
EcMsAcRegUserSrvr
EcMsAcOrderSrvr
EcMsAcOrderGUI
EcMsAcSMCRegUserGUI
SMCSybaseReplicationServer
ASE(Sybase)
UserProfileDatabase
EcMsAcRegUserSrvr
SybaseReplicationServer
(At SMC)
Subsystems and CSCIs: MSS (Cont.)AMS Architecture and Interfaces
UserProfile request informa-tion
Insert/UpdateUser Profile information
293625-EMD-002, Rev. 02
O
MSS
MCI - Accountability
EcMsAcDAACRegUserGUI
ASE(Sybase)
Order TrackingDatabase
User ProfileDatabase
EcMsAcRegUserSrvr
EcMsAcOrderSrvr
EcMsAcOrderGUI
EcMsAcSMCRegUserGUI
SMCSybaseReplicationServer
ASE(Sybase)
UserProfileDatabase
EcMsAcRegUserSrvr
SybaseReplicationServer
(At SMC)
Subsystems and CSCIs: MSS (Cont.)AMS Architecture and Interfaces
Create/update/view User Profiles(DAAC operator logged in remotely)
UserProfile informa-tion
294625-EMD-002, Rev. 02
O
MSS
MCI - Accountability
EcMsAcDAACRegUserGUI
ASE(Sybase)
Order TrackingDatabase
User ProfileDatabase
EcMsAcRegUserSrvr
EcMsAcOrderSrvr
EcMsAcOrderGUI
EcMsAcSMCRegUserGUI
SMCSybaseReplicationServer
ASE(Sybase)
UserProfileDatabase
EcMsAcRegUserSrvr
SybaseReplicationServer
(At SMC)
Subsystems and CSCIs: MSS (Cont.)AMS Architecture and Interfaces
User Profile database replication
295625-EMD-002, Rev. 02
O
MSS
MCI - Accountability
EcMsAcDAACRegUserGUI
ASE(Sybase)
Order TrackingDatabase
User ProfileDatabase
EcMsAcRegUserSrvr
EcMsAcOrderSrvr
EcMsAcOrderGUI
EcMsAcSMCRegUserGUI
SMCSybaseReplicationServer
ASE(Sybase)
UserProfileDatabase
EcMsAcRegUserSrvr
SybaseReplicationServer
(At SMC)
Subsystems and CSCIs: MSS (Cont.)AMS Architecture and Interfaces
User Profile information
User Profile request
296625-EMD-002, Rev. 02
O
MSS
MCI - Accountability
EcMsAcDAACRegUserGUI
ASE(Sybase)
Order TrackingDatabase
User ProfileDatabase
EcMsAcRegUserSrvr
EcMsAcOrderSrvr
EcMsAcOrderGUI
EcMsAcSMCRegUserGUI
SMCSybaseReplicationServer
ASE(Sybase)
UserProfileDatabase
EcMsAcRegUserSrvr
SybaseReplicationServer
(At SMC)
Subsystems and CSCIs: MSS (Cont.)AMS Architecture and Interfaces
User Profiles
Request User Profiles (read only)
297625-EMD-002, Rev. 02
O
MSS
MCI - Accountability
EcMsAcDAACRegUserGUI
ASE(Sybase)
Order TrackingDatabase
User ProfileDatabase
EcMsAcRegUserSrvr
EcMsAcOrderSrvr
EcMsAcOrderGUI
EcMsAcSMCRegUserGUI
SMCSybaseReplicationServer
ASE(Sybase)
UserProfileDatabase
EcMsAcRegUserSrvr
SybaseReplicationServer
(At SMC)
Subsystems and CSCIs: MSS (Cont.)AMS Architecture and Interfaces
Order/request tracking information
Request order; request order
status
Insert/update order; request
order status
298625-EMD-002, Rev. 02
O
MSS
MCI - Accountability
EcMsAcDAACRegUserGUI
ASE(Sybase)
Order TrackingDatabase
User ProfileDatabase
EcMsAcRegUserSrvr
EcMsAcOrderSrvr
EcMsAcOrderGUI
EcMsAcSMCRegUserGUI
SMCSybaseReplicationServer
ASE(Sybase)
UserProfileDatabase
EcMsAcRegUserSrvr
SybaseReplicationServer
(At SMC)
Subsystems and CSCIs: MSS (Cont.)AMS Architecture and Interfaces
Update order status
Update order status
299625-EMD-002, Rev. 02
O
MSS
MCI - Accountability
EcMsAcDAACRegUserGUI
ASE(Sybase)
Order TrackingDatabase
User ProfileDatabase
EcMsAcRegUserSrvr
EcMsAcOrderSrvr
EcMsAcOrderGUI
EcMsAcSMCRegUserGUI
SMCSybaseReplicationServer
ASE(Sybase)
UserProfileDatabase
EcMsAcRegUserSrvr
SybaseReplicationServer
(At SMC)
Subsystems and CSCIs: MSS (Cont.)AMS Architecture and Interfaces
Order tracking
information;
request communications
support
Request order status (MTM Gwy); configuration parameters, media
options (Registry)
300625-EMD-002, Rev. 02
O
MSS
MCI - Accountability
EcMsAcDAACRegUserGUI
ASE(Sybase)
Order TrackingDatabase
User ProfileDatabase
EcMsAcRegUserSrvr
EcMsAcOrderSrvr
EcMsAcOrderGUI
EcMsAcSMCRegUserGUI
SMCSybaseReplicationServer
ASE(Sybase)
UserProfileDatabase
EcMsAcRegUserSrvr
SybaseReplicationServer
(At SMC)
Subsystems and CSCIs: MSS (Cont.)AMS Architecture and Interfaces
Create order(ODFRM)
301625-EMD-002, Rev. 02
O
MSS
MCI - Accountability
EcMsAcDAACRegUserGUI
ASE(Sybase)
Order TrackingDatabase
User ProfileDatabase
EcMsAcRegUserSrvr
EcMsAcOrderSrvr
EcMsAcOrderGUI
EcMsAcSMCRegUserGUI
SMCSybaseReplicationServer
ASE(Sybase)
UserProfileDatabase
EcMsAcRegUserSrvr
SybaseReplicationServer
(At SMC)
Subsystems and CSCIs: MSS (Cont.)AMS Architecture and Interfaces
Order/request status/ information
Update order; request order
status/information
302625-EMD-002, Rev. 02
O
MSS
MCI - Accountability
EcMsAcDAACRegUserGUI
ASE(Sybase)
Order TrackingDatabase
User ProfileDatabase
EcMsAcRegUserSrvr
EcMsAcOrderSrvr
EcMsAcOrderGUI
EcMsAcSMCRegUserGUI
SMCSybaseReplicationServer
ASE(Sybase)
UserProfileDatabase
EcMsAcRegUserSrvr
SybaseReplicationServer
(At SMC)
Subsystems and CSCIs: MSS (Cont.)AMS Architecture and Interfaces
Management information (e.g., User Profile data, order information)
Request management services
303625-EMD-002, Rev. 02
Subsystems and CSCIs: MSS (Cont.)
• Management Logistics CSCI (MLCI)– Implements Configuration Management services– Inventory/Logistics/Maintenance (ILM) Manager component
- Customized COTS software: a Remedy application- Tracks and maintains key data on ECS contract-purchased
equipment, hardware, COTS software, COTS documentation (hardware and software), spares and consumable items, and Government Furnished Equipment (GFE)
- Stores and maintains detailed maintenance data on hardware to the component level, including corrective maintenance
304625-EMD-002, Rev. 02
Subsystems and CSCIs: MSS (Cont.)
• Management Logistics CSCI (MLCI) (Cont.)– Software Change Manager component
- Consists of COTS and custom software- ClearCase (with some customization)- Supporting UNIX scripts
- Helps organize and partition software, control software changes and versions, and assemble sets of software for release
305625-EMD-002, Rev. 02
Subsystems and CSCIs: MSS (Cont.)
• Management Logistics CSCI (MLCI) (Cont.)– Software License Manager component
- COTS software- FLEXlm server daemon (lmgrd) with its associated command
line utilities- FLEXlm-enabled COTS Application - client software within
vendor products communicates with FLEXlm’s license server and vendor daemons to request licenses for product users to run
- Manages network licensing activities associated with using COTS products; maintains information about license provisions, metersuse of installed licenses, and reports on licensing events and statistics
306625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
Communications (CSS)and Internetworking (ISS)
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
System MgmtSubsystem
MSS
Subsystems and CSCIs: MSS (Cont.)
ManagementLogistics CSCI
(MLCI)
Inventory/Logistics/Maintenance Mgr
Remedy ARS (COTS)
SoftwareChange MgrClearCase (COTS)
SoftwareLicense Mgr
lmgrd (FLEXlm server daemon) (COTS)FLEXlm-enabled COTS Application
ManagementCSCI (MCI)
Network andEnterprise
ManagementWhatsUp GoldWhazzup???
Securityanlpasswd (COTS)
TCP Wrappers (COTS)Tripwire (COTS)Crack (COTS)
AccountabilityEcMsAcRegUserSrvr
EcMsAcSMCRegUserGUIEcMsAcDAACRegUserGUI
EcMsAcOrderSrvrEcMsAcOrderGUI
Sybase Replication ServerSybase ASE
Trouble Ticketaruser GUI (COTS)
aradmin GUI (COTS)notifier GUI (COTS)
notifier daemon (COTS)Remedy ARS (COTS)
armail daemon (COTS)Sendmail Advanced Message
Server (SAMS) (COTS)Sybase ASE (COTS)
NetworkBackup/RestoreNetworker Server (COTS)Networker Client (COTS)
ASTER E-mailHeader Handler
UNIX sendmail daemon (COTS)EcMsAsAddHeader.pl
EcMsAsRemoveHeader.ple-mail utility (COTS)
307625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
Subsystems and CSCIs: CSS
CSS/DistributedCommunications Software
308625-EMD-002, Rev. 02
Subsystems and CSCIs: CSS (Cont.)
• Communications Subsystem (CSS)– Provides for interconnection of users and
service providers and transfer of information within ECS and between ECS and other EOSDIS components, including a machine-to-machine gateway for SIPS access to ECS data
– Supports and interacts with the System Management Subsystem (MSS), ECS Mission Operations Segment (EMOS), and all other subsystems
– Uses several COTS tools: RogueWave class libraries, Builder Xcessory (GUI Builder tool), Sybase ASE Server (for Subscription Server insert, search, and update), UNIX Network Services
CSS/DistributedCommunications Software
309625-EMD-002, Rev. 02
Subsystems and CSCIs: CSS (Cont.)
• Distributed Computing Configuration Item (DCCI)– Subscription Server (SBSRV) and GUI components
- Detects previously defined events- Performs specified actions for clients that have previously
subscribed to those events (e.g., science granule insertion, metadata update, science granule deletion)
- Being replaced by Spatial Subscription Server (NBSRV)– ASTER DAR Gateway Server (hosted at the LP DAAC)
- Provides interoperability between ASTER DAR Client GUI tool and the DAR API which interfaces to the ASTER servers
– ASTER E-Mail Parser Gateway Server- Support for automated delivery of ASTER Expedited Data Sets
(EDS) from ECS to ASTER Ground Data System (GDS)– Message-Oriented Java Object (MOJO) Gateway Server
- Gateway for access by the ASTER DAR Tool to all ECS Services; directs DARs to GDS via ASTER DAR Gateway
310625-EMD-002, Rev. 02
Subsystems and CSCIs: CSS (Cont.)
• Distributed Computing Configuration Item (DCCI) (Cont.)– Configuration Registry Server
- Provides a single interface to retrieve configuration attribute-value pairs for ECS servers from the Configuration Registry Database, via Sybase Server
- Provides an interface to retrieve distribution options for ECS Servers from the Configuration Registry Database
– Machine-to-Machine Gateway Server- Provides an automated search and order capability to allow the
Science Investigator-Led Processing Systems (SIPS) to reprocess data externally from the ECS
– CCS Middleware Support Group- CCS Name Server enables clients to locate and communicate with
the various ECS servers
311625-EMD-002, Rev. 02
Subsystems and CSCIs: CSS (Cont.)
• Distributed Computing Configuration Item (DCCI) (Cont.)– File Transfer Protocol (FTP) component (Remote File Access
Group)- Standard application for file transfers (standard UNIX utility with
CSS wrapper classes applied)– FTP Notification component (Remote File Access Group)
- Provides notification of successful FTP pulls from a pull area– Bulk Data Server (BDS) component (Remote File Access
Group)- COTS product (BDSpro) that permits fast file transfer over high-
speed networks (such as Gigabit Ethernet)– Network File System (NFS) component (Remote File Access
Group)- COTS product that provides a file-sharing system among
computers
312625-EMD-002, Rev. 02
Subsystems and CSCIs: CSS (Cont.)
• Distributed Computing Configuration Item (DCCI) (Cont.)– Filecopy component (Remote File Access Group)
- A simple utility to copy large files from a specified source to a specifed destination, with compression options
– Mail Support Group- Custom and COTS software that provides electronic mail, with an
interactive interface and an object-oriented application program interface
– Virtual Terminal component- COTS software (Telnet) that provides ECS Operations personnel
with the capability for remote logon from one ECS machine to another
313625-EMD-002, Rev. 02
Subsystems and CSCIs: CSS (Cont.)
• Distributed Computing Configuration Item (DCCI) (Cont.)– Cryptographic Management Interface (CMI)
- CMI classes provide the requesting process with a server accountand a randomly generated password so the server can access security required services (i.e., Sybase ASE)
- Passwords (and optionally login names) are generated dynamicallybased on a psuedo-random number used as the seed for the password
– Domain Name Server (DNS)- COTS product that provides information about host names and
addresses on a network by querying and answering queries- Performs naming between hosts within the local administrative
domain and across domain boundaries- Note: The external DNS is located on the Firewall in ISS
314625-EMD-002, Rev. 02
Subsystems and CSCIs: CSS (Cont.)
• Distributed Computing Configuration Item (DCCI) (Cont.)– Infrastructure Libraries Group
- Provides a set of services to facilitate the implementation of client-server applications; includes …- Process Framework (PF)- Service Request Framework (SRF)- Message Passing- Universal References (URs)- Event Logging- Server Locator- Failure Recovery Framework- EcPo Connections- Time Service
315625-EMD-002, Rev. 02
ScienceSoftwareand Data
DataData Pool
(DPL)DATAPOOL
ElectronicProducts
Data ProcessingSubsystem
(DPS) Data Proc
0110001011001011001011010111010110001011
InventorySearch
ExternalData Sourcesand Providers
Archive
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Insertand
Retrieve
PUSH
Data ProcessingRequests
PlanningSubsystem
(PLS) Planning
abc
def
ghi
jkl
mno
PULLSystem Mgmt
SubsystemMSS
AcquireRequests
ProductOrders
Order ManagerSubsystem
(OMS)Hard MediaOrders
Meta-data
ProductRequests
EMOS
User Registration, OrderStatus, User Profile
Searchand
Access
Client (CLS)External/InternalUsers
Data MgmtSubsystem
(DMS)
External/InternalUsers
Searchand
Download
ProductDistributionSystem (PDS)
Spatial Subscrip-tion Server
(SSS)Hard MediaProducts
AcquireData/Services
Data
DataIngest
Subsystem(INS) Ingest
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotification
CsFtFTPNotify
Bulk DataServer
BDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExecVirtual
TerminalVT
CryptographicManagement
InterfaceCMI
InfrastructureLibrariesPF, SRF, URs,
Message Passing,Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.EcCsRegistryServer
Machine-to-Machine Gwy.
EcCsMtMGateway
Subsystems and CSCIs: CSS (Cont.)
316625-EMD-002, Rev. 02
ECS Operational FunctioningIN
SERT
ION
RETRIEVAL
PRODUCTION
SubscribeSearch& Order
Store
ExternalData Provider User
Deliver
Generate Plan
Ingest
CAUTIONSome of the following pages display numbers that may
indicate the sequence in which graphic elements appear in the dynamic presentations. These numbers do not
reliably indicate the sequence of events in ECS operation.For more complete event sequence information, consult
Document 313-EMD-001.
318625-EMD-002, Rev. 02
ASTER Scenario
ASTER Goals- ASTER DAR Tool Usage- On-Demand Processing and Chaining- SCF QA Metadata Update Workaround- Simplified ASTER Expedited Data Support- Data Tape Ingest
ASTER PreconditionsASTER ESDTs Inserted into ECS-AST_ANC, AST_EXP, AST_L1A, AST_L1BT,AST_09T, AST_04, AST_05, AST_08, GDAS0ZFH
ASTER PGEs passed SSI&T and installed- ACT, ETS, BTS
Ancillary data inserted into Data Server
319625-EMD-002, Rev. 02
ASTER Scenario: DAR SupportIN
SERT
ION
RETRIEVAL
PRODUCTION
SubscribeSearch& Order
Store
ExternalData Provider User
Deliver
Generate Plan
Ingest
DAR SubmissionData SubscriptionOn-Demand Request
320625-EMD-002, Rev. 02
RETRIEVAL
SubscribeRegister
Subscription
Deliver
ASTER Scientist decidesto request ASTER datarequiring a Data AcquisitionRequest
Search& Order
OrderProducts
Store
DAR Support
321625-EMD-002, Rev. 02
ASTER: Client Request Process
ASTER Scientist determines an area of interest. The scientistdecides to request an ASTER data take over that area, using theASTER DAR Tool.
1. Request DataAcquisition
2. SubmitDAR
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
CommunicationsSubsystemMJGWY
Data MgmtSubsystem
DDICT
IngestSubsystem Ingest
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
Systems MgmtSubsystem
(MSS)
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
322625-EMD-002, Rev. 02
System MgmtSubsystem
(MSS)
ManagementCSCI (MCI)
Network andEnterprise
ManagementWhatsUp GoldWhazzup???
Securityanlpasswd (COTS)
TCP Wrappers (COTS)Tripwire (COTS)Crack (COTS)
AccountabilityEcMsAcRegUserSrvr
EcMsAcSMCRegUserGUIEcMsAcDAACRegUserGUI
EcMsAcOrderSrvrEcMsAcOrderGUI
Sybase Replication ServerSybase ASE
Trouble Ticketaruser GUI (COTS)
aradmin GUI (COTS)notifier GUI (COTS)
notifier daemon (COTS)Remedy ARS (COTS)
armail daemon (COTS)Sendmail Advanced Message
Server (SAMS) (COTS)Sybase ASE (COTS)
NetworkBackup/Restore
Networker Server (COTS)Networker Client (COTS)
ASTER E-mailHeader Handler
UNIX sendmail daemon (COTS)EcMsAsAddHeader.pl
EcMsAsRemoveHeader.ple-mail utility (COTS)
ManagementLogistics CSCI
(MLCI)
Inventory/Logistics/Maintenance Mgr
Remedy ARS (COTS)
SoftwareChange MgrClearCase (COTS)
SoftwareLicense Mgr
lmgrd (FLEXlm server daemon) (COTS)FLEXlm-enabled COTS Application
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotificationCsFtFTPNotify
Bulk DataServerBDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExec
VirtualTerminal
VT
CryptographicManagement
InterfaceCMI
InfrastructureLibraries
PF, SRF, URs,Message Passing,
Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.EcCsRegistryServer
Machine-to-Machine Gwy.EcCsMtMGateway
ASTER: CSCI/Component Rolein Client Request
Client (CLS)External/InternalUsers
On-Demand FormRequest Manager
(ODFRM) CSCI
HTML UserInterface
ODFRM
CGIPrograms
EcClOdRequest
Desktop(DESKT) CSCI
User ProfileGateway
EcClDtUserProfileGateway
Workbench(WKBCH) CSCI
HDF ProductViewing Tool
EOSView
ASTER DARTool
EcClWbJdt
Provides GUI to specify parameters for data take; submits request to ASTER GDS through CSS
5
Server; provides UserProfile Information andauthenticates privileges
2
Holds User Profile
information,
including special
privileges1
Submits user information
to CCS Middleware;
provides access to
ECS services from the
Java front end of ASTER
DAR Tool3
Provides session ID
to Mojo Gateway
4
323625-EMD-002, Rev. 02
RETRIEVAL
SubscribeRegister
Subscription
Deliver
ECS submits DAR to ASTER GroundData System (GDS) in Japanreceiving a DARid in return
Store
DAR Support (Cont.)
GDS
Search& Order
OrderProducts
324625-EMD-002, Rev. 02
ASTER: Request Data Take Process
ASTER DAR Gateway submits a request for a data take over the areaof interest. GDS responds with a DARid.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
CommunicationsSubsystemDAR GW
IngestSubsystem Ingest
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
GDS1. Request Data Take
Data MgmtSubsystem
DDICT
Systems MgmtSubsystem
(MSS)
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
325625-EMD-002, Rev. 02
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotification
CsFtFTPNotify
Bulk DataServer
BDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExecVirtual
TerminalVT
CryptographicManagement
InterfaceCMI
InfrastructureLibrariesPF, SRF, URs,
Message Passing,Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.EcCsRegistryServer
Machine-to-Machine Gwy.
EcCsMtMGateway
ASTER: CSCI/Component Rolein Data Take Request
Submits request to ASTER GDS; receives DARid from GDS
1
Provides access byASTER DAR Tool toECS Services
2
326625-EMD-002, Rev. 02
RETRIEVAL
Deliver
Subscription is submitted onbehalf of user for notificationon receipt of AST_L1BT (TIR)data
Store
DAR Support (Cont.)
Search& Order
OrderProducts
Subscribe
RegisterSubscription
MoJoGateway
327625-EMD-002, Rev. 02
ASTER: Submit Subscription Process
MoJo Gateway submits subscription for notification on the occurrence ofAST_L1BT:Insert event, qualified with the DARid.
1. Subscribe (L1BT:Insert, e-mailnotification)
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
CommunicationsSubsystem
SBSRV
IngestSubsystem Ingest
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
Systems MgmtSubsystem
(MSS)
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
328625-EMD-002, Rev. 02
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotification
CsFtFTPNotify
Bulk DataServer
BDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExecVirtual
TerminalVT
CryptographicManagement
InterfaceCMI
InfrastructureLibrariesPF, SRF, URs,
Message Passing,Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.EcCsRegistryServer
Machine-to-Machine Gwy.
EcCsMtMGateway
ASTER: CSCI/Component Rolein Subscription Submission
Registers subscription and assignsit to persistent storage
2
Submits subscription(Subscribe toL1BT:Insert)
1
329625-EMD-002, Rev. 02
RETRIEVAL
SubscribeRegister
Subscription
Deliver
ASTER Scientist decidesto request production ofa higher level ASTERproduct from the data thatare to be collected
Search& Order
OrderProducts
Store
DAR Support (Cont.)
330625-EMD-002, Rev. 02
ASTER: On-Demand Data Processing Request Process
ASTER Scientist wants AST_08 (L2 Surface Temperature product) based onthe AST_L1BT (TIR - Thermal InfraRed - product) resulting from GDSinitial processing of the data collected for the DAR, and uses the EOS DataGateway (EDG) Web Client to submit the request.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
CommunicationsSubsystem
SBSRV
IngestSubsystem Ingest
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
Systems MgmtSubsystem
(MSS)
PlanningSubsystem Planning
abc
def
ghi
jkl
mno 2. Submit selections (e.g., input,output, distribution)
3. Create order-trackingelements in the MSSorder-tracking database
EDG
UserAuthentication
4. Submit on-demandrequest
1. Request higher-level data product
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
331625-EMD-002, Rev. 02
Production Planning(PLANG) CSCI
DataStore
Sybase Database (COTS)
Resource PlanningWorkbench
EcPlRpReEcPlRpSiEcPlRpTl
ProductionRequest Editor
EcPlPrEditor_IF
ProductionPlanning Workbench
EcPlWbEcPlTl
EcPlDetermineChain.pl
On-DemandProduction RequestManager (ODPRM)
EcPlOdMgr
SubscriptionManagerEcPlSubMgr
ProductionStrategies GUI
EcPlProdStrat
ProductionRequest Generator
EcPlPRGenerator
PlanningSubsystem
(PLS) Planning
abcdefghijklmno
System MgmtSubsystem
(MSS)
ManagementCSCI (MCI)
Network andEnterprise
ManagementWhatsUp GoldWhazzup???
Securityanlpasswd (COTS)
TCP Wrappers (COTS)Tripwire (COTS)Crack (COTS)
AccountabilityEcMsAcRegUserSrvr
EcMsAcSMCRegUserGUIEcMsAcDAACRegUserGUI
EcMsAcOrderSrvrEcMsAcOrderGUI
Sybase Replication ServerSybase ASE
Trouble Ticketaruser GUI (COTS)
aradmin GUI (COTS)notifier GUI (COTS)
notifier daemon (COTS)Remedy ARS (COTS)
armail daemon (COTS)Sendmail Advanced Message
Server (SAMS) (COTS)Sybase ASE (COTS)
NetworkBackup/Restore
Networker Server (COTS)Networker Client (COTS)
ASTER E-mailHeader Handler
UNIX sendmail daemon (COTS)EcMsAsAddHeader.pl
EcMsAsRemoveHeader.ple-mail utility (COTS)
ManagementLogistics CSCI
(MLCI)
Inventory/Logistics/Maintenance Mgr
Remedy ARS (COTS)
SoftwareChange MgrClearCase (COTS)
SoftwareLicense Mgr
lmgrd (FLEXlm server daemon) (COTS)FLEXlm-enabled COTS Application
Client (CLS)External/InternalUsers
On-Demand FormRequest Manager
(ODFRM) CSCI
HTML UserInterface
ODFRM
CGIPrograms
EcClOdRequest
Desktop(DESKT) CSCI
User ProfileGateway
EcClDtUserProfileGateway
Workbench(WKBCH) CSCI
HDF ProductViewing Tool
EOSView
ASTER DARTool
EcClWbJdt
ASTER: CSCI/Component Role in On-Demand Request
Holds User Profileinformation, including specialprivileges
1
Data MgmtSubsystem
(DMS)
Data Dictionary(DDICT) CSCI
Data DictionaryMaintenance Tool
EcDmMaintTool
Data DictionaryASE Server
Sybase Server (COTS)
Data DictionaryServer
EcDmDictService
Version Zero Gateway(V0 GTWAY) CSCI
V0 to ECSGateway Server
EcDmV0ToECSGateway
Records order-tracking elements in the Order-Tracking Database
4
Provides User Profile
information;
authenticates user
privileges
2
Receives user selections for AST_08 from EDG; submits request to MSS to create astanding order for FTPpush of AST_08; passes theon-demand order to PLS with OrderID and RequestID
3
ODPRM receives standing order (FTPpush of AST_08) and registers subscription for notification upon arrival of L1B product
5
332625-EMD-002, Rev. 02
ASTER Scenario: Chaining and On-Demand Production
INSE
RTIO
NRETRIEVAL
PRODUCTION
SubscribeSearch& Order
Store
ExternalData Provider User
Deliver
Generate Plan
Ingest
Data InsertionData NotificationOn-demand ProductionStanding Order DeliveryQA Update
333625-EMD-002, Rev. 02
INSE
RTIO
N
Subscribe
Notification
Ingest
Media
StoreArchive
Data
CatalogueData
Some time later, after receivingSony DTF-2 tape from GDS,operator mounts tape and beginsingest of data
Chaining and On-Demand Production (Cont.)
SonyDTF-2Tape
334625-EMD-002, Rev. 02
ASTER: DTF-2 Tape Ingest Process
After receiving Sony DTF-2 tape in a shipment, DAAC Operatormounts tape and begins ingest activities.Tape contains AST_L1BT (L1B TIR) data.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
CommunicationsSubsystem
SBSRV
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
IngestSubsystem Ingest1. Mount tape
4. Get mcf (AST_L1BT)
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
2. Read PDR file3. Read AST_L1BT
files
SonyDTF-2Tape
335625-EMD-002, Rev. 02
IngestSubsystem
(INS) Ingest
Ingest(INGST) CSCI
Polling IngestClient Interface
ProcessEcInPolling
MediaIngest Interface
ProcessEcInGUI
EcInTapeReaderGUI
IngestRequest Manager
ProcessEcInReqMgr
IngestGranule Server
ProcessEcInGran
IngestDatabase
Sybase Server (COTS)
Cross-ModeIngest Interface
EcInEmailGWServer
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Rolein Ingest DTF-2 Tape Operations
Packages Ingest Request into Granules
3
OperatorInterface; reads delivery record file and data files from DTF-2 Tape
1
Staging Disk Server
receives request from
Ingest to create staging
disk and reserves space
6Provides metadata configuration file (mcf)
5
Detects PDR; creates ingest request
2
Requests metadata configuration file (mcf) from SDSRV; requests staging disk4
Request ManagerEcDsStRequestManagerServer
336625-EMD-002, Rev. 02
INSE
RTIO
N
Subscribe
Notification
StoreArchive
Data
CatalogueData
Ingested AST_L1BTdata are archived
Chaining and On-Demand Production (Cont.)
Ingest
Media
337625-EMD-002, Rev. 02
ASTER: Ingest Archive Insertion Process
Archive AST_L1BT (L1B TIR) data granules.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
CommunicationsSubsystem
SBSRV
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
IngestSubsystem Ingest
1. Insert (AST_L1BT)
2. Insert (AST_L1BTfiles)
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
338625-EMD-002, Rev. 02
IngestSubsystem
(INS) Ingest
Ingest(INGST) CSCI
Polling IngestClient Interface
ProcessEcInPolling
MediaIngest Interface
ProcessEcInGUI
EcInTapeReaderGUI
IngestRequest Manager
ProcessEcInReqMgr
IngestGranule Server
ProcessEcInGran
IngestDatabase
Sybase Server (COTS)
Cross-ModeIngest Interface
EcInEmailGWServer
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Rolein Ingest Archive Insertion
Builds metadata
file; requests
SDSRV to
validate
metadata;
requests
SDSRV to insert
received files
1
Validates metadata; requests archiving of granule files
2
Request ManagerEcDsStRequestManagerServer
Reads inserted files from Ingest staging disks
3
339625-EMD-002, Rev. 02
INSE
RTIO
N
Subscribe
Notification
StoreArchive
Data
CatalogueData
Update catalogue with referenceto new L1BT data
Ingest
Media
Chaining and On-Demand Production (Cont.)
340625-EMD-002, Rev. 02
ASTER: Inventory (Metadata) Update Process
Add metadata for AST_L1BT (L1B TIR) data granulesto the Sybase/SQS database.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
CommunicationsSubsystem
SBSRV
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
IngestSubsystem Ingest
1. Add AST_L1BT metadata
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
341625-EMD-002, Rev. 02
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager ProcessEcDsStCacheManagerServer
EcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Rolein Inventory (Metadata) Update
Adds validated metadata to the inventory
2
Sends validated metadata to database 1
342625-EMD-002, Rev. 02
PRODUCTION
Subscribe
NotificationRegister
Subscription
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
ProcessOn
Demand
Insert terminates with an insert event notification to Subscribe. Subscribee-mails ASTER Scientist notice of a completed AST_L1BT granule insert, and also notifies Planning
Chaining and On-Demand Production (Cont.)
343625-EMD-002, Rev. 02
ASTER: Event Notification Process
Notify all AST_L1BT:Insert event subscribers whoseDARid numbers are matched with the ingested granules.
CommunicationsSubsystem
SBSRV
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
IngestSubsystem Ingest
1. Trigger(AST_L1BT:Insert,XARids)
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
2. E-mail notification3. Notify (AST_L1BT:Insert,
direct)
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
344625-EMD-002, Rev. 02
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotificationCsFtFTPNotify
Bulk DataServerBDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExec
VirtualTerminal
VT
CryptographicManagement
InterfaceCMI
InfrastructureLibraries
PF, SRF, URs,Message Passing,
Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.EcCsRegistryServer
Machine-to-Machine Gwy.EcCsMtMGateway
Production Planning(PLANG) CSCI
DataStore
Sybase Database (COTS)
Resource PlanningWorkbench
EcPlRpReEcPlRpSiEcPlRpTl
ProductionRequest Editor
EcPlPrEditor_IF
ProductionPlanning Workbench
EcPlWbEcPlTl
EcPlDetermineChain.pl
On-DemandProduction RequestManager (ODPRM)
EcPlOdMgr
SubscriptionManagerEcPlSubMgr
ProductionStrategies GUI
EcPlProdStrat
ProductionRequest Generator
EcPlPRGenerator
PlanningSubsystem
(PLS) Planning
abcdefghijklmno
ASTER: CSCI/Component Rolein Event Notification
Sendmail deliverse-mail message for user to mailhost
3
Receives notice of AST_L1BT:Insert4
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
Triggers AST_L1BT:Insert event; notifies the correct subscription server
1
Fires subscriptions; sends notifications of the AST_L1BT:Insert event
2
345625-EMD-002, Rev. 02
Chaining and On-Demand Production (Cont.)
PRODUCTION
ProcessOn
Demand
Subscribe
NotificationRegister
Subscription
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
Planning recognizes the need to run ACT PGE before ETS. Creates data processing requests (DPRs) for ACT and ETS, and registers subscriptions for input products that are not available in the archive.
346625-EMD-002, Rev. 02
ASTER: PGE Chaining
Input: 1 AST_L1BT
Requested Output: 1 AST_08
ETS (PGE)
1 AST_05
1 AST_ANC1 AST_09T
ACT (PGE)
1 GDAS0ZFH 24 AST_ANC1 AST_L1BT
1 AST_08
347625-EMD-002, Rev. 02
ASTER: Sequenced Production Request ProcessPlanning recognizes that, in order to run ETS algorithm on AST_L1BT (L1B TIR),ACT algorithm must be run first. Planning creates DPRs for ACT and ETS, with the AST_09T (L2 Surface Radiance TIR) output feeding the ETSalgorithm, submitting subscriptions for data not available in the archive.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
IngestSubsystem Ingest
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
CommunicationsSubsystem
SBSRV1. Submitsubscriptions
2. Submit searchfor inputgranules
3. Search database
4. Return results
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
348625-EMD-002, Rev. 02
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotificationCsFtFTPNotify
Bulk DataServerBDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExec
VirtualTerminal
VT
CryptographicManagement
InterfaceCMI
InfrastructureLibraries
PF, SRF, URs,Message Passing,
Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.EcCsRegistryServer
Machine-to-Machine Gwy.EcCsMtMGateway
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
Production Planning(PLANG) CSCI
DataStore
Sybase Database (COTS)
Resource PlanningWorkbench
EcPlRpReEcPlRpSiEcPlRpTl
ProductionRequest Editor
EcPlPrEditor_IF
ProductionPlanning Workbench
EcPlWbEcPlTl
EcPlDetermineChain.pl
On-DemandProduction RequestManager (ODPRM)
EcPlOdMgr
SubscriptionManagerEcPlSubMgr
ProductionStrategies GUI
EcPlProdStrat
ProductionRequest Generator
EcPlPRGenerator
PlanningSubsystem
(PLS) Planning
abcdefghijklmno
ASTER: CSCI/Component Rolein Sequenced Production Request
Stores subscriptionsin the Sybase Database
2
Stores DataProcessing Requests5 Breaks down
Query and
generates calls
to inventory
database;
packages and
returns results
Database; contains metadata for search and retrieval
3
4
ODPRM creates DPRs;submits subscriptions;searches SDSRV for input granules
1
625-EMD-000 Revised 01/10/2005
349625-EMD-002, Rev. 02
Chaining and On-Demand Production (Cont.)
PRODUCTION
Subscribe
NotificationRegister
Subscription
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
Planning releases the On-Demand jobs, including the DPR for ACT and a dependent one for ETS
ProcessOn
Demand
350625-EMD-002, Rev. 02
ASTER: Job Activation Process
Planning releases the On-Demand jobs; the release activates the ACT DPR forprocessing.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
IngestSubsystem Ingest
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
CommunicationsSubsystem
SBSRV
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
1. On-Demand jobsrelease
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
351625-EMD-002, Rev. 02
Data ProcessingSubsystem(DPS) Data Proc
0110001011001011001011010111010110001011
Algorithm Integration and Test Tools(AITTL) CSCI
SSAP GUIEcDpAtSSAPGui
SSIT ManagerEcDpAtMgr
Define PGEEcDpAtCheckODL
EcDpAtCreateODLTemplateEcDpAtRegisterPGE
EcDpAtOpDbGui
View/CompareData ToolsEcDpAtBinDiffGui
EcDpAtCheckHdfFileEcDpAtMgrXdiff
Check SoftwareTools
EcDpAtCheckPCFEcDpAtBadFuncGui
EcDpAtBadFuncEcDpAtPrologs
Insert/AcquireTools
EcDpAtINSTALLEcDpAtInsertStaticFile
EcDpAtInsertExeTarFileEcDpAtInsertTestFile
EcDpAtStageDAPEcDpAtGetMCF
SDP Toolkit(SDPTK) CSCI
(Separatesoftware libraries
for SSI&T)
Processing(PRONG) CSCI
ExecutionManagement
EcDpPrEMDpPrDM (library)
PGEManagement
EcDpPrRunPGEEcDpPrRusage
DeletionServer
EcDpPrDeletion
Job ManagementEcDpPrJobMgmt
EcDpPrJobMgmtClientEcDpPrViewJobStates
AutoSysCOTS Product
QAMonitor
EcDpPrQaMonitorGUI
DataStore
Sybase Database (COTS)
Ground EventProcessEcDpPrGE
Data PreprocessingEcDpPrAm1EdosEphAttDPREP_PGEEcDpPrAm1FddAttitudeDPREP_PGE
EcDpPrAm1FddEphemerisDPREP_PGEEcDpPrDumpAttitudeDPREP
EcDpPrDumpEphemerisDPREPEcDpPrPm1FddEphemerisDPREP_PGE
EcDpPrPm1AttitudeDPREP_PGEEcDpPrAuraEphemerisDPREP_PGE
EcDpPrAuraAttitudeDPREP_PGE
Production Planning(PLANG) CSCI
DataStore
Sybase Database (COTS)
Resource PlanningWorkbench
EcPlRpReEcPlRpSiEcPlRpTl
ProductionRequest Editor
EcPlPrEditor_IF
ProductionPlanning Workbench
EcPlWbEcPlTl
EcPlDetermineChain.pl
On-DemandProduction RequestManager (ODPRM)
EcPlOdMgr
SubscriptionManagerEcPlSubMgr
ProductionStrategies GUI
EcPlProdStrat
ProductionRequest Generator
EcPlPRGenerator
PlanningSubsystem
(PLS) Planning
abcdefghijklmno
ASTER: CSCI/Component Rolein Job Activation
Releases the DPR for ACTto AutoSys for jobprocessing 2
Adds UR for new granule (AST_L1BT)to data context; matches metadata against job input needs; notifies Job Management to release ACT
1
352625-EMD-002, Rev. 02
Chaining and On-Demand Production (Cont.)
PRODUCTION
Subscribe
NotificationRegister
Subscription
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
Job locates AST_L1BT, AST_ANC (ASTER ancillary data set), and GDAS0ZFH (NCEP ancillary) data required for ACT algorithm
ProcessOn
Demand
353625-EMD-002, Rev. 02
ASTER: Input Data Location Process
ACT locates required AST_L1BT (L1B TIR), AST_ANC (ASTER ancillary dataset), and GDAS0ZFH (NCEP ancillary) input data granules.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
IngestSubsystem Ingest
Client
CommunicationsSubsystem
SBSRV
1. Search (AST_L1BT, AST_ANC, &GDAS0ZFH)
2. Query Sybase
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
354625-EMD-002, Rev. 02
Data ProcessingSubsystem(DPS) Data Proc
0110001011001011001011010111010110001011
Algorithm Integration and Test Tools(AITTL) CSCI
SSAP GUIEcDpAtSSAPGui
SSIT ManagerEcDpAtMgr
Define PGEEcDpAtCheckODL
EcDpAtCreateODLTemplateEcDpAtRegisterPGE
EcDpAtOpDbGui
View/CompareData ToolsEcDpAtBinDiffGui
EcDpAtCheckHdfFileEcDpAtMgrXdiff
Check SoftwareTools
EcDpAtCheckPCFEcDpAtBadFuncGui
EcDpAtBadFuncEcDpAtPrologs
Insert/AcquireTools
EcDpAtINSTALLEcDpAtInsertStaticFile
EcDpAtInsertExeTarFileEcDpAtInsertTestFile
EcDpAtStageDAPEcDpAtGetMCF
SDP Toolkit(SDPTK) CSCI
(Separatesoftware libraries
for SSI&T)
Processing(PRONG) CSCI
ExecutionManagement
EcDpPrEMDpPrDM (library)
PGEManagement
EcDpPrRunPGEEcDpPrRusage
DeletionServer
EcDpPrDeletion
Job ManagementEcDpPrJobMgmt
EcDpPrJobMgmtClientEcDpPrViewJobStates
AutoSysCOTS Product
QAMonitor
EcDpPrQaMonitorGUI
DataStore
Sybase Database (COTS)
Ground EventProcessEcDpPrGE
Data PreprocessingEcDpPrAm1EdosEphAttDPREP_PGEEcDpPrAm1FddAttitudeDPREP_PGE
EcDpPrAm1FddEphemerisDPREP_PGEEcDpPrDumpAttitudeDPREP
EcDpPrDumpEphemerisDPREPEcDpPrPm1FddEphemerisDPREP_PGE
EcDpPrPm1AttitudeDPREP_PGEEcDpPrAuraEphemerisDPREP_PGE
EcDpPrAuraAttitudeDPREP_PGE
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Rolein Input Data Location
Releases and sequencesDPRs (Jobs)
1
Controls staging andprocessing activities 2
Database; contains metadata for search and retrieval
Retrieves metadata for requested input granules from the Sybase/SQS database
3
4
355625-EMD-002, Rev. 02
Chaining and On-Demand Production (Cont.)
PRODUCTION
Subscribe
NotificationRegister
Subscription
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
Retrieve AST_L1B, AST_ANC (ASTER ancillary data set), and GDAS0ZFH granules as input to ACT; PGE execution begins
ProcessOn
Demand
356625-EMD-002, Rev. 02
ASTER: Job Staging Process
ACT production job retrieves required AST_L1BT (L1B TIR), AST_ANC (ASTER ancillary data set), and GDAS0ZFH (NCEP ancillary) input datagranules.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
IngestSubsystem Ingest
Client
CommunicationsSubsystem
SBSRV
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
1. Acquire (AST_L1BT,AST_ANC, &GDAS0ZFH, ftpPush)
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
2. Distribution Requests (AST_L1BT, AST_ANC, & GDAS0ZFH files, ftpPush)
3. Copy (AST_L1BT, AST_ANC, &GDAS0ZFH files, ftpPush)
4. ftpPush (AST_L1BT,AST_ANC, &GDAS0ZFH files)
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
357625-EMD-002, Rev. 02
Data ProcessingSubsystem(DPS) Data Proc
0110001011001011001011010111010110001011
Algorithm Integration and Test Tools(AITTL) CSCI
SSAP GUIEcDpAtSSAPGui
SSIT ManagerEcDpAtMgr
Define PGEEcDpAtCheckODL
EcDpAtCreateODLTemplateEcDpAtRegisterPGE
EcDpAtOpDbGui
View/CompareData ToolsEcDpAtBinDiffGui
EcDpAtCheckHdfFileEcDpAtMgrXdiff
Check SoftwareTools
EcDpAtCheckPCFEcDpAtBadFuncGui
EcDpAtBadFuncEcDpAtPrologs
Insert/AcquireTools
EcDpAtINSTALLEcDpAtInsertStaticFile
EcDpAtInsertExeTarFileEcDpAtInsertTestFile
EcDpAtStageDAPEcDpAtGetMCF
SDP Toolkit(SDPTK) CSCI
(Separatesoftware libraries
for SSI&T)
Processing(PRONG) CSCI
ExecutionManagement
EcDpPrEMDpPrDM (library)
PGEManagement
EcDpPrRunPGEEcDpPrRusage
DeletionServer
EcDpPrDeletion
Job ManagementEcDpPrJobMgmt
EcDpPrJobMgmtClientEcDpPrViewJobStates
AutoSysCOTS Product
QAMonitor
EcDpPrQaMonitorGUI
DataStore
Sybase Database (COTS)
Ground EventProcessEcDpPrGE
Data PreprocessingEcDpPrAm1EdosEphAttDPREP_PGEEcDpPrAm1FddAttitudeDPREP_PGE
EcDpPrAm1FddEphemerisDPREP_PGEEcDpPrDumpAttitudeDPREP
EcDpPrDumpEphemerisDPREPEcDpPrPm1FddEphemerisDPREP_PGE
EcDpPrPm1AttitudeDPREP_PGEEcDpPrAuraEphemerisDPREP_PGE
EcDpPrAuraAttitudeDPREP_PGE
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Rolein Job Staging
Allocates space in response to Distribution Requests 6
Releases and
sequencesDPRs (Jobs)
1
Requests staging disks for granule files; requests granule file retrieval by STMGT
5
Stages granule files to read-only cache disks
7
Controls staging and processing activities; requests MCF; handles requests to the Archive for retrieving granules
2
Request ManagerEcDsStRequestManagerServer
Provides Resource
Manager for ftpPushes
8Database; contains metadata for search and retrieval
Provides Metadata Configuration File (MCF); retrieves metadata; requests metadata Staging Disks; creates and passes distribution requests and metadata files to DDIST
4
3
358625-EMD-002, Rev. 02
Chaining and On-Demand Production (Cont.)
PRODUCTION
Subscribe
NotificationRegister
Subscription
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
Archive newly created AST_09T (L2 Surface Radiance TIR) granule after completion of ACT PGE;
ProcessOn
Demand
update catalogue with reference to AST_09T
359625-EMD-002, Rev. 02
ASTER: PGE Execution and Output Insertion Process
ACT PGE is successfully executed and newly created AST_09T(L2 Surface Radiance TIR) granule is archived; inventory is updated.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
IngestSubsystem Ingest
Client
CommunicationsSubsystem
SBSRV
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
1. Insert(AST_09T)
2. Store (AST_09T files)
3. Add (AST_09T metadata)
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
360625-EMD-002, Rev. 02
Data ProcessingSubsystem(DPS) Data Proc
0110001011001011001011010111010110001011
Algorithm Integration and Test Tools(AITTL) CSCI
SSAP GUIEcDpAtSSAPGui
SSIT ManagerEcDpAtMgr
Define PGEEcDpAtCheckODL
EcDpAtCreateODLTemplateEcDpAtRegisterPGE
EcDpAtOpDbGui
View/CompareData ToolsEcDpAtBinDiffGui
EcDpAtCheckHdfFileEcDpAtMgrXdiff
Check SoftwareTools
EcDpAtCheckPCFEcDpAtBadFuncGui
EcDpAtBadFuncEcDpAtPrologs
Insert/AcquireTools
EcDpAtINSTALLEcDpAtInsertStaticFile
EcDpAtInsertExeTarFileEcDpAtInsertTestFile
EcDpAtStageDAPEcDpAtGetMCF
SDP Toolkit(SDPTK) CSCI
(Separatesoftware libraries
for SSI&T)
Processing(PRONG) CSCI
ExecutionManagement
EcDpPrEMDpPrDM (library)
PGEManagement
EcDpPrRunPGEEcDpPrRusage
DeletionServer
EcDpPrDeletion
Job ManagementEcDpPrJobMgmt
EcDpPrJobMgmtClientEcDpPrViewJobStates
AutoSysCOTS Product
QAMonitor
EcDpPrQaMonitorGUI
DataStore
Sybase Database (COTS)
Ground EventProcessEcDpPrGE
Data PreprocessingEcDpPrAm1EdosEphAttDPREP_PGEEcDpPrAm1FddAttitudeDPREP_PGE
EcDpPrAm1FddEphemerisDPREP_PGEEcDpPrDumpAttitudeDPREP
EcDpPrDumpEphemerisDPREPEcDpPrPm1FddEphemerisDPREP_PGE
EcDpPrPm1AttitudeDPREP_PGEEcDpPrAuraEphemerisDPREP_PGE
EcDpPrAuraAttitudeDPREP_PGE
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Role in PGE Execution and Output Insertion
Database; receives and stores metadata
Requests STMGT to insert output granule files; adds metadata to inventory
3
4
Reads files from DPS
disks and stores
files in Archive
5
Monitors execution of
PGE; places output files
in output directory
1
2
Handles requests to the Archive for inserting granules
Request ManagerEcDsStRequestManagerServer
361625-EMD-002, Rev. 02
Chaining and On-Demand Production (Cont.)
PRODUCTION
Subscribe
NotificationRegister
Subscription
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
ProcessOn
Demand
Insert terminates with an insert event notification to Subscribe,which in turn triggers initiation of ETS algorithm
362625-EMD-002, Rev. 02
ASTER: Notification and Subscription Triggering Process
Notify all AST_09T:Insert subscribers. This includes notification of thePlanning Subsystem, for chained processing.
IngestSubsystem Ingest
Client
1. TriggerAST_09T:Insertevent
2. Notify (AST_09T:Insert,direct)
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
3. Data ProcessingRequest (ETS)
CommunicationsSubsystem
SBSRV
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
363625-EMD-002, Rev. 02
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotification
CsFtFTPNotify
Bulk DataServer
BDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExec
VirtualTerminal
VT
CryptographicManagement
InterfaceCMI
InfrastructureLibrariesPF, SRF, URs,
Message Passing,Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.
EcCsRegistryServer
Machine-to-Machine Gwy.
EcCsMtMGateway
Data ProcessingSubsystem(DPS) Data Proc
0110001011001011001011010111010110001011
Algorithm Integration and Test Tools(AITTL) CSCI
SSAP GUIEcDpAtSSAPGui
SSIT ManagerEcDpAtMgr
Define PGEEcDpAtCheckODL
EcDpAtCreateODLTemplateEcDpAtRegisterPGE
EcDpAtOpDbGui
View/CompareData ToolsEcDpAtBinDiffGui
EcDpAtCheckHdfFileEcDpAtMgrXdiff
Check SoftwareTools
EcDpAtCheckPCFEcDpAtBadFuncGui
EcDpAtBadFuncEcDpAtPrologs
Insert/AcquireTools
EcDpAtINSTALLEcDpAtInsertStaticFile
EcDpAtInsertExeTarFileEcDpAtInsertTestFile
EcDpAtStageDAPEcDpAtGetMCF
SDP Toolkit(SDPTK) CSCI
(Separatesoftware libraries
for SSI&T)
Processing(PRONG) CSCI
ExecutionManagement
EcDpPrEMDpPrDM (library)
PGEManagement
EcDpPrRunPGEEcDpPrRusage
DeletionServer
EcDpPrDeletion
Job ManagementEcDpPrJobMgmt
EcDpPrJobMgmtClientEcDpPrViewJobStates
AutoSysCOTS Product
QAMonitor
EcDpPrQaMonitorGUI
DataStore
Sybase Database (COTS)
Ground EventProcessEcDpPrGE
Data PreprocessingEcDpPrAm1EdosEphAttDPREP_PGEEcDpPrAm1FddAttitudeDPREP_PGE
EcDpPrAm1FddEphemerisDPREP_PGEEcDpPrDumpAttitudeDPREP
EcDpPrDumpEphemerisDPREPEcDpPrPm1FddEphemerisDPREP_PGE
EcDpPrPm1AttitudeDPREP_PGEEcDpPrAuraEphemerisDPREP_PGE
EcDpPrAuraAttitudeDPREP_PGE
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
Production Planning(PLANG) CSCI
DataStore
Sybase Database (COTS)
Resource PlanningWorkbench
EcPlRpReEcPlRpSiEcPlRpTl
ProductionRequest Editor
EcPlPrEditor_IF
ProductionPlanning Workbench
EcPlWbEcPlTl
EcPlDetermineChain.pl
On-DemandProduction RequestManager (ODPRM)
EcPlOdMgr
SubscriptionManagerEcPlSubMgr
ProductionStrategies GUI
EcPlProdStrat
ProductionRequest Generator
EcPlPRGenerator
PlanningSubsystem
(PLS) Planning
abcdefghijklmno
ASTER: CSCI/Component Role in Notification/Subscription Triggering
Database; contains
metadata for search
and retrieval2
Triggers event; retrieves metadata for
requested input granules from the
Sybase/SQS database1
Receives eventnotice from SDSRV;queries database;notifies Planning
3
Begins ETS staging andprocessing activities 6
Adds UR for new granule (AST_09T)to data context; matches metadata against job input needs; notifies Job Management to release ETS
4
Releas
es ETS
job
5
364625-EMD-002, Rev. 02
Chaining and On-Demand Production (Cont.)
PRODUCTION
Subscribe
NotificationRegister
Subscription
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
Retrieve AST_ANC (ASTER ancillary data set) granule as input to ETS production job; PGE execution begins
ProcessOn
Demand
365625-EMD-002, Rev. 02
ASTER: Job (ETS) Staging Process
ETS production job retrieves required AST_ANC(ASTER ancillary data set) input data granule(Note: AST_09T is already available on DPS resources).
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
IngestSubsystem Ingest
Client
CommunicationsSubsystem
SBSRV
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
1. Acquire (AST_ANC,ftpPush)
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
2. Distribution Request (AST_ANC file, ftpPush)
3. Copy (AST_ANCfile, ftpPush)
4. ftpPush (AST_ANCfile)
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
366625-EMD-002, Rev. 02
Data ProcessingSubsystem(DPS) Data Proc
0110001011001011001011010111010110001011
Algorithm Integration and Test Tools(AITTL) CSCI
SSAP GUIEcDpAtSSAPGui
SSIT ManagerEcDpAtMgr
Define PGEEcDpAtCheckODL
EcDpAtCreateODLTemplateEcDpAtRegisterPGE
EcDpAtOpDbGui
View/CompareData ToolsEcDpAtBinDiffGui
EcDpAtCheckHdfFileEcDpAtMgrXdiff
Check SoftwareTools
EcDpAtCheckPCFEcDpAtBadFuncGui
EcDpAtBadFuncEcDpAtPrologs
Insert/AcquireTools
EcDpAtINSTALLEcDpAtInsertStaticFile
EcDpAtInsertExeTarFileEcDpAtInsertTestFile
EcDpAtStageDAPEcDpAtGetMCF
SDP Toolkit(SDPTK) CSCI
(Separatesoftware libraries
for SSI&T)
Processing(PRONG) CSCI
ExecutionManagement
EcDpPrEMDpPrDM (library)
PGEManagement
EcDpPrRunPGEEcDpPrRusage
DeletionServer
EcDpPrDeletion
Job ManagementEcDpPrJobMgmt
EcDpPrJobMgmtClientEcDpPrViewJobStates
AutoSysCOTS Product
QAMonitor
EcDpPrQaMonitorGUI
DataStore
Sybase Database (COTS)
Ground EventProcessEcDpPrGE
Data PreprocessingEcDpPrAm1EdosEphAttDPREP_PGEEcDpPrAm1FddAttitudeDPREP_PGE
EcDpPrAm1FddEphemerisDPREP_PGEEcDpPrDumpAttitudeDPREP
EcDpPrDumpEphemerisDPREPEcDpPrPm1FddEphemerisDPREP_PGE
EcDpPrPm1AttitudeDPREP_PGEEcDpPrAuraEphemerisDPREP_PGE
EcDpPrAuraAttitudeDPREP_PGE
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Rolein Job (ETS) Staging
Allocates space in response to Distribution Requests 6
Releases and
sequencesDPRs (Jobs)
1
Requests staging disks for granule files; requests granule file retrieval by STMGT
5
Stages granule files to read-only cache disks
7
Request ManagerEcDsStRequestManagerServer
Provides Resource
Manager for ftpPushes
8
Controls staging and processing activities; requests MCF; handles requests to the Archive for retrieving granules
2
Database; contains metadata for search and retrieval
Provides Metadata Configuration File (MCF); retrieves metadata; requests metadata Staging Disks; creates and passes distribution requests and metadata files to DDIST
4
3
367625-EMD-002, Rev. 02
Chaining and On-Demand Production (Cont.)
PRODUCTION
Subscribe
NotificationRegister
Subscription
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
Archive newly created AST_08 (L2 Surface Temperature) and AST_05 (L2 Surface Emissivity) granules after completion of ETS PGE;
ProcessOn
Demand
update catalogue with references to AST_08 and AST_05
368625-EMD-002, Rev. 02
ASTER: PGE (ETS) Execution and Output Insertion Process
ETS PGE is successfully executed and newly created AST_08(L2 Surface Temperature) and AST_05 (L2 Surface Emissivity)granules are archived; inventory is updated.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
IngestSubsystem Ingest
Client
CommunicationsSubsystem
SBSRV
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
1. Insert(AST_08& AST_05)
2. Store (AST_08 &AST_05 files)
3. Add (AST_08 & AST_05 metadata)
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
369625-EMD-002, Rev. 02
Data ProcessingSubsystem(DPS) Data Proc
0110001011001011001011010111010110001011
Algorithm Integration and Test Tools(AITTL) CSCI
SSAP GUIEcDpAtSSAPGui
SSIT ManagerEcDpAtMgr
Define PGEEcDpAtCheckODL
EcDpAtCreateODLTemplateEcDpAtRegisterPGE
EcDpAtOpDbGui
View/CompareData ToolsEcDpAtBinDiffGui
EcDpAtCheckHdfFileEcDpAtMgrXdiff
Check SoftwareTools
EcDpAtCheckPCFEcDpAtBadFuncGui
EcDpAtBadFuncEcDpAtPrologs
Insert/AcquireTools
EcDpAtINSTALLEcDpAtInsertStaticFile
EcDpAtInsertExeTarFileEcDpAtInsertTestFile
EcDpAtStageDAPEcDpAtGetMCF
SDP Toolkit(SDPTK) CSCI
(Separatesoftware libraries
for SSI&T)
Processing(PRONG) CSCI
ExecutionManagement
EcDpPrEMDpPrDM (library)
PGEManagement
EcDpPrRunPGEEcDpPrRusage
DeletionServer
EcDpPrDeletion
Job ManagementEcDpPrJobMgmt
EcDpPrJobMgmtClientEcDpPrViewJobStates
AutoSysCOTS Product
QAMonitor
EcDpPrQaMonitorGUI
DataStore
Sybase Database (COTS)
Ground EventProcessEcDpPrGE
Data PreprocessingEcDpPrAm1EdosEphAttDPREP_PGEEcDpPrAm1FddAttitudeDPREP_PGE
EcDpPrAm1FddEphemerisDPREP_PGEEcDpPrDumpAttitudeDPREP
EcDpPrDumpEphemerisDPREPEcDpPrPm1FddEphemerisDPREP_PGE
EcDpPrPm1AttitudeDPREP_PGEEcDpPrAuraEphemerisDPREP_PGE
EcDpPrAuraAttitudeDPREP_PGE
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Role in PGE Execution and Output Insertion
Reads files from DPS
disks and stores
files in Archive
5
Monitors execution of
PGE; places output
files in output directory
1
Database; receives and stores metadata
Requests STMGT to insert output granule files; adds metadata to inventory
3
4
2
Handles requests to the Archive for inserting granules
Request ManagerEcDsStRequestManagerServer
370625-EMD-002, Rev. 02
Chaining and On-Demand Production (Cont.)
Insert terminates with an insert event notification to Subscribe,
RETRIEVAL
Deliver
Electronic
Search& Order
OrderProducts
Subscribe
Notification StandingOrders
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
which triggerse-mail notification to the Science User that the AST_08 granule has been inserted; standing order processing for new AST_08 granule can begin
371625-EMD-002, Rev. 02
ASTER: Notification & Subscription Triggering (AST_08) Process
With insertion of the AST_08 (L2 Surface Temperature) granule,the ASTER Scientist is notified by e-mail; processing can beginfor the standing order
IngestSubsystem Ingest
Client
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
1. TriggerAST_08:Insertevent
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
CommunicationsSubsystem
SBSRV
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
2. Notify (AST_08:Insert,e-mail)
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
372625-EMD-002, Rev. 02
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotification
CsFtFTPNotify
Bulk DataServer
BDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExec
VirtualTerminal
VT
CryptographicManagement
InterfaceCMI
InfrastructureLibrariesPF, SRF, URs,
Message Passing,Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.
EcCsRegistryServer
Machine-to-Machine Gwy.
EcCsMtMGateway
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Role in Notification/Subscription Trigger
Sendmail deliverse-mail messageto mail host
3
Receives event notice from SDSRV; queries database; executes e-mail notification
2
Triggers event (AST_08:Insert)1
373625-EMD-002, Rev. 02
Chaining and On-Demand Production (Cont.)
Submit acquire request for new AST_08granule on behalf of scientist
RETRIEVAL
Deliver
Electronic
Search& Order
OrderProducts
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
Subscribe
Notification StandingOrders
374625-EMD-002, Rev. 02
ASTER: Standing Order, Acquire Submission Process
Subscription Server submits acquire request for AST_08 (L2 SurfaceTemperature) data, via ftpPush, on behalf of the Science User
IngestSubsystem Ingest
Client
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
CommunicationsSubsystem
SBSRV
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
1. Acquire (AST_08:ftpPush)
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
375625-EMD-002, Rev. 02
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotification
CsFtFTPNotify
Bulk DataServer
BDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExec
VirtualTerminal
VT
CryptographicManagement
InterfaceCMI
InfrastructureLibrariesPF, SRF, URs,
Message Passing,Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.
EcCsRegistryServer
Machine-to-Machine Gwy.
EcCsMtMGateway
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Role in Standing Order, Acq. Submission
Retrieves metadata
Database; stores metadata for search and retrieval
3
2
Connects to SDSRV; establishes data context; submits Acquire request for ftpPush of granule and request for e-mail notice of distribution
1
376625-EMD-002, Rev. 02
Chaining and On-Demand Production (Cont.)
Retrieve newly created AST_08 granule
RETRIEVAL
Deliver
Electronic
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
Subscribe
Notification StandingOrders
Search& Order
OrderProducts
377625-EMD-002, Rev. 02
ASTER: Retrieval of Data for Distribution Process
Retrieve newly created AST_08 (L2 Surface Temperature) granulefrom the Archive and update the order tracking information.
IngestSubsystem Ingest
Client
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
CommunicationsSubsystem
SBSRV
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
1. Distribution Request (AST_08:ftpPush)
2. Copy (AST_08, ftpPush)
Systems MgmtSubsystem
MSS3. Update Order
(AST_08) Data Mgmt
SubsystemDDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
378625-EMD-002, Rev. 02
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
System MgmtSubsystem
(MSS)
ManagementCSCI (MCI)
Network andEnterprise
ManagementWhatsUp GoldWhazzup???
Securityanlpasswd (COTS)
TCP Wrappers (COTS)Tripwire (COTS)Crack (COTS)
AccountabilityEcMsAcRegUserSrvr
EcMsAcSMCRegUserGUIEcMsAcDAACRegUserGUI
EcMsAcOrderSrvrEcMsAcOrderGUI
Sybase Replication ServerSybase ASE
Trouble Ticketaruser GUI (COTS)
aradmin GUI (COTS)notifier GUI (COTS)
notifier daemon (COTS)Remedy ARS (COTS)
armail daemon (COTS)Sendmail Advanced Message
Server (SAMS) (COTS)Sybase ASE (COTS)
NetworkBackup/Restore
Networker Server (COTS)Networker Client (COTS)
ASTER E-mailHeader Handler
UNIX sendmail daemon (COTS)EcMsAsAddHeader.pl
EcMsAsRemoveHeader.ple-mail utility (COTS)
ManagementLogistics CSCI
(MLCI)
Inventory/Logistics/Maintenance Mgr
Remedy ARS (COTS)
SoftwareChange MgrClearCase (COTS)
SoftwareLicense Mgr
lmgrd (FLEXlm server daemon) (COTS)FLEXlm-enabled COTS Application
ASTER: CSCI/Component Role in Retrieval of Data for Distribution
Database; contains metadata for search and retrieval
2
Requests meta-
data Staging
Disks;
creates and
passes
distributio
n
request
and metadata
files to
DDIST
1
Allocates space
in response to
Distribution Requests 4 Requests
staging disks for granule files; requests granule file retrieval by STMGT; updates order
3
Stages granulefiles to read-only cache disks
5
Provides Resource
Manager for
ftpPushes
6
Updates order
tracking data
7
Request ManagerEcDsStRequestManagerServer
379625-EMD-002, Rev. 02
Chaining and On-Demand Production (Cont.)
Ftp newly created AST_08 granule to scientist’s workstation and send ane-mail notification of the distribution
RETRIEVAL
Subscribe
Notification StandingOrders
Search& Order
OrderProducts
Deliver
Electronic
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
380625-EMD-002, Rev. 02
ASTER: Electronic Data Push Distribution Process
Ftp AST_08 (L2 Surface Temperature) granule toASTER Scientist’s workstation.
IngestSubsystem Ingest
Client
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
1. Ftp AST_08 data
2. Build e-mailnotification
3. Send e-mailnotification ofAST_08 distribution
CommunicationsSubsystem
FTP / E-Mail
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
381625-EMD-002, Rev. 02
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotification
CsFtFTPNotify
Bulk DataServer
BDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExec
VirtualTerminal
VT
CryptographicManagement
InterfaceCMI
InfrastructureLibrariesPF, SRF, URs,
Message Passing,Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.
EcCsRegistryServer
Machine-to-Machine Gwy.
EcCsMtMGateway
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Role in Electronic Data Push Distribution
Client for CSS
execution of actual
ftp
1
Builds e-mail notifica-tion that user’s order has been filled;sends it to the scientist
3
Executes ftp 2
Sendmail deliverse-mail messageto mail host
4
382625-EMD-002, Rev. 02
Chaining and On-Demand Production (Cont.)
RETRIEVAL
Subscribe
Notification StandingOrders
Search& Order
OrderProducts
Deliver
Electronic
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
Scientist uses EOSView to perform Quality Assurance (QA) check on the AST_08 product and e-mails DAAC Operations; DAAC Operations updates AST_08 inventory metadata
383625-EMD-002, Rev. 02
ASTER: QA Metadata Update Process
Science User uses EOSView tool to review AST_08 (L2 SurfaceTemperature) product and sends e-mail request to DAAC Operationsfor update of the Science QA Metadata for the granule; DAAC Operationsuses QA Monitor tool for the update.
IngestSubsystem Ingest
Client
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
CommunicationsSubsystem
SBSRV
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
1. Invoke EOSViewto view AST_08
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
2. Scientist e-mails DAAC withrequest to update the Science QAmetadata for the granule
3. InvokeQA Monitor
4. Request update ofQA metadata
5. Update metadata inventory, AST_08
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
384625-EMD-002, Rev. 02
Data ProcessingSubsystem(DPS) Data Proc
0110001011001011001011010111010110001011
Algorithm Integration and Test Tools(AITTL) CSCI
SSAP GUIEcDpAtSSAPGui
SSIT ManagerEcDpAtMgr
Define PGEEcDpAtCheckODL
EcDpAtCreateODLTemplateEcDpAtRegisterPGE
EcDpAtOpDbGui
View/CompareData ToolsEcDpAtBinDiffGui
EcDpAtCheckHdfFileEcDpAtMgrXdiff
Check SoftwareTools
EcDpAtCheckPCFEcDpAtBadFuncGui
EcDpAtBadFuncEcDpAtPrologs
Insert/AcquireTools
EcDpAtINSTALLEcDpAtInsertStaticFile
EcDpAtInsertExeTarFileEcDpAtInsertTestFile
EcDpAtStageDAPEcDpAtGetMCF
SDP Toolkit(SDPTK) CSCI
(Separatesoftware libraries
for SSI&T)
Processing(PRONG) CSCI
ExecutionManagement
EcDpPrEMDpPrDM (library)
PGEManagement
EcDpPrRunPGEEcDpPrRusage
DeletionServer
EcDpPrDeletion
Job ManagementEcDpPrJobMgmt
EcDpPrJobMgmtClientEcDpPrViewJobStates
AutoSysCOTS Product
QAMonitor
EcDpPrQaMonitorGUI
DataStore
Sybase Database (COTS)
Ground EventProcessEcDpPrGE
Data PreprocessingEcDpPrAm1EdosEphAttDPREP_PGEEcDpPrAm1FddAttitudeDPREP_PGE
EcDpPrAm1FddEphemerisDPREP_PGEEcDpPrDumpAttitudeDPREP
EcDpPrDumpEphemerisDPREPEcDpPrPm1FddEphemerisDPREP_PGE
EcDpPrPm1AttitudeDPREP_PGEEcDpPrAuraEphemerisDPREP_PGE
EcDpPrAuraAttitudeDPREP_PGE
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Role inQA Metadata Update
Submits request to
SDSRV for update
of granule metadata
2
Client (CLS)External/InternalUsers
On-Demand FormRequest Manager
(ODFRM) CSCI
HTML UserInterface
ODFRM
CGIPrograms
EcClOdRequest
Desktop(DESKT) CSCI
User ProfileGateway
EcClDtUserProfileGateway
Workbench(WKBCH) CSCI
HDF ProductViewing Tool
EOSView
ASTER DARTool
EcClWbJdt
Generates query calls to inventory database; packages and returns results; updates metadata
Database; contains metadata 4
3
Displays HDF files
for visualization 1
385625-EMD-002, Rev. 02
ASTER Scenario: Expedited DataIN
SERT
ION
RETRIEVAL
PRODUCTION
SubscribeSearch& Order
Store
ExternalData Provider User
Deliver
Generate Plan
Ingest
Data SubscriptionData InsertionData Notification
386625-EMD-002, Rev. 02
INSE
RTIO
N
Ingest
Polling
DAAC submits subscription, on behalf of scientist for notification of new ASTER Expedited data
Expedited Data
Subscribe
NotificationRegister
Subscription
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
387625-EMD-002, Rev. 02
ASTER: User Subscription Registration for AST_EXP Process
DAAC enters subscription, on behalf of scientist, for e-mail notificationof newly inserted ASTER Expedited Data product.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
IngestSubsystem Ingest
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
1. Subscribe (AST_EXP, e-mail)Communications
SubsystemRegistry
2. Get distribution options
Spatial Subscrip-tion Server
(SSS)
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
388625-EMD-002, Rev. 02
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotification
CsFtFTPNotify
Bulk DataServer
BDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExec
VirtualTerminal
VT
CryptographicManagement
InterfaceCMI
InfrastructureLibrariesPF, SRF, URs,
Message Passing,Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.
EcCsRegistryServer
Machine-to-Machine Gwy.
EcCsMtMGateway
Spatial Subscription Server(NBSRV) CSCI
Action DriverEcNbActionDriver.pl
NBSRV GUIEcNbGUI
Subscribed EventDriver
EcNbSubscribedEventDriver.pl
Deletion DriverEcNbDeleteRequestDriver.pl
SSS (NBSRV)Database
Sybase Server (COTS)
Recovery DriverEcNbRecoverDriver.pl
Spatial Subscrip-tion Server
(SSS)
ASTER: CSCI/Component Role in AST_EXP Subscription Registration
GUI used by User ServicesRepresentative to createsubscription for Science User
1
Stores subscription 3
Providesdistributionoptions
2
389625-EMD-002, Rev. 02
INSE
RTIO
N
Ingest
Polling
EDOS initiates AST_EXP (Expedited Data) ingestion via EDOSExpedited Data Set Delivery Record (EDR)
Expedited Data (Cont.)
Subscribe
NotificationRegister
Subscription
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
EDOS
390625-EMD-002, Rev. 02
ASTER: Polling Ingest for AST_EXP Process
Ingest begins polling specified location (directory), looking for an Expedited Data Set Delivery Record (EDR) file.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
IngestSubsystem Ingest
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
CommunicationsSubsystem
SBSRV
1. Ingest Subsystempolling for files
EDOS2. Move EDR
file todirectory
3. Read EDR file
4. Get mcf (AST_EXP)
5. Read AST_EXPfiles
EDOS transfers the file, via ftp, to thepredetermined directory. (Location, directory, username, and passwordare as specified in the EDOS-ECS Operations Agreement.)
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
391625-EMD-002, Rev. 02
IngestSubsystem
(INS) Ingest
Ingest(INGST) CSCI
Polling IngestClient Interface
ProcessEcInPolling
MediaIngest Interface
ProcessEcInGUI
EcInTapeReaderGUI
IngestRequest Manager
ProcessEcInReqMgr
IngestGranule Server
ProcessEcInGran
IngestDatabase
Sybase Server (COTS)
Cross-ModeIngest Interface
EcInEmailGWServer
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Role in Polling Ingest for AST_EXP
Polls remote location for files named *.EDR.XFR; packages requests
1
PackagesIngest Requestinto Granules
4
Staging Disk Server
receives request from
Ingest to create staging
disk and reserves space
5 Provides metadata configuration file (mcf)
3
Ingest reads delivery
record file and data files
from provider site via ftp 6
Requests metadata configuration file (mcf) from SDSRV; requests staging disk2
Request ManagerEcDsStRequestManagerServer
392625-EMD-002, Rev. 02
INSE
RTIO
N
Ingest
Polling
Expedited Data (Cont.)
Subscribe
NotificationRegister
Subscription
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
Archive ingested AST_EXP data; update catalogue with reference to AST_EXP
393625-EMD-002, Rev. 02
ASTER: AST_EXP Insertion Process
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
IngestSubsystem Ingest
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
CommunicationsSubsystem
SBSRV
Newly ingested AST_EXP granule is archived; inventory is updated.
1. Insert (AST_EXP)
2. Store (AST_EXP files)
3. Add (AST_EXP metadata)
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
394625-EMD-002, Rev. 02
IngestSubsystem
(INS) Ingest
Ingest(INGST) CSCI
Polling IngestClient Interface
ProcessEcInPolling
MediaIngest Interface
ProcessEcInGUI
EcInTapeReaderGUI
IngestRequest Manager
ProcessEcInReqMgr
IngestGranule Server
ProcessEcInGran
IngestDatabase
Sybase Server (COTS)
Cross-ModeIngest Interface
EcInEmailGWServer
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Rolein AST_EXP Insertion
Stores va
lidate
d
metadata
in th
e
inventory
4
Validates metadata;
requests archiving of
granule files; adds
metadata to inventory
2
Builds metadata
file; requests
SDSRV to
validate
metadata;
requests
SDSRV to insert
received files
1
Request ManagerEcDsStRequestManagerServer
Reads inserted
files from Ingest
staging disks
3
395625-EMD-002, Rev. 02
INSE
RTIO
N
Ingest
Polling
Expedited Data (Cont.)
Subscribe
NotificationRegister
Subscription
Store
ArchiveData
LocateData
CatalogueData
RetrieveData
Insert in the SDSRV database triggers an event in the NBSRV database. Subscribe e-mails ASTER Scientist notice of a completed AST_EXP granule insert.
396625-EMD-002, Rev. 02
ASTER: AST_EXP Event Notification Process
Notify all AST_EXP:Insert event subscribers.
CommunicationsSubsystem
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
IngestSubsystem Ingest
1. Trigger(AST_EXP:Insert)
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
2. Notify (AST_EXP:Insert,e-mail)
Spatial Subscrip-tion Server
(SSS)Data Mgmt
SubsystemDDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
397625-EMD-002, Rev. 02
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotification
CsFtFTPNotify
Bulk DataServer
BDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExec
VirtualTerminal
VT
CryptographicManagement
InterfaceCMI
InfrastructureLibrariesPF, SRF, URs,
Message Passing,Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.
EcCsRegistryServer
Machine-to-Machine Gwy.
EcCsMtMGateway
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
ASTER: CSCI/Component Rolein AST_EXP Event Notification
Sendmail deliverse-mail messageto mail host
4
AST_EXP:Insert triggers
event; places the event in
the NBSRV database
1
Spatial Subscription Server(NBSRV) CSCI
Action DriverEcNbActionDriver.pl
NBSRV GUIEcNbGUI
Subscribed EventDriver
EcNbSubscribedEventDriver.pl
Deletion DriverEcNbDeleteRequestDriver.pl
SSS (NBSRV)Database
Sybase Server (COTS)
Recovery DriverEcNbRecoverDriver.pl
Spatial Subscrip-tion Server
(SSS) Detects event; gets metadata from SDSRV
database and stores it in NBSRV database;
matches event to subscription and places
subscription into the Action Queue
2
Detects
subscription;
composes and
sends e-mail
notification of
the AST_EXP:Insert
event3
398625-EMD-002, Rev. 02
User Registration
Science User starts the form for first-time User Registration from the EOS Data Gateway Web Client and submits registration information.
RETRIEVAL
UserAccess
UserRegistrationSubscribe
Deliver
ManageSystemMonitor &Coordinate
ManageApplications
Logistics
399625-EMD-002, Rev. 02
User Registration Request Process
Scientist invokes the form for first-time User Registration from the EOS DataGateway web client and then enters and submits registration data.
CommunicationsSubsystem
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
IngestSubsystem Ingest
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection 1. Request UserRegistration
Systems MgmtSubsystem
(MSS)Data Mgmt
SubsystemDDICT2. Submit
RegistrationRequest Order Manager
Subsystem(OMS)
ProductDistributionSystem (PDS)
400625-EMD-002, Rev. 02
System MgmtSubsystem
(MSS)
ManagementLogistics CSCI
(MLCI)
Inventory/Logistics/Maintenance Mgr
Remedy ARS (COTS)
SoftwareChange MgrClearCase (COTS)
SoftwareLicense Mgr
lmgrd (FLEXlm server daemon) (COTS)FLEXlm-enabled COTS Application
ManagementCSCI (MCI)
Network andEnterprise
ManagementWhatsUp GoldWhazzup???
Securityanlpasswd (COTS)
TCP Wrappers (COTS)Tripwire (COTS)
Crack (COTS)
AccountabilityEcMsAcRegUserSrvr
EcMsAcSMCRegUserGUIEcMsAcDAACRegUserGUI
EcMsAcOrderSrvrEcMsAcOrderGUI
Sybase Replication ServerSybase ASE
Trouble Ticketaruser GUI (COTS)
aradmin GUI (COTS)notifier GUI (COTS)
notifier daemon (COTS)Remedy ARS (COTS)
armail daemon (COTS)Sendmail Advanced Message
Server (SAMS) (COTS)Sybase ASE (COTS)
NetworkBackup/RestoreNetworker Server (COTS)Networker Client (COTS)
ASTER E-mailHeader Handler
UNIX sendmail daemon (COTS)EcMsAsAddHeader.pl
EcMsAsRemoveHeader.ple-mail utility (COTS)
CSCI/Component Role inUser Registration Request
Client (CLS)External/InternalUsers
On-Demand FormRequest Manager
(ODFRM) CSCI
HTML UserInterface
ODFRM
CGIPrograms
EcClOdRequest
Desktop(DESKT) CSCI
User ProfileGateway
EcClDtUserProfileGateway
Workbench(WKBCH) CSCI
HDF ProductViewing Tool
EOSView
ASTER DARTool
EcClWbJdt
Receives request from EDG and submits it to User Registration Server (at SMC)
1
Saves request as a new User
Profile at SMC, for replication
to each remote DAAC2
401625-EMD-002, Rev. 02
User Registration (Cont.)
Science User calls home DAAC Operations (User Services) to request any needed privileges (e.g., access to restricted granules, ASTER DAR privileges); DAAC Operations logs into SMC remotely, retrieves User Profile, and assigns approved privileges
RETRIEVAL
UserAccess
UserRegistrationSubscribe
ManageSystemMonitor &Coordinate
ManageApplications
Logistics
Deliver
402625-EMD-002, Rev. 02
User Registration Approval Process
DAAC User Services Representative at home DAAC uses User RegistrationServer Graphical User Interface (GUI) at SMC to retrieve User Profile andupdate the MSS User Profile approved privileges, which are then replicated.
CommunicationsSubsystem
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
IngestSubsystem Ingest
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
1. Retrieve UserProfile andupdate theprivileges
Systems MgmtSubsystem
(MSS)Data Mgmt
SubsystemDDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
403625-EMD-002, Rev. 02
System MgmtSubsystem
(MSS)
ManagementLogistics CSCI
(MLCI)
Inventory/Logistics/Maintenance Mgr
Remedy ARS (COTS)
SoftwareChange MgrClearCase (COTS)
SoftwareLicense Mgr
lmgrd (FLEXlm server daemon) (COTS)FLEXlm-enabled COTS Application
ManagementCSCI (MCI)
Network andEnterprise
ManagementWhatsUp GoldWhazzup???
Securityanlpasswd (COTS)
TCP Wrappers (COTS)Tripwire (COTS)
Crack (COTS)
AccountabilityEcMsAcRegUserSrvr
EcMsAcSMCRegUserGUIEcMsAcDAACRegUserGUI
EcMsAcOrderSrvrEcMsAcOrderGUI
Sybase Replication ServerSybase ASE
Trouble Ticketaruser GUI (COTS)
aradmin GUI (COTS)notifier GUI (COTS)
notifier daemon (COTS)Remedy ARS (COTS)
armail daemon (COTS)Sendmail Advanced Message
Server (SAMS) (COTS)Sybase ASE (COTS)
NetworkBackup/RestoreNetworker Server (COTS)Networker Client (COTS)
ASTER E-mailHeader Handler
UNIX sendmail daemon (COTS)EcMsAsAddHeader.pl
EcMsAsRemoveHeader.ple-mail utility (COTS)
CSCI/Component Role inUser Registration Approval
GUI at SMC retrieves User
Profile; operator updates User
Profile information; server
stores the updated User
Profile, which is replicated for
read-only access at all sites
404625-EMD-002, Rev. 02
User Registration (Cont.)
RETRIEVAL
UserAccess
UserRegistrationSubscribe
ManageSystemMonitor &Coordinate
ManageApplications
Logistics
Deliver
System Management Subsystem (MSS) sends e-mail to the Science User confirming registration and anyspecial privileges
405625-EMD-002, Rev. 02
User Registration Confirmation Process
System Management Subsystem (MSS) User Registration Servercreates and sends e-mail to the Science User confirming registration andany special privileges.
CommunicationsSubsystem
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
IngestSubsystem Ingest
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
1. Build user e-mailmessage
2. E-mail notificationSystems MgmtSubsystem
(MSS)
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
406625-EMD-002, Rev. 02
System MgmtSubsystem
(MSS)
ManagementLogistics CSCI
(MLCI)
Inventory/Logistics/Maintenance Mgr
Remedy ARS (COTS)
SoftwareChange MgrClearCase (COTS)
SoftwareLicense Mgr
lmgrd (FLEXlm server daemon) (COTS)FLEXlm-enabled COTS Application
ManagementCSCI (MCI)
Network andEnterprise
ManagementWhatsUp GoldWhazzup???
Securityanlpasswd (COTS)
TCP Wrappers (COTS)Tripwire (COTS)
Crack (COTS)
AccountabilityEcMsAcRegUserSrvr
EcMsAcSMCRegUserGUIEcMsAcDAACRegUserGUI
EcMsAcOrderSrvrEcMsAcOrderGUI
Sybase Replication ServerSybase ASE
Trouble Ticketaruser GUI (COTS)
aradmin GUI (COTS)notifier GUI (COTS)
notifier daemon (COTS)Remedy ARS (COTS)
armail daemon (COTS)Sendmail Advanced Message
Server (SAMS) (COTS)Sybase ASE (COTS)
NetworkBackup/RestoreNetworker Server (COTS)Networker Client (COTS)
ASTER E-mailHeader Handler
UNIX sendmail daemon (COTS)EcMsAsAddHeader.pl
EcMsAsRemoveHeader.ple-mail utility (COTS)
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotification
CsFtFTPNotify
Bulk DataServer
BDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExec
VirtualTerminal
VT
CryptographicManagement
InterfaceCMI
InfrastructureLibrariesPF, SRF, URs,
Message Passing,Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.
EcCsRegistryServer
Machine-to-Machine Gwy.
EcCsMtMGateway
CSCI/Component Role inUser Registration Confirmation
User Registration Server builds
e-mail message confirming
registration and privileges1
Sends e-mail notification confirming registrationand privileges
2
407625-EMD-002, Rev. 02
MODIS Scenario
MODIS Goals- Standing Order Submittal- Polling Ingest- Standard Production and Chaining- Data Access
MODIS PreconditionsMODIS ESDTs Inserted into ECS- MOD000, MOD01, MOD02, MOD03, MOD07, MOD35,MODVOLC, MOD35ANC, GDAS_0ZF, Others
MODIS PGEs passed SSI&T and installed- PGE01, PGE02, PGE03
Ancillary and static data inserted into Data Server
MODIS
Automatic PGE Scheduling
Chaining of PGEs
Standing Orders
408625-EMD-002, Rev. 02
MODIS Scenario: Data AccessIN
SERT
ION
RETRIEVAL
PRODUCTION
SubscribeSearch& Order
Store
ExternalData Provider User
Deliver
Generate Plan
Ingest
Search & Order Granules
409625-EMD-002, Rev. 02
MODIS: Data Access
RETRIEVAL
Subscribe
Science User searches for MODIS granules that are inECS data holdings
Search& OrderInventorySearch
OrderProducts
Deliver
Electronic
Media
StoreLocateData
RetrieveData
410625-EMD-002, Rev. 02
MODIS: Inventory Search Process
Science User uses EOS Data Gateway (EDG) Web Client to perform an InventorySearch for MOD07_L2 data (MODIS Level 2 Joint Atmosphere Product of profiles, ozone, water, and other components) over a specified spatial/temporaldomain.
1. Search forMOD07_L2granules
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
IngestSubsystem Ingest
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
CommunicationsSubsystem
2. Submit query (MOD07_L2,space&time)
3. Submit translatedquery to SDSRV
EDG
UserAuthentication
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
411625-EMD-002, Rev. 02
Data MgmtSubsystem
(DMS)
Data Dictionary(DDICT) CSCI
Data DictionaryMaintenance Tool
EcDmMaintTool
Data DictionaryASE Server
Sybase Server (COTS)
Data DictionaryServer
EcDmDictService
Version Zero Gateway(V0 GTWAY) CSCI
V0 to ECSGateway Server
EcDmV0ToECSGateway
MODIS: CSCI/Component Rolein Inventory Search
Translates search criteria to a search object and submits query to SDSRV Search service
3
Client (CLS)External/InternalUsers
On-Demand FormRequest Manager
(ODFRM) CSCI
HTML UserInterface
ODFRM
CGIPrograms
EcClOdRequest
Desktop(DESKT) CSCI
User ProfileGateway
EcClDtUserProfileGateway
Workbench(WKBCH) CSCI
HDF ProductViewing Tool
EOSView
ASTER DARTool
EcClWbJdt
Provides user authentication and privileges
2
System MgmtSubsystem
(MSS)
ManagementCSCI (MCI)
Network andEnterprise
ManagementWhatsUp GoldWhazzup???
Securityanlpasswd (COTS)
TCP Wrappers (COTS)Tripwire (COTS)Crack (COTS)
AccountabilityEcMsAcRegUserSrvr
EcMsAcSMCRegUserGUIEcMsAcDAACRegUserGUI
EcMsAcOrderSrvrEcMsAcOrderGUI
Sybase Replication ServerSybase ASE
Trouble Ticketaruser GUI (COTS)
aradmin GUI (COTS)notifier GUI (COTS)
notifier daemon (COTS)Remedy ARS (COTS)
armail daemon (COTS)Sendmail Advanced Message
Server (SAMS) (COTS)Sybase ASE (COTS)
NetworkBackup/Restore
Networker Server (COTS)Networker Client (COTS)
ASTER E-mailHeader Handler
UNIX sendmail daemon (COTS)EcMsAsAddHeader.pl
EcMsAsRemoveHeader.ple-mail utility (COTS)
ManagementLogistics CSCI
(MLCI)
Inventory/Logistics/Maintenance Mgr
Remedy ARS (COTS)
SoftwareChange MgrClearCase (COTS)
SoftwareLicense Mgr
lmgrd (FLEXlm server daemon) (COTS)FLEXlm-enabled COTS Application
Holds User Profile information 1
412625-EMD-002, Rev. 02
MODIS: Data Access (Cont.)
Granules of interest are locatedin the ECS data holdings
RETRIEVAL
Subscribe
Search& OrderInventorySearch
OrderProducts
Deliver
Electronic
Media
StoreLocateData
RetrieveData
413625-EMD-002, Rev. 02
MODIS: Data Search Process
SDSRV queries Sybase/SQS database for granules meeting search criteriaand returns results.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
IngestSubsystem Ingest
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
1. Query Sybase
Client
CommunicationsSubsystemRegistry
2. Package andreturn results 4. Return translated results for
display to the user
3. Retrieve mediaoptions fromRegistry Server
Data MgmtSubsystem
DDICT
Order ManagerSubsystem
(OMS)
ProductDistributionSystem (PDS)
414625-EMD-002, Rev. 02
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotification
CsFtFTPNotify
Bulk DataServer
BDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExec
VirtualTerminal
VT
CryptographicManagement
InterfaceCMI
InfrastructureLibrariesPF, SRF, URs,
Message Passing,Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.
EcCsRegistryServer
Machine-to-Machine Gwy.
EcCsMtMGateway
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
MODIS: CSCI/Component Rolein Data Search
Data MgmtSubsystem
(DMS)
Data Dictionary(DDICT) CSCI
Data DictionaryMaintenance Tool
EcDmMaintTool
Data DictionaryASE Server
Sybase Server (COTS)
Data DictionaryServer
EcDmDictService
Version Zero Gateway(V0 GTWAY) CSCI
V0 to ECSGateway Server
EcDmV0ToECSGatewayBreaks down Query and generates calls to inventory database; packages and returns results
Database; contains metadata for search and retrieval
1
2
Provides
media options
associated
with archived
data to V0
Gateway3
Translates results and returns them to the EOS Data Gateway Web Client for display to the user
4
415625-EMD-002, Rev. 02
MODIS: Data Access (Cont.)
Science User decides to ordergranules of interest
RETRIEVAL
Subscribe
Search& OrderInventorySearch
OrderProducts
Deliver
Electronic
Media
StoreLocateData
RetrieveData
416625-EMD-002, Rev. 02
CommunicationsSubsystemRegistry
IngestSubsystem Ingest
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
2. Submit order(MOD07_L2, ftpPush)
EDG
UserAuthentication
1. Order MOD07_L2granules
Science User uses EOS Data Gateway Web Client to order MOD07_L2 Granules via ftp push.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
3. InsertProductDistributionRequest
Data MgmtSubsystem
DDICT
ProductDistributionSystem (PDS)
Data Pool(DPL)
DATAPOOL
Order ManagerSubsystem
(OMS)
MODIS: Order MOD07_L2
417625-EMD-002, Rev. 02
System MgmtSubsystem
(MSS)
ManagementCSCI (MCI)
Network andEnterprise
ManagementWhatsUp GoldWhazzup???
Securityanlpasswd (COTS)
TCP Wrappers (COTS)Tripwire (COTS)Crack (COTS)
AccountabilityEcMsAcRegUserSrvr
EcMsAcSMCRegUserGUIEcMsAcDAACRegUserGUI
EcMsAcOrderSrvrEcMsAcOrderGUI
Sybase Replication ServerSybase ASE
Trouble Ticketaruser GUI (COTS)
aradmin GUI (COTS)notifier GUI (COTS)
notifier daemon (COTS)Remedy ARS (COTS)
armail daemon (COTS)Sendmail Advanced Message
Server (SAMS) (COTS)Sybase ASE (COTS)
NetworkBackup/Restore
Networker Server (COTS)Networker Client (COTS)
ASTER E-mailHeader Handler
UNIX sendmail daemon (COTS)EcMsAsAddHeader.pl
EcMsAsRemoveHeader.ple-mail utility (COTS)
ManagementLogistics CSCI
(MLCI)
Inventory/Logistics/Maintenance Mgr
Remedy ARS (COTS)
SoftwareChange MgrClearCase (COTS)
SoftwareLicense Mgr
lmgrd (FLEXlm server daemon) (COTS)FLEXlm-enabled COTS Application
MODIS: CSCI/Component Role in Order MOD07_L2
Data MgmtSubsystem
(DMS)
Data Dictionary(DDICT) CSCI
Data DictionaryMaintenance Tool
EcDmMaintTool
Data DictionaryASE Server
Sybase Server (COTS)
Data DictionaryServer
EcDmDictServer
Version Zero Gateway(V0 GTWAY) CSCI
V0 to ECSGateway ServerEcDmV0ToECSGateway
Passes request forMOD07_L2 granules to Order Manager Database
4
Client (CLS)External/InternalUsers
On-Demand FormRequest Manager
(ODFRM) CSCI
HTML UserInterface
ODFRM
CGIPrograms
EcClOdRequest
Desktop(DESKT) CSCI
User ProfileGateway
EcClDtUserProfileGateway
Workbench(WKBCH) CSCI
User RegistrationTool (URT) Scripts
EcClWbUrEcClWbUrConfirmEcClWbUrSubmit
HDF ProductViewing Tool
EOSView
ASTER DARTool
EcClWbJdt
Order Manager GUI(OMGUI) CSCI
Order Manager GUIEcOmGui
Order Manager Server(OMSRV) CSCI
Order ManagerServer
EcOmOrderManager
Order ManagerASE Server
Sybase Server (COTS)
Order ManagerSubsystem
(OMS)
Production Module CSCI
ProductionModule
EcOmPdMediaProduction
Provides ECS names for V0 terms
3
Stores request in database
5
Transmits User Profile
to EOS Data Gateway
Web Client
1
Provides UserProfile2
418625-EMD-002, Rev. 02
Ftp data to Science User;notify Science User ofdistribution
MODIS: Data Access (Cont.)
RETRIEVAL
Subscribe
Search& OrderInventorySearch
OrderProducts
Deliver
Electronic
Media
StoreLocateData
RetrieveData
419625-EMD-002, Rev. 02
IngestSubsystem Ingest
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
EDG
Push MOD07_L2 data to Science User’s workstation. Send e-mail to Science User with notification of distribution.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
ProductDistributionSystem (PDS)
5. Ftp MOD07_L2data
CommunicationsSubsystem
1. Submit granule staging requests2. Requestdata insert toData Pool
6. Build e-mailnotification
Data Pool(DPL)
DATAPOOL
4. Copy granules from Data Pool
7. Send e-mail notificationof MOD07_L2 distribution
Order ManagerSubsystem
(OMS)
3. Copy granules
MODIS: Distribute MOD07_L2
420625-EMD-002, Rev. 02
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotification
CsFtFTPNotify
Bulk DataServer
BDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExec
VirtualTerminal
VT
CryptographicManagement
InterfaceCMI
InfrastructureLibrariesPF, SRF, URs,
Message Passing,Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.
EcCsRegistryServer
Machine-to-Machine Gwy.
EcCsMtMGateway
MODIS: CSCI/Component Role in Distribute MOD07_L2
Data Server Subsystem (DSS)
SDSRVDDIST
STMGT
Inven-toryDB
Collection
Storage Management(STMGT) CSCI
Cache/StagingManager Process
EcDsStCacheManagerServerEcDsStStagingDiskServer
MediaServer Process
EcDsSt8MMServerEcDsStDTFServerEcDsStFtpServer
EcDsStCopyServer
Pull MonitorProcess
EcDsStPullMonitorServer
Archive ServerEcDsStArchiveServer
EcDsStmgtGui
Data BaseSybase Server (COTS)
Request ManagerEcDsStRequest Manager Server
Science Data Server(SDSRV) CSCI
ScienceData Server
EcDsScienceDataServer
HDF EOS ServerEcDsHdfEosServer
Science Data Server GUIEcDsSdsrvGui
SybaseSybase Server (COTS)
Granule DeletionAdministration Tool
EcDsGranuleDelete
SQSSpatial Query Server (COTS)
SDSRV Command Line Interface
EcDsSCLI
Data Distribution(DDIST) CSCI
DataDistribution
EcDsDistributionServer
DataDistribution GUI
EcDsDdistGui
Data BaseSybase Server (COTS)
EPD Serverepdserver DCLI
EcDsDdDCLI
Data Pool(DPL)
DATAPOOL
Data Pool Management(DPMGT) CSCI
COTS FTPServer
wuftp
Data PoolWeb Access GUI
EcDlWebAccess
Data PoolDatabase
Sybase Server (COTS)
Data PoolInsert UtilityEcDlInsertUtilityEcDlActionDriverEcDlQuickServer
EcDlM2XTApp
Data Pool Maintenance(DPMNT) CSCI
Data PoolUpdate Granule
ExpirationEcDlUpdateGranule.pl
DPM GUIEcDlDpmHomepage.pl
Data Pool AccessStatistics UtilityEcDlRollupWebLogs.plEcDlRollupFtpLogs.pl
Data PoolArchive Access
Statistics ScriptsDlDbArchiveAccessStatDlDbDeleteAccessStat
DlDbRestoreAccessStat
Data Pool CleanupEcDlCleanupDataPool.pl
Data PoolMost RecentInsert Utility
EcDlMostRecentInsert.pl
Data PoolCollection
Remapping UtilityEcDlRemap.pl
Data PoolDensity Map
UtilityEcDlDensityMapUtility.pl
Data PoolStatistics Table
Population UtilityEcDlPopulateStatTables.pl
Order Manager GUI(OMGUI) CSCI
Order Manager GUIEcOmGui
Order Manager Server(OMSRV) CSCI
Order ManagerServer
EcOmOrderManager
Order ManagerASE Server
Sybase Server (COTS)
Order ManagerSubsystem
(OMS)
Production Module CSCI
ProductionModule
EcOmPdMediaProduction
Insert granule staging requests
2
Fetch request forMOD07_L2 granules from Order Manager Database; submit granule staging requests; request ftp push; request e-mail support
1
Request data insert to Data Pool from cache or archive
3
Copy granules to Data Pool4
Sendmail deliverse-mail messageto mail host
5
421625-EMD-002, Rev. 02
MODIS: Data Access (Cont.)
RETRIEVAL
Subscribe
Science User searches for MODIS granules that are inthe Data Pool
Search& OrderInventorySearch
OrderProducts
Deliver
Electronic
Media
StoreLocateData
RetrieveData
422625-EMD-002, Rev. 02
MODIS: Data Pool Search and Order ProcessScience User uses the Data Pool Web Access GUI to search for MOD11_L2 data (MODIS/Terra Land Surface Temperature/Emissivity 5-Min L2 Swath 1km)over a specified spatial/temporal domain and submit an order for delivery on DVD.
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
IngestSubsystem Ingest
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
CommunicationsSubsystem
1. Submit query (MOD11_L2,space&time)
ProductDistributionSystem (PDS)
4. Select mediaoption
2. Request mediaoptions
5. Submit hard media order
(MOD11_L2, DVD)
3. Report media options
Order ManagerSubsystem
(OMS)
Data Pool(DPL)
DATAPOOL
Systems MgmtSubsystem
(MSS)
423625-EMD-002, Rev. 02
Data Pool(DPL)
DATAPOOL
Data Pool Management(DPMGT) CSCI
COTS FTPServer
wuftp
Data PoolWeb Access GUI
EcDlWebAccess
Data PoolDatabase
Sybase Server (COTS)
Data PoolInsert UtilityEcDlInsertUtilityEcDlActionDriverEcDlQuickServer
EcDlM2XTApp
Data Pool Maintenance(DPMNT) CSCI
Data PoolUpdate Granule
ExpirationEcDlUpdateGranule.pl
DPM GUIEcDlDpmHomepage.pl
Data Pool AccessStatistics UtilityEcDlRollupWebLogs.plEcDlRollupFtpLogs.pl
Data PoolArchive Access
Statistics ScriptsDlDbArchiveAccessStatDlDbDeleteAccessStat
DlDbRestoreAccessStat
Data Pool CleanupEcDlCleanupDataPool.pl
Data PoolMost RecentInsert Utility
EcDlMostRecentInsert.pl
Data PoolCollection
Remapping UtilityEcDlRemap.pl
Data PoolDensity Map
UtilityEcDlDensityMapUtility.pl
Data PoolStatistics Table
Population UtilityEcDlPopulateStatTables.pl
MODIS: CSCI/Component Rolein Data Pool Search and Order
Order Manager GUI(OMGUI) CSCI
Order Manager GUIEcOmGui
Order Manager Server(OMSRV) CSCI
Order ManagerServer
EcOmOrderManager
Order ManagerASE Server
Sybase Server (COTS)
Order ManagerSubsystem
(OMS)
Production Module CSCI
ProductionModule
EcOmPdMediaProduction
Determine media options for specified granule(s);
2
Locate granules of interest; request media options; select media option; submit order
1
Insert order 3
424625-EMD-002, Rev. 02
RETRIEVAL
Subscribe
Search& OrderInventorySearch
OrderProducts
StoreLocateData
RetrieveData
Request for physical media distribution of granules is dispatched to the relevant Production Module; granules are copied to DVDand prepared for shipment to scientist
MODIS: Data Access (Cont.)
Deliver
Electronic
Media
425625-EMD-002, Rev. 02
Data Server Subsystem
SDSRVDDIST
STMGT
Inven-toryDB
Collection
IngestSubsystem Ingest
DataProcessingSubsystem Data Proc
0110001011001011001011010111010110001011
Client
PlanningSubsystem Planning
abc
def
ghi
jkl
mno
Order ManagerSubsystem
(OMS)
Systems MgmtSubsystem
(MSS)
Data Pool(DPL)
DATAPOOL
Dispatch the validated request for MODIS granules to the appropriate production module. Copy granules from Data Pool to DVD. Prepare shippingdocuments and e-mail notification of shipment. Ship tape to Science User.Send e-mail notification of shipment.
MODIS: DVD Creation/Distribution Process
3. Build e-mailnotification
4. Send e-mail notification
5. Update OrderTracking Status
1. Copy granules
ProductDistributionSystem (PDS)
CommunicationsSubsystem
2. Ship DVD
426625-EMD-002, Rev. 02
CSS/DistributedCommunications Software
DomainName Server
resolver
Mail SupportE-Mail Server
Distributed ComputingConfiguration Item (DCCI)
ASTER DARGateway
EcGwDARServer
E-Mail ParserGateway
EcCsEmailParser
FTPNotification
CsFtFTPNotify
Bulk DataServer
BDSpro
NetworkFile System
NFS
CCSMiddleware
SupportEcCsIdNameServer
SubscriptionServer
EcSbSubServerEcSbGui
File TransferProtocol
FTP
FilecopyEcUtFileCopyEcUtFileExec
VirtualTerminal
VT
CryptographicManagement
InterfaceCMI
InfrastructureLibrariesPF, SRF, URs,
Message Passing,Event Logging,Server Locator,
Failure RecoveryFramework,
EcPo Connections,Time Service
MOJOGateway
EcCsMojoGateway
ConfigurationRegistry Srvr.
EcCsRegistryServer
Machine-to-Machine Gwy.
EcCsMtMGateway
System MgmtSubsystem
(MSS)
ManagementCSCI (MCI)
Network andEnterprise
ManagementWhatsUp GoldWhazzup???
Securityanlpasswd (COTS)
TCP Wrappers (COTS)Tripwire (COTS)Crack (COTS)
AccountabilityEcMsAcRegUserSrvr
EcMsAcSMCRegUserGUIEcMsAcDAACRegUserGUI
EcMsAcOrderSrvrEcMsAcOrderGUI
Sybase Replication ServerSybase ASE
Trouble Ticketaruser GUI (COTS)
aradmin GUI (COTS)notifier GUI (COTS)
notifier daemon (COTS)Remedy ARS (COTS)
armail daemon (COTS)Sendmail Advanced Message
Server (SAMS) (COTS)Sybase ASE (COTS)
NetworkBackup/Restore
Networker Server (COTS)Networker Client (COTS)
ASTER E-mailHeader Handler
UNIX sendmail daemon (COTS)EcMsAsAddHeader.pl
EcMsAsRemoveHeader.ple-mail utility (COTS)
ManagementLogistics CSCI
(MLCI)
Inventory/Logistics/Maintenance Mgr
Remedy ARS (COTS)
SoftwareChange MgrClearCase (COTS)
SoftwareLicense Mgr
lmgrd (FLEXlm server daemon) (COTS)FLEXlm-enabled COTS Application
MODIS: CSCI/Component Role in Distribute on Physical Media
Order Manager GUI(OMGUI) CSCI
Order Manager GUIEcOmGui
Order Manager Server(OMSRV) CSCI
Order ManagerServer
EcOmOrderManager
Order ManagerASE Server
Sybase Server (COTS)
Order ManagerSubsystem
(OMS)
Production Module CSCI
ProductionModule
EcOmPdMediaProduction
Posts OrderTracking
status updates from OMS to database
5
Sendmaildeliverse-mail messageto mail host
63
Displays request/volume status; activates PMD requests; marks changes in request status
3
Copies data from Data Pool; creates image files; prints jewel case insert; prints summary
4
Dispatches request to production module; provides order-tracking status updates to MSS; prints packing list; prints shipping labels; prepares Distribution Notice for e-mail
1
Stores PMD request information
2
427625-EMD-002, Rev. 02
Summary
• Thirteen subsystems• Special Access: ASTER
– DAR– Expedited Data Support
• Data Transfer from External Provider to ECS– DTF-2 Tape Ingest– Polling Ingest
• Planning and Data Processing– Standing Orders– On-Demand Processing: Chaining– QA Update
428625-EMD-002, Rev. 02
Summary (Cont.)
• Data Access and Distribution– User Registration– Archive Data Search– Electronic Distribution– Data Pool Search– Media Distribution