Page 1 JSOC Peer Review 17Mar2005

JSOC Pipeline Processing System Components

Database Server

SUMSStorage Unit

Management System

DRMSData Record

Management SystemSUMS Tape Farm

SUMS Disks

Pipeline Program, “module”

Record Manage

ment

Keyword Access

Data Access

DRMS Library

Link Manage

ment

Utility LibrariesJSOC Science

Libraries

Record Cache

PUIPipeline User

Interface

Pipeline processing

plan

Processing script, “mapfile”

List of pipeline modules with needed datasets for input, output

Pipeline Operato

r

Processing History Log

Page 2 JSOC Peer Review 17Mar2005

Storage Unit Management Sybsystem (SUMS) API

• SUM *SUM_open(char *dbname)

Start a session with the SUMS

• int SUM_close(SUM *sum)

End a session with the SUMS

• int SUM_alloc(SUM *sum)

Allocate a storage unit on the disks

• int SUM_get(SUM *sum)

Get the requested storage units

• int SUM_put(SUM *sum)

Put a previously allocated storage unit

• int SUM_poll(SUM *sum)

Check if a previous request is complete

• int SUM_wait(SUM *sum)

Wait until previous request is complete

Page 3 JSOC Peer Review 17Mar2005

SUM_get() API Call Sequence

DRMS Client

SUM Server

SUM Library

SUM_get()

get Procedure

resultpend

Sample SUM Call

Oracle Server

get storage units

offline

Tape Server

retrieve storage units

resultpend

SUM_poll() orSUM_wait()

ack

SUM_flow.vsd

online loc

ack

retrieve storage units

online loc

response Procedure

Page 4 JSOC Peer Review 17Mar2005

HMI Dataset Sequence

tlm packets

camera& hk data~1700bytes

HMI 55Mbs

filtergrams

n:132MBImage

n:m

observables

32MBImage

/home/jim/soffice/user/work/dataset_seq3.sxd

tlm

lev0

lev1

Ancillary Data

HK

Page 5 JSOC Peer Review 17Mar2005

Pipeline User Interface (PUI) Block Diagram

Data Product

Plan Table

Data Query

Oracle DB

Map BuildingScripts

Map Execution

PUI GUI

pui_svc

pipelineexecution

Page 6 JSOC Peer Review 17Mar2005

Pipeline User Interface (PUI)

Page 7 JSOC Peer Review 17Mar2005

Pipeline User Interface (PUI)

Page 8 JSOC Peer Review 17Mar2005

Pipeline User Interface (PUI)

Page 9 JSOC Peer Review 17Mar2005

Pipeline User Interface (PUI)

Page 10 JSOC Peer Review 17Mar2005

Pipeline User Interface (PUI)

Page 11 JSOC Peer Review 17Mar2005

HK Data Flow

MOCDDS

Science SOC T & C SOC

GSFC

dailyLR

realtimeLR APID

Lockheed

WSC

Stanford

socket ftp

realtimeHR APID

ftpdaily LR APID

FDS/PLANNING

HK DefinitionTables

HK KeywordDB

realtime LR APID

FDS/PLAN

Page 12 JSOC Peer Review 17Mar2005

CM with CVS