Implementation Review 1

Moving Pre-Archive Pipeline Processing

March 14, 2003Forrest Hamilton/OPUS Ops

Implementation Review 2

Goals Switch over will not disrupt the routine flow of

science data processing or archiving Free up processing power for OTFR on

ACDSDOPS Will help when ACS data become public

Gain experience to use when we switch over the more complex OTFR pipelines

Implementation Review 3

Plan Switch pre-archive science processing space over

from ACDSDOPS storage array to EMC Migrate pre-archive science processing on

ACDSDOPS to the Sun Fire 15K Both of these configurations will have already

been successfully tested in the new test environment

Implementation Review 4

Implementation Review 5

EMC StorageArray

2GB Switch

SUN FIRE15K

EMC BackupManager

SUN BackupManager

HST

FROM: ScienceCommunity(thru StarView)

FUSEEngineering Team

TO: ScienceCommunity

Fatkat

Imagemaker

Fiber Channel

Ethernet 10/100Connections to

Institute Networkand Internet

ARCHC SCARAB

Ops-Jukeboxes

Corsair

Dev-Jukeboxes

DVD DrivesW DVDW DVD

W DVD

DVD DrivesW DVDW DVD

W DVD

CD Burners

CD Burners

Implementation Review 6

OPUS Installation Primer

opus_login.csh

# setenv SOGS_DISK setenv OAPI_LIB setenv OPUS_DEFINITION setenv OPUS_HOME_DIR setenv DSQUERY setenv OPUS_DB setenv jref setenv jtab

opus_definitions_dir =

*.path

STAGE_FILE =OPUS_HOME_DIR =OPUS_TEMP_DIR =POD_IN_DIR = ACS_CAL_DIR = STI_CAL_DIR = NIC_CAL_DIR =WF2_CAL_DIR

*.pipeline

dp_acs professor acdsdopsss_acs professor acdsdopsdv_acs professor acdsdopswcsacs professor acdsdopsgc_acs professor acdsdopscalacs professor acdsdops

/home/opus/defs/ /store/opus14_2/defs/unix/

source opus_login.csh

.alias

Pre-archive pipeline process

Implementation Review 7

Post-installation Configuration

opus_definitions_dir

/home/opus/defs/

Ops

files

OPUS Installation Primer

opus_definitions_dir

Nominal Configuration

Ops

files

DSB

files

/home/opus/defs/

/store/opus14_2/defs/unix/

Pre-installation Configuration

Ops

files

DSB

files

Modified Ops files

New DSB files

/home/opus/defs/

/home/opus/opus14_3/

/store/opus14_2/defs/unix/

/store/opus14_3/defs/unix/

opus_definitions_dir

Modified Ops files

/store/opus14_3/defs/unix/

New DSB

files

Implementation Review 8

Migration Process: ACDSDOPS EMC Perform pipeline account access checks Deployment Review

Needed to determine readiness and appropriate time for switch to EMC

Copy OPUS software onto EMC No new OPUS software installation required

Create a copy of pipeline configuration files and edit in the changes needed to point to the EMC

Standard procedure used for OPUS software upgrades Terminate the CDBS (reference files) pipeline Copy reference data onto EMC

Implementation Review 9

Migration Process: ACDSDOPS EMC

Terminate pre-archive pipeline processing to allow data to clear off of the ACDSDOPS disks Done to prevent major cleanups if problems arise Migrate any trouble data over

Activate modified configuration files Bring up pre-archive processing using EMC

Implementation Review 10

Migration Process: ACDSDOPS Sun Fire 15K Verify PACOR-A interface Verify DADS interface Migrate OPUS pipeline configuration directory hierarchy

from ACDSDOPS to Sun Fire 15K Similar process has already been done to activate the EDPS

Verify pipeline account access Deployment Review Terminate pre-archive processing on ACDSDOPS Install the OPUS software version that supports Sun Fire

15K processing

Implementation Review 11

Migration Process: ACDSDOPS Sun Fire 15k Activate configuration files pointing to the Sun

Fire 15K Make use of new reference file area (GEIS format is

OS dependent)

Bring up pre-archive processing using the Sun Fire 15K

Implementation Review 12

Risk Management Transitioning the easiest and most stable pipeline over

onto new hardware first Migration of OTFR, which is a more complex pipeline, will occur

later If problems arise, we can revert back to ACDSDOPS until

problem resolution This is the case for either transition

ACDSDOPS EMC ACDSDOPS Sun Fire 15K

The pre-archive pipeline will contain as little data as possible (a.k.a draining the pipes)

Again this will be done for both transitions

Implementation Review 13

Schedule: EMC

March 26, 2003: Connect 0.5 TB of EMC space to ACDSDOPS

March 28, 2003: Aliveness Test for ACDSDOPS/EMC

April 7, 2003: Acceptance Test for ACDSDOPS/EMC

April 14, 2003: Deployment Review April 21, 2003: ACDSDOPS/EMC operational

Implementation Review 14

Schedule: Sun Fire 15K