themis fdmo review pre-launch preparations update − 1 october 5, 2004 pre-launch preparations...

THEMIS FDMO Review Pre-Launch Preparations Update − 1 October 5, 2004

Pre-Launch Preparations Update

Mark Lewis


Pre-Launch Preparations Update

Overview• MOC Development• Flight Control Systems• End-to-end Data Flows• Mission Simulations• FOT Training• IT Security


Mission Operations Center

BGS Antenna, Equipment Racks and FOT Workstations at the Mission Operations Center


Expanded MOC


MOC Console Assignment

Mission Operations• THEMIS (OPS 1-9)

• STEREO (OPS 10)

• CHIPS (OPS 11-13)

• RHESSI (OPS 14-17)

• FAST (OPS 18-19)

Flight Dynamics• Maneuver Planning, Orbit & Attitude Determination (FD 1-2)

• Automated Product Generation (FD 3-4)

Mission Planning• Command Load Generation (MPS 1-2)

Spacecraft Emergency Response System• Spacecraft State-of-health Monitoring (SERS 1-2)


MOC Expansion Tasks

MOC Expansion for THEMIS Support• Relocation of Flight Controller Office (Completed)

• Cardkey Entry and Alarm System Extension (Completed)

• NASA Security Audit (Completed for Current Configuration)

• Structural Modification (Doorway & Electrical Outlets Completed)

• Furniture Installation (Completed)

• Relocation of FAST and RHESSI Operations (Completed)

• Ground Station Equipment Upgrades (Almost Completed)

• Purchase of Additional Computer Workstations

• Software Installation and Firewall Upgrade

• Configuration of Operations Consoles

• Initial End-to-end Data Flows Between MOC and Probe I&T Area

• Minimum Impact to Currently Operating Missions

• All To Be Completed by End of CY 2004


Overview of Software Tools

Tool Developer Function Platform CommentsGTDS GSFC Ephemeris Generation, Mission

Design, Orbit DeterminationSolaris In Operation at MOC Already

GMAN GSFC Maneuver Planning Solaris In Operation at MOC Already

FreeFlyer AI Solutions Maneuver Planning Windows XP Alternate Commercial Solution

SatTrack BTS Orbit Analysis, Pass Scheduling, Product Generation, Networking, 3-D Visualization, Ground Station Control

Solaris, Linux In Operation at MOC Already

MSASS GSFC Attitude Determination Solaris, Windows XP

In Operation at MOC Already, Upgrades To Be Provided by GSFC

MTASS GSFC Attitude Determination Solaris, Windows XP

To Be Provided by GSFC

ITOS Hammers Probe Command and Control, Telemetry Monitoring, Limit Checking

Solaris, Linux In Operation at MOC Already

VirtualSat Hammers Probe Simulation Windows XP To Be Supplied with FlatSat

MPS GSFC Command Load Generation Solaris In Operation at MOC Already

BEARS UCB Emergency Response System Solaris Under Development at SSL as Replacement for SERS

BTAPS UCB Probe Bus and Instrument Trend Analysis

Solaris Under Development at SSL for MOC and SOC

APGEN JPL Task and Event Scheduling Solaris In Operation at MOC Already

TeamTrack TeamShare Anomaly Tracking Windows XP Currently Used by Hammers


In-house Software Development

Berkeley Emergency & Anomaly Response System (BEARS)• New In-house Solution as Replacement for SERS

Berkeley Trending and Plotting System (BTAPS)• New In-house Solution for Data Trending and Plotting• Based on MySQL Database• Database Population Via ITOS Telemetry Decommutation• Web Based Interface for Data Access and Plotting• Developed as Student Project

Modification of MPS• Adapt FAST Version of MPS to THEMIS

Additional Scripts for Interfacing Major Tools• Miscellaneous Shell Scripts for Telemetry Data Processing and

Validation• Systems Automation


Workspace Configuration

Several Workspaces• Generic Top Level THEMIS Workspace: themisws

– Holds Common Software and Configuration Files– BEARS, BTAPS, MPS, APGEN – ITOS Pages and Procs Directories Common to All Probes

• Each Probe Will Have a Dedicated Workspace: e.g. themisa– Holds Databases and Configuration Files Specific to Each Probe

– ITOS Telemetry and Command Databases– Configuration Files for BEARS, BTAPS, MPS

– ITOS Pages and Procs Specific to Each Probe– Page Header Files


Workspace Configuration themisws | | ----------------------------------------------------------------------------------------------------- | | | | | | | | | | |cfg Codemgr_wsdata deleted_files idl include itos_resource plmod shellfunc src SunOS.5.9 linux | | | | | | | | backup ------- | ---------------------------------------------------------------- | | | | | | | | | | | SCCS src | archive config docs idlUtil include itos_resource OpsUtil SCCS | | | | | ----------------------------------- | | | | | | | | | | | | | cfgmon pages procs themis[a-f] SCCS | | | | | ----------------------------------- | | | | | | | | | | | | | | | cfgmon dbx pages procs tcvol2 SCCS | | -------------------------------------------------------------------------------------------- | | | | | | | | | | | | | | | | | | | | <itosVersion#> cfgmon pages procs themisa themisb themisc themisd themise themisf | | | | | | | | | | | | | | | | | | ------------- | --------------- | | | | | | | | | | | | | | | | | odb pdb tcvol2 | ground inst scbus | | | ------------------------------- | | | | | | | | | | | | | ground include inst prints scbus | | | --------------------------------------------- | | | | | | | | | | | | | | <itosVersion#> cfgmon odb pages pdb procs tcvol2 | | | | | | ------------- | --------------- | | | | | | | | | | | | | | odb pdb tcvol2 | ground inst scbus | | ------------------------------- | | | | | | | | | | ground include inst prints scbus


Configuration Control

Software Configuration Control• Each Workspace is Under SCCS Configuration Control

– Continue to Use for Software Already Under Configuration Control– ITOS, MPS, APGEN

• Also Use SCCS for Configuration Control of New Tools– BEARS, BTAPS

Probe Configuration Control File• Each Probe Will Have a Configuration File Under

SCCS– Includes Physical Configuration

– Deploy Status of Booms, Fuel Remaining, etc.– Known Hardware Failures or Anomalous Behaviors– Expected Clock Drift

– Software Configuration– Onboard FSW Version and Patch Level– Probe Specific Telemetry and Command Database Versions


Configuration Control

Software Test and Development Environment• Provide Workspace for Off-line Testing

– Demilitarized Zone (DMZ)– Developers Have Their Own Copy of Workspaces Allowing for

Simultaneous Development and Testing• Virtual Network Computing (VNC) Environment

– VNC Protocol Runs on Detached X-Server– Client/Server Environment

• Configuration Control Board (CCB)– Approves Software Before Moving to Secure Operations Net– Approves Hardware Changes to MOC Equipment

• Probe Bus and IDPU TLM/CMD Databases– Probe Bus Configuration Control at Swales until L-60– Transfers to Berkeley at L-60– IDPU Configuration Control at UCB– Master Database Compiled at UCB and Swales


ITOS Requirements

General ITOS Requirements• THEMIS Will Be Using the Commercial Version of ITOS

– Same as I&T System• Similar to FAST and RHESSI

– See ITOS Standard Documentation

Implementation of New Features (Descoped)• Decompression of THEMIS Telemetry Data in Real-time

– May Be Enabled or Disabled, Depending on Instrument Data Type• Desirable Features for Constellation Operations

– Telemetry Server for Distribution of Data Streams to Multiple ITOS Clients

– Status Displays for Telemetry Points Across Multiple Probes– ITOS Telemetry Server


ITOS Configuration

Dedicated Workstations• Basic Configuration Is One Dedicated ITOS System per Probe

– Same Workstation will Always Support Each Probe– Back-up Workstation Used in Case of Problem with Primary

Flexibility Built Into System for Dynamic Allocation of Workstations • Capability to Route Telemetry to Multiple Workstations

– Allows Many Engineers to Monitor Probes During LEO Operations– Command Only Enabled on Primary Workstation

• Multiple Hot Back-up Systems

Upgrade Path To More Complex Future Missions• ITOS Connects to SatTrack Gateway Server (SGS)

– ITOS Specifies Mission (e.g. THEMIS) and Leaves Individual Object and Facility Unspecified

– SGS Assigns Individual ITOS System to a Particular Pass Support– Routing of Telemetry and Command Connections Via

FrameRouter Software Matrix Switch– ITOS in Turn Loads TLM/CMD Databases and Supports Pass


ITOS Page Layout

Standard Features in ITOS Page Template• Uniform Page Header and Page Layout• Probe Identifiers: Probe Bus and Constellation ID• UTC and Probe System Time• Sub-headers, Data Columns, Grouping of Parameters• SI Units: V, A, s, m, kg, C, K, N, Pa, T• Rules for Java Based Graphics


Mission Control Network

Network ArchitectureAll Ground Stations and I&T Systems Connect to Dedicated Ports on Primary IP Router to Establish TLM and CMD Socket Connections

Router Switches TLM and CMD Socket Connections Through to Multiple ITOS Systems

Process Controlled by Automated Scheduling System


Mission Control Network

Secure Network TopologyCheckPoint Firewall at MOC Controls Network Traffic Between Four Secure Subnets:

1. Isolated Operations Network (ION) Hosts ITOS Workstations and Flight Dynamics Systems

2. Berkeley Ground Station (BGS) Subnet Hosts All Ground Station Systems

3. NASA Open/Closed IONet Access Via T-1 Line

4. THEMIS I&T Subnet Via VPN Tunnel Across Open Internet

Firewall Rules Configured to Only Allow Very Restricted Access Between Subnets, Ensuring Proper Security On All Network Segments

ITOS Workstations Configured and Tested During I&T − No Change for On-orbit Operations Required


TLM & CMD Data Routing

Telemetry & Command Routing Scheme• FrameRouter Acts as Server and Provide Matrix Switching

Capability• Incoming Connections from Ground Stations and ITOS Clients• Port Assignment

– Ground Stations: Each Has Dedicated Ports for TLM & CMD– I&T Systems: Probes and FlatSat – ITOS Clients: Can Connect to Any Ground Station by Selecting

Appropriate TLM and CMD Ports– Science Data Tool (SDT): Connects to Secondary Router– Standard Frame Format with EDOS Header for All Data Flows

– I&T End-to-end Tests– Ground Stations

• FrameRouter Establishes Connections Based on Scheduling Information from Gateway Server


Telemetry Data Format

Virtual Channel Priority Data Type Data Source

VC0 0 Real-time Probe State-of-health Data (BAU)

VC1 1 Playback Probe State-of-health Data (BAU)

VC2 2 Real-time Instrument Data (IDPU)

VC3 3 Playback Instrument Data (IDPU)

VC4 N/A Not Used

VC5 N/A Not Used

VC6 4 Playback Probe Bus Events (BAU)

VC7 5 Fill Data Generated Fill Data (CIM)

Telemetry Data Format• CCSDS V1 Frame Format

– Telemetry Data Routed by Virtual Channels• Annotated Channel Access Data Units (CADUs): 1274 Bytes

– Telemetry Frame Delivery Header: 10 bytes(Attached at Ground Station)

– Attached Synchronization Marker: 4 bytes– Virtual Channel Data Unit: 1100 bytes– Reed-Solomon Code Trailer: 160 bytes


Telemetry Data Delivery

Data Routing Transfer Medium Virtual Channels Maximum Data Rate

BGS to MOC Secure Local Area Network(100 Mbps) VC0, VC1, VC2, VC6 1048.576 kbps

NASA/GN to MOCSecure, Shared TCP/IP

Network Connection(T1 Line, 1 Mbps)

VC0, VC2, VC6 65.536 kbps

HBK to MOC Dial-Up ISDN Line (9.6 kbps) VC0 8.192 kbps

WDISC to MOCSecure, Shared TCP/IP

Network Connection(T1 Line, 1 Mbps)

VC0 4.096 kbps

I&T to MOC Secure TCP/IP Connection via VPN

VC0, VC1, VC2, VC3, VC6 Limited by Open Internet

Delivery of Real-time Telemetry Data• TCP/IP Network Socket Connection (BGS/GN/SN)• Dial-up ISDN Line (HBK)• Initiated by Ground Station


Command Data Routing

Command Data Format• CCSDS V1 Frame Format

– Command Data Routed by Virtual Channels– VC1: Commands Decoded and Executed in Software

– BAU Switched Service Commands– IDPU Commands– FSW Commands– Table Load Commands– Data

– VC2: Commands Decoded and Executed in Hardware– Processor Resets and Others

• Command Link Transmission Units (CLTUs): Variable Length– Command Delivery Header: 24 bytes

(Stripped at Ground Station)– Acquisition Sequence: 18 bytes– Start Sequence: 2 bytes– Telecommand Codeblocks: 37 x 8 bytes (Maximum)– Tail Sequence: 8 bytes


Constellation Status Monitoring

SatTrack Interface to ITOS Data Point Server (DPS)• SatTrack DPS Client Program Connects to SatTrack Gateway Server• One Instance of DPS Client Can Handle Multiple Probes

Simultaneously• Receives Pass Schedule Information in Real-time• Connects/Disconnects to/from ITOS Supporting a Pass for a Given

Probe• Connects/Disconnects Can Be Interleaved for Multiple ITOS Systems• Polls Values for List of Mnemonics and Saves Values in Local

Database (One File for Each Probe)– Mnemonic: Status String

• Periodically Generates Web Page with Constellation Status– Top Level Constellation Overview– Contact Schedule Summary– Probe Bus Status– Instrument Status

• Performs Yellow and Red Limit Checking with FOT Notification


Maneuver Command Loads

Maneuver Command Sequence• Maneuver Preparation

– Configure Telemetry Filter Table for Thrust Mode– May Require Several Commands

• First Attitude Maneuver– Catalyst Bed Heater On (for 60 min)– Catalyst Bed Heater Off (Just Prior to Thrusting)– Enable Thrusters– Select Thruster Mode– Set Thrust Phase Angle, Pulse Width and Number of Pulses

– May Be Repeated Up to 10 Times

– Set Thrust Start and Stop Times– May Be Repeated Up to 10 Times

– Disable Thrusters


Maneuver Command Loads

Maneuver Command Sequence• Orbit Maneuver

– Catalyst Bed Heater On (for 60 min)– Catalyst Bed Heater Off (Just Prior to Thrusting)– Enable Thrusters– Select Thruster Mode– Set Thrust Phase Angle, Pulse Width and Number of Pulses

(for Pulse Thrusting of Tangential Thrusters)– Set Thrust Start and Stop Times– Disable Thrusters

• Second Attitude Maneuver– Same as First Attitude Maneuver

• Shutdown– Configure Telemetry Filter Table for Normal Mode

– May Require Several Commands


Software Verification Plan

Tool Heritage Function Method of Verification

ITOS FAST, RHESSI Real-time Telemetry Acquisition Data Flow Tests from Ground Stations and TDRSS

Post-pass Telemetry Playback Testing During I&T

Telemetry Decoding Testing During I&T

Limit Checking Testing During I&T

Real-time Command Transmission

Data Flow Tests to Ground Stations and TDRSS

Real-time Commanding Testing During I&T

Table Load Uplink Testing During I&T

MPS FAST, RHESSI ATS Load Generation Testing at MOC, Testing on FlatSat, Using THEMIS Products

Generation of Integrated Prints Testing at MOC, Using THEMIS Products

BEARS None Limit Violation Checking FlatSat Data, FAST/RHESSI Data

Persistent Paging Two-Way Alpha Numeric Pagers

Anomaly Reports, Web Access Test Using FAST/RHESSI Data,Test During I&T

BTAPS None Data Ingest to Databases Test Using RHESSI Data, I&T

Plot Creation Test Using RHESSI Data, I&T

Web Access Test Using RHESSI Data, I&T


Anomaly Tracking System

TeamTrack• Provided by TeamShare• Used by Hammers to Track ITOS Bugs• May Be Linked to Microsoft Access Database Used by Swales

to Track Spacecraft Problems• Option for Status and Anomaly Tracking at Berkeley MOC

– UCB Team Already Has Two Accounts


Pre-launch Operations

Pre-launch Operations• Procedure Testing and Validation

– Launch & Early Orbit, Normal Operations and Contingencies• End-to-End Data Flow Tests with All Ground System Elements

– Ground Stations, TDRSS, FlatSat and Flight Dynamics to MOC• Launch and Mission Simulations with Green Card Exercises

– Practice of Contingency Detection and Recovery• Dress Rehearsals

– Multiple Cycles of Launch Readiness Testing

FOT Training• Participation as Test Conductors in Mission I&T• Extensive Flight Simulator Training• Training on Usage of Various Software Tools• Classroom Sessions Given by Subsystems Engineers• Team Member Certification


Ground System Testing

Remote Access to I&T Facilities• Passive Remote Monitoring• Active Remote Operation• Procedure Development and Verification• Simulations• End-to-end Data Flow Testing


End-to-End Data Flows

SwalesI&T Facility

Berkeley Mission Operations Center

White SandsComplex(WDISC)

Compatibility Test Van

GSFC NMC

Wallops Ground Station

Different Routes for End-to-End Data Flows Between Probes and GDS Elements

BerkeleyGround Station

THEMISProbe

SecureIntranet

TDRS

Secure VPN Tunnel Via Open Internet

ClosedIONet

Restricted IONet

Hartebeesthoek Ground Station

Dial-up ISDN Line

Santiago Ground Station

Merritt IslandGround Station


FlatSat Environment

FlatSat• Swales Building Mission Simulator (FlatSat)

– ColdFire CPU– Communications Card– Power Card– Load/Dev PC– VirtualSat (Spacecraft Dynamics and IDPU)– ACS Sensors/Actuator

• Remote Access Available for SSL– Primarily After Swales Day Shift

• Used Extensively for Testing Flight Software and Operations• FlatSat Will Move to SSL After Completion of I&T

– Facilitate Launch and Contingency Simulations– Post-launch Testing of Flight Software Patches and New

Procedures– On-going FOT Proficiency Testing


FlatSat Environment

FlatSat

* COTS for A/D-D/A conversion and IDPU I/F

Ethernet

ITOSS/C Command and Telemetry

Realtime Control

ITOSS/C Command and Telemetry

Realtime Control

ACSActuator/Sensor I/O*

ACSActuator/Sensor I/O*

Hard-lineUp-Down link

Berkley Ground Data System

Connected ViaSecure InternetSerial I/F

Serial I/F Serial I/F

SpacecraftProcessorColdFire-5802

512KB E2PROM 512KB RAM

SpacecraftProcessorColdFire-5802

512KB E2PROM 512KB RAM

COMMCard

COMMCardPower

Card

PowerCard

Load/DevPCLoad/DevPC

Bus Avionics Unit (BAU)

VirtualSatPRO

(S/C DynamicsIDPU Simulation)

VirtualSatPRO

(S/C DynamicsIDPU Simulation)

1-Hz Pulse

Digital & Analog

Interfaces


Green Card Exercises

Examples of Green Card Exercises• Launch Related Anomalies

– No Separation from Probe Carrier– High Probability of Re-contact Due to Off-nominal Probe

Separation• Communications

– Negative Telemetry Acquisition Via Ground Station or TDRSS– Multiple Probes Radiating, Causing Interference at Ground

Station– Loss of TDRSS Contact– Command Uplink Carrier Down

• Command and Data Handling– Commands Not Verified

• State-of-health Limit Violations– Battery State-of-charge High or Low– Over-current or Under-voltage Conditions Detected– Temperature Sensors Indicate High or Low Limit Violation


Green Card Exercises – Cont.

Examples of Green Card Exercises - Continued• Processors, Memory and Flight Software

– BAU or IDPU Reset– SSR Full at Beginning of Contact

• Attitude Control System– Probe Attitude Off-nominal – Probe Spin Rate Out of Limits

• Reaction Control System– Only One Latch Valve Can Be Opened– Thruster Appears to Be Leaking– Pressure Transducer Fails

• Maneuver Execution– Maneuver Sequencing Not Executed On-board– Attitude Maneuver Halted by FDC Because of Detection of High

Angular Rates– Thruster Pulse Firing Sequence Indicates Incorrect Phase Angle– Real-time Doppler Data Deviate from Predicted Track



Examples of Green Card Exercises - Continued• Instrument Power-on

– Unsuccessful IDPU or Instrument Power-on– High-voltage Arcing in ESA or SST

• Boom Deploy– Unsuccessful Magnetometer Boom Deploy– Radial or Axial Boom Deploy Procedure Not Executed On-board– Radial Boom Deploys Partially Only

• Constellation Related Problems– Early Orbit Timeline Halted Because of Probe Anomaly– Probe Anomaly Requires Re-scheduling of Pass Sequence– FSW Bug Detected on One Probe with Potential to Occur on Other

Probes– Simultaneous Anomaly on Two Probes

• Space Environment– Major Solar Storm – Potentially Affecting All Probes



Examples of Green Card Exercises - Continued• Ground Systems

– Ground Station Down– Pass Missed Because of Network Outage– Command Workstation Failure During Contact

• Flight Dynamics– Misidentification of Probes After Separation– Critical Ground Station Does Not Deliver Doppler Tracking Data– Two-way Doppler Data from Ground Station Are Noisy– Orbit Determination Run Does Not Converge– Orbit Determination Shows Bias Due to Wrong Site Coordinates– Orbit Determination Yields Large Discrepancy Between Previous

and New State Vectors– Attitude Determination Provides Erroneous Results

• Staff Related Problems– ACS Engineer Missing During Maneuver Contact


Network Plan & IT Security

NASA Network Security Requirements Driven by NPG 2810.1, Security of Information Technology• GSFC IT Security Outlined in Document 290-004,

IONet Access Protection Policy and Requirements• Establishes IT Network Systems Security Measures and

Controls for Access to Critical NASA Resources

Documentation Requirements• Information Technology Security, Risk Management and

Contingency Plans

Closed IONet Security Compliance• Required for TDRSS Access• New Option Is Restricted IONet Between WDISC and MOC• Configuration Control by System Administrator• Network Certification and Scanning• Personnel Screening

themis fdmo review pre-launch preparations update − 1 october 5, 2004 pre-launch preparations...

Documents