tempo instrument operations center

TEMPO InstrumentOperations Center

John E. DavisJohn C. Houck

Ewan O’SullivanRaid M. Suleiman

Smithsonian AstrophysicalObservatory

OutlineOutline

� Overview of the TEMPO ground systems

� IOC design considerations

� Commanding and command planning

� Telemetry Processing

� Summary

2018 June 6-7 TEMPO Science Team Meeting, Boulder, CO 2/21

TEMPO Ground SystemsTEMPO Ground Systems


IOC Driving RequirementsIOC Driving Requirements

� Autonomous processing of raw instrument data to L0 in real time

� Autonomous monitoring of TEMPO instrument health and statusdata in real time and alert operators of any out of limit conditions

� Provide IOC personnel with remote access to the IOC data

� Command generation and validation

� Tools for instrument operations planning


IOC ArchitectureIOC Architecture

SOC

TEMPO

SOCUnwrap

4.1.1-2Ephemeris & Raw Telemetry

Raw to L0Processor

4.1.3

4.4.1-2

FeedCOSMOS

Commandgeneration,validation, anduplink

SOC Wrap

4.1.1

COSMOS

4.2.1-2,4

4.6.1 4.7.1-4

AlertSystem

4.2.3

H&SExtract

4.2.1-2

LVTH&S

Archive

4.2.6 4.10.1

AlertTable

Trending

4.2.5

Reports

4.3.1

LogFiles

InstrumentSimulator

ScienceExtract

4.1.4

SDPC

L0 ScienceMonitor

4.2.4

L0 ScienceCache

RemoteMonitor

4.2.4,7 4.5.1

Scan tailoring/planning

L0Monitor

4.4.1

Commands & Telemetry


Commanding OverviewCommanding Overview

� The instrument will operate out of a 14 day command sequence,with a new command sequence uploaded weekly

� The command sequence will include daily Earth scans andcalibration activities

• Earth scan sequences will be based upon the SDPC-generatedscan planning and scan tailoring tables

• May include special observations

� Commands sequences will be validated with the help of aBall-supplied hardware instrument simulator

� Interaction with the simulator will use the Ball Aerospace opensource COSMOS command and telemetry system<http://www.cosmosrb.com>

� COSMOS will also be used for short-term telemetry monitoring


Command TypesCommand Types

� Atomic Commands

• Lowest level of commanding provided by the flight software(described by the command and telemetry handbook)

� Command Load

• A sequence of CCSDS command packets containing oneatomic command per packet

• Transferred to the SOC for uplink per the IOC-SOC ICD aftervalidation by the IOC using the TEMPO simulator

• Each atomic command is executed upon receipt by the flightsoftware after the command has been validated for the currentmode (Safe or Operate)

• Command loads consist primarily of flight-software (FSW) tableuploads using the TBL_* atomic commands


Flight Software TablesFlight Software Tables

The TEMPO instrument is largely table driven. The most importanttables include the command block and command sequence tables, aswell as the mirror scan tables that control the position of the steppingmirror within a scan.

� Mirror Scan Tables

• Contains up to 4096 (∆x,∆y) offsets that control the mirrorposition from one step to the next

• The TEMPO instrument can store up to 4 scan tables• Multiple tables may be used for optimized scanning, or special

observations


Command Block MacrosCommand Block Macros

� Command Block Macros (CBM)

• Set of time-ordered atomic commands that, as a whole, carryout some task, e.g., Safing the instrument, or performing asingle Earth scan

• Each command in the CBM has an execution start time-tag thatis relative to the execution start time of the CBM

• The CBM is invoked using the CMD_BLOCK_EXEC atomiccommand

• Only one CBM may be active at a time (CMD_BLOCK_EXEC isnot permitted in a CBM)

• Each CBM will be validated on the TEMPO simulator priorbefore being uplinked to the instrument


Command SequencesCommand Sequences

� Command Sequence Macro (CSM)

• Similar to a CBM, except it uses absolute time-tags• May invoke CBMs via the CMD_BLOCK_EXEC atomic command• Started/Stopped with the CMD_SEQ_START/CMD_SEQ_STOP

commands• The FSW supports two such tables (a primary and a

secondary), which will be used in a “ping-pong” fashion• The standard concept of operations is for the instrument to

operate out of a 14 day CSM, with a new CSM uploaded weekly• Each CSM will be validated prior to being sent to the SOC for

uplink


Example Daily TimelineExample Daily Timeline

(Figure extracted from TEMPO Concept of Operations DRD SE-13)


Weekly ConopsWeekly Conops


IOC ArchitectureIOC Architecture

SOC

TEMPO

SOCUnwrap

4.1.1-2Ephemeris & Raw Telemetry

Raw to L0Processor

4.1.3

4.4.1-2

FeedCOSMOS

Commandgeneration,validation, anduplink

SOC Wrap

4.1.1

COSMOS

4.2.1-2,4

4.6.1 4.7.1-4

AlertSystem

4.2.3

H&SExtract

4.2.1-2

LVTH&S

Archive

4.2.6 4.10.1

AlertTable

Trending

4.2.5

Reports

4.3.1

LogFiles

InstrumentSimulator

ScienceExtract

4.1.4

SDPC

L0 ScienceMonitor

4.2.4

L0 ScienceCache

RemoteMonitor

4.2.4,7 4.5.1

Scan tailoring/planning

L0Monitor

4.4.1

Commands & Telemetry


Telemetry Processing Code ModulesTelemetry Processing Code Modules

� Raw to L0 Processor

• Receives packets from the Host Ground System in the form ofCCSDS (Consultative Committee for Space Data Systems)packets embedded inside GRDDP (GOES-R Reliable DataDelivery Protocol packets)

• Extracts the CCSDS packets from the GRDDP packet stream• Removes duplicates and time-orders the CCSDS packets• Multi-threaded, uses a priority queue and a memory-mapped

ring buffer

� H&S Extract

• Extracts telemetry point values from the CCSDS packet stream• Converts digital numbers to engineering units• Performs limit-checks and generates out-of-limit messages• Archives the raw values in the IOC H&S Archive• Derived parameter support: dk = fk(a0, a1, . . .)



� Alert System

• Listens for out-of-limit messages from H&S Extract• Sends alert emails and text messages to desiginated personnel

and manages responses• Maintains an alert state table for the Remote Monitor

� Science Extract

• Decodes packets containing CCD and other telemetry itemsrequired by the SDPC for science processing

• Produces L0 exposure records (reconstructed raw CCD framesplus metadata), Inertial Reference Unit and Scan MechanismController data files for the SDPC



� Trending

• Generates a PDF trending report• Time-series plots, plots with multiple telemetry points (TPs)• Performs trend-line analysis for each TP• Durbin-Watson analysis of fit residuals• Can run from a cron job for autonomous trending• Daily trending during commissioning• Transition to weekly trending as the mission progresses


Web-based Telemetry MonitoringWeb-based Telemetry Monitoring

� Remote Monitor

• Web-based system that runs in a javascript-enabled browser• Real time display of TP values and out-of-limit conditions• Interactive plotting of all TPs for any selected time interval• Real time display of various packet statistics (packets received,

CRC errors, sequence errors, etc)• Real time display of raw CCD images• Provide remote access to trending reports, processing log files,

and TP data


Remote MonitorRemote Monitor


APID MonitorAPID Monitor


IOC Software DevelopmentIOC Software Development

� Much of it is new development with an emphasis on libraries forcode-reuse

� Written primarily in C (other languages: PHP, Javascript, S-Lang,Ruby)

� Software developed for 64-bit Linux (CentOS 7 x86_64)

� Iterative software development methodology

� Classified as NASA Class C

� GIT is used for revision control and Redmine is used for trackingissues

� The nightly builds run all unit tests and generate code coveragemaps


Summary and StatusSummary and Status

� Version 1 was completed last year

� Version 2 phase 1 (telemetry) will be tested using instrument datalater this summer

� The instrument simulator will be delivered to SAO in November of2018

� Version 2 phase 2 (commanding) development and testing with thesimulator will start after the simulator has been delivered

� System level testing in will begin in June of 2019

� SOC interface development will begin after a host has beenselected and the IOC-SOC ICD has been made available


tempo instrument operations center

Documents