Bringing the Vision of Plug-and-play to High-Performance Computing on Orbit

Presentation to HPEC 200922 Sept 2009

Outline

• Introduction• Space Plug-and-Play Avionics (SPA)• Extending SPA to HPEC• Conclusions

Space PnP Avionics (SPA)

• Introduction• Key Features• Status

driver

USB interface chip

plug-and-playcomponent

“platform”

electronicdatasheetcomponent

Ap

pli

qu

éS

enso

r In

terf

ace

Mo

du

le

(AS

IM)

interface module

“platform”

plug-and-playcomponent

Analogy of Consumer PnP with SPA

Space Plug-and-play AvionicsKey Features

• Single-point interfaces (e.g. SPA-S) and protocols

• Appliqué Sensor Interface Module (ASIM)• Electronic datasheets (XTEDS)• Software -- Satellite data model (SDM)• Test bypass• Pushbutton toolflow

Interfaces

• SPA-U (Data transport = USB 1.1, limited to 12 Mbps for entire bus)

• SPA-S (Data transport = Spacewire, limited to 600 Mbps per direction per link)

SPA-device

Data transport (commands, config, data)

Power (two pins) ( 4.5A(L) or 30A(U) )n

Devices with basic interfaces

Sync (two pins) (RS-422, 5V )*

primary interface

test bypass interface (TBI)

2

2

Very low data rate (< 10 kilobit/sec)“SPA-1” (future)

low data rate (< 1 Mbit/sec)SPA-U

high data rate (< 620 Mbit/sec)

SPA-S

Very high data rate “SPA-Optical”

Number of components

Perf

orm

ance

of c

ompo

nent

s

Distribution of bandwidth in systems

SPA-x Network

SPA-y Network

Bridge Node

Heterogeneity – Mixture of SPA networks

XML-basedElectronic

Data Sheet(xTEDS)

XML-basedElectronic

Data Sheet(xTEDS)

Your

Device

here Ap

pliq

ué

Sen

sor

Inte

rfac

e M

odul

e (A

SIM

)

8031

mem

ory

map

Processor(ex. 8031)

Bypass storage

Non-volatile memory: (XTEDS) Time

synchronizationstate machine

Test bypass engine state

machine

Power mgt

x-interface (ex. “U” =

USB)

Dig

ital U

ser

in/o

utp

ut

Analo

g

Use

r in

put

Analo

g

Use

r outp

ut

Pow

er

Use

r outp

ut

Mis

c.U

ser

in/o

utp

ut

Test bypass

SPA-x

Non-volatile memory:

program/data

RAM memory

8031

mem

ory

map

Processor(ex. 8031)

Bypass storage

Non-volatile memory: (XTEDS) Time

synchronizationstate machine

Test bypass engine state

machine

Power mgt

x-interface (ex. “U” =

USB)

Dig

ital U

ser

in/o

utp

ut

Analo

g

Use

r in

put

Analo

g

Use

r outp

ut

Pow

er

Use

r outp

ut

Mis

c.U

ser

in/o

utp

ut

Test bypass

SPA-x

Non-volatile memory:

program/data

RAM memory

Applique Sensor Interface Module (ASIM) – Simplifying SPA Engineering and SPA Compliance

eXtended Transducer Electronic Datasheet (xTEDS)

• Primary mechanism for self-description– Embedded in hardware and software

applications– Describes “knobs” and “measurands”

• Conveys “semantic precision” through a common data dictionary (CDD)

• Enforces order in the “LEGO universe” of SPA (features only exist if known through XTEDS)

• Recently released to public domain– Studied as possible AIAA and ISO

standard

XTEDS

(facet)Interface

(facet)Interface

Message

Variable

Message

Variable

CDD

CameraThermometer

GNC CompCurrentMonitor

RF

Application#1

Application#2

Application#N

Mission Code / Scripts

Application#i

Sensor Manager (SM) SM SMSM

CPU

ProcessorManager

Task Manager Data Manager

Sate

llit

e D

ata

Mo

del

The Satellite Data Model (SDM) – Building Awareness into Plug-and-play

A/D

pre-amp / filter xTEDS

embeddedprocessor

SP

A i

nte

rfa

ce

data source

testbypass interface

normal

normalbypass

Applique sensor interface module

SPA (plug-and-play) thermometer

Test Bypass – Automating Support for Hardware-in-the-Loop

Push-button Tool Flow(aka Satellite Design Automation)

Component Icons

Connections

Drag & Drop Design

Mission Goals and Requirements Component Capabilities

************************************************************************** CATEGORY RULES **************************************************************************predCategory( catidReferenceFrame ).predElementOf( catidReferenceFrame, catidReferenceFrame ).

predCategory( catidCoordinateSystem ).predElementOf( catidCoordinateSystem, catidCoordinateSystem ).

************************************************************************** INTERFACE RULES **************************************************************************

predInterface( iidIEnvironmentObject ).predElementOf( iidIEnvironmentObject, catidEnvironment ).

predInterface( iidIMomentumStorage ).predElementOf( iidIMomentumStorage, catidActuator ).

************************************************************************** COMPONENT RULES **************************************************************************predComponent( clsidCEarth ).predElementOf( clsidCEarth, catidReferenceFrame ).predElementOf( clsidCEarth, catidEnvironment ).fncIn( iidIEnvironmentObject, clsidCEarth ).

Des

ign

Verifi

catio

n Ru

les

Engi

ne

Automatic

Verification

Performance Modeling

Itera

te

1.

MISSIONCAPTURE

2.

SPACE-CRAFT

PROFILER

3.

AUTO-GENERATE

“EVERYTHING”

4.

COMPARESIM VS. THE

ORIGINALMISSION

Other Tenets of SPA

• Seek OS independence• Seek decentralization• Seek to conceal (unnecessary) complexity

through encapsulation

SPA Status• SPA Workshops (eight from 2004-2006)• Creation of Responsive Space Testbed (Kirtland AFB)• Flight developments

– RESE (SPA-U, 4-port) – Launched and operated September 2007

– TacSat 3 (SPA-U, 4-port) – Integration into TacSat 3 (Launched in 2009)

• Adoption of SPA as central interface approach for TacSat 5

• Creation of outreach concepts for SPA-based CubeSats

• International agreement (with Sweden) and pursuit of national/international standards for SPA

Plug-and-play Satellite (PnPSat)• First spacecraft ever built

entirely on PnP principles– Decentralized, scalable

computation– Use of satellite data model– All components (even panels)

are SPA devices– up to 48 mounting sites

• Ambitious development schedule– Targeting flight in 2009

Reaction Wheel and Electronics (3)

Torque Rod (3)

Coarse Sun Sensor Module(2)

Battery Assembly (2)Magnetometer

Transceiver and Comsec

Primary Experiment (Example Only)

HPCOO (2)

Charge Control Electronics

Component and Experiment Accommodations

• A full complement of PnPSat components shown

– By recessing electrical infrastructure and harnessing, we significantly increase flexibility for component and experiment mounting

– Initial version of PnPSat may have fewer spacecraft components than the version shown

Solar Array

Encapsulation (complexity hiding)

Hub

SpW

HWIL

HCB

HCB

HCB

Hub

SpW

HWIL

HCB

HCB

HCB

(a) (b)

(c)

Encapsulation (complexity hiding)

Miniaturization CubeFlow = SPA+CubeSat

• Targeting PnP platforms as small as cubesats (100mm)

• Supports increased payload mass fraction and creation of PnP nanosatellites

• Compact nanosat modular form factor (NMF)standard (70mm x 70mmx12.5mm)

CubeFlow Training

• “Eli Whitney meets spacecraft”

• Short course based on the principles of SPA embedded in take-apart Cubesats– Entire system (with

laptop console) fits in briefcase

– Fifteen+ kits distributed so far (May 2009 course)

– More CubeFlow courses planned

SPA for high-performance embedded systems?

• Scaling of SPA interfaces currently limited• Complex processing architectures far

from plug-and-play

Example Processing Chain Framework for high-performance (surveillance) sensor

Sensor –Pixels, # rows, # cols, # frames/sec, modes (windowing)

Analog processing / conversion

TDP ODP MDP

Digital Processing

Processes that are done on every single pixel, intensive processing, but usually relatively simple. generates objects

Processes that are done on every single object. Much reduced data rate compared to raw data. Generates enhance objects

Processes that are done on enhanced objects. Much reduced data rates, but much greater # ops / enh.object

Back-end processingFront-end processing

Generic Processing System

ASIC ASIC

ODPDSP

MDP

TDP ODP

Sens

or d

ata

ASIC

ASICODPDSP

ODPDSP

ODPDSP

ODPDSP

Example 1: TacSat 2 Processing System

FPGA

FPGA

FPGA

FPGA

FPGA

ASIC

Mem

ODPVLIW

MDPRAD750

TDP ODP

Sens

or d

ata

Example 2: Sensor And Fusion Engine (SAFE) Processing System

FPGA FPGA

ODPVLIW

MDP

TDP

ODP

Sens

or d

ata

MDPLi

nk

FPGA FPGA

TDP

MDP

Link

ODPVLIW

ODPVLIWODP

VLIW

ODPVLIW

ODP 1-12(WSSP)

MDP

Problems With Ad Hoc HPEC Frameworks

• Constant reinvention of reconfigurable computation architectures

• Fragile, proprietary link structures• Difficult migration across heterogenous

partitions

How could SPA concepts be applied?

Avoiding the “yet another reconfigurable computer” syndrome

• Nodes based on single computation device• Ok to have heterogeneous node composition• Regular socket and messaging infrastructure• Not ok to have disparate

socket/interface/messaging infrastructure• Pray for the existence of adequate tools to

handle amortizing code (circuitize-able) into the fabric of distributed nodes

• Use SPA-like ideas to manage the whole thing

(a)

(b)

(c)

MPP Platform to study high-bisection bandwidth reconfigurable computing architectures

Conceptual “HPEC SPA” network without optical (multiple ports/device)

SPA Device SPARouter

SPA Device

SPA Device

SPA DeviceSPA

Router

SPA DeviceHardware in loop interface

HWILS

Conceptual HPEC-SPA network based on optical transport

SPA Device SPARouter

SPA Device

SPA Device

SPA DeviceSPA

Router

SPA DeviceHardware in loop interface

HWILS

Idealized optical backplane

SPA-Optical “exec summary”• Also referred to as SPA-10 (original Gbps target, just a

label now)• Expect to have properties similar to (nonscalable)

SPA-S, but higher link speed– Use of embedded clock recovery

• Desire to support optical physical layer for data, command, synchronization– Allows >Tbps scaling through WDM– Allows flexibility in “provisioning” (i.e. assigning particular

wavelengths, protocols, to particular SPA-10 ports)– Allows greater flexibility in managing topology, routing

policies, faults

SPA-10 Device concepts

SPA-10 Device

Sensor(camera, radar,

comm, etc.)

Mass storage

Processing node

RAW Device TypesInterface schemesFixed wavelength

Tuned wavelength

WDM within single device

SPA-10 Possible Interface DetailsHybrid E/O

SerDes

config

power

sync

uP

XTEDS

Optically Enhanced ASIM (OASIM)

Breakout I/O

RAWDEVICE

Hi BW in

Hi BW out

OAS

IMRAWDEVICE

SPA-10 Device

SPA Computation

• Addressing interconnection bottleneck leaves the problem of efficiently mapping computation problems to resources

Complex (multi-FPGA board) Circuit Representation

Partition into Unit-sized Portions

A

B

C

D

Insertion of Socketing Infrastructure

B

A

D

C

Wavelength Assignments to Sockets

B

A

D

C

Transferral to Idealized Backplane

B

A D C

Idealized Optical Backplane

SPA-10 Modules

A B C

C

Idealized Optical Backplane

(Wavelength multiplexing drawn as spatial multiplexing for illustrative purposes)

Idealized Optical Backplane

Colorless optical transport

SPA-10 SPA-10 SPA-10 SPA-10

Idealized (vs. practical) optical backplanes

• Idealized: as described in Gilder’s Telecosm– Infinite resource, every actor has own wavelength

• Practical: limited by finite resources and protocol barriers– Limited number of physical channels (fibers)– Limited number of wavelengths (CWDM,DWDM)– Differing channel characteristics (transceiver data rates,

single-vs-multi-mode, transceiver spectral characteristics)– Time-slotting (time-division multiple access)– Protocol assignment (matching disparate OSI stacks)– Limitations of optical resources (e.g., outages due to time

necessary to implement switch re-assignments)

Practical implementation

SPA-10 SPA-10 SPA-10 SPA-10

How to “LEGO-ize” Anything(generalization of plug-and-play)

Payload

ASIMxTEDS Config.

Transport portal

Controlportal

Transport portal

Core XTEDS

extension

extension

Source: http://www.kasahara.elec.waseda.ac.jp/schedule/

Challenges in “plug-and-play” provisioning

• Mapping algorithms into a variety of node types– FPGA-based– Single/multicore processors

• Coordinating socketing – Messaging protocol– Establishing finite fabric

resource allocation effectively with tolerable gaps in time due to transitions in provisioned configurations

Advent of Megacompilers?

Mega-compiler

Problem represented in neutral format

SPA-10

SPA-10

SPA-10

SPA-10

SPA-10

SPA-x Network

SPA-y Network

Bridge Node

Awareness of target architecture constraints

Awareness of network topology constraints

Number and type of target objects

Bitstreams for each target object

Topology for connecting objects

Provisioning constraints

In-house SPA-O R&D Testbed (plan)

Optical Switch/Router

1st High Data Rate Sensor

Large Memory Storage

On Board Processor

External PRBS Data In

2nd High Data Rate Sensor

Optical TX/RX

SPA S Connector

Optical Signal

Electrical Signal

SDM/xTEDS/Apps

Router control

High Speed Scope

Summary

• Space plug-and-play (SPA) continues to gain momentum (completion of PnPSat 1, start of PnPSat 2, TacSat 5, ORS Chileworks, CubeFlow, standardization)

• SPA-Optical / SPA-10 represents a collection of concepts to extend SPA to high-performance embedded computation

• Early work on SPA-Optical testbed underway at AFRL (Kirtland AFB)