OVSA ExpansionSoftware Overview

Gordon Hurford

Kickoff Meeting NJIT 25-Oct-2010

Monitor and Control Operator

Data Packaging Processor

DSPU Antennas, receivers, analog subsystems

Interim Data Base

Data selection, averaging, reformatting & calibration

Burst identification

Archive Databases

~6 GB/day

Users

Quick look & metadata products

Map generation

Tohban

Map display

Light curve, spectra generation

NJIT

GH/JM+NJI

T

OVSA Expansion Software and Data Handling23-Oct-10

OVSA-specific

CASPER-based

Miriad-based

RHESSI-based

Light curve, spectra display

IDL Shell Browser

Hi-Level Analysis

Monitor and Control Operator

Data Packaging Processor

DSPU Antennas, receivers, analog subsystems

Interim Data Base

Data selection, averaging, reformatting & calibration

Burst identification

Archive Databases

~6 GB/day

Users

Quick look & metadata products

Map generation

Tohban

Map display

Light curve, spectra generation

NJIT

GH/JM+NJI

T

OVSA Expansion Software and Data Handling23-Oct-10

OVSA-specific

CASPER-based

Miriad-based

RHESSI-based

Light curve, spectra display

IDL Shell Browser

Hi-Level Analysis

DPP Time-Multiplex Architecture

Correlator

Frequency averagingRFI excision

Frequency averagingRFI excision

Time-independent calibration & Formatting

Parallel outputs on separate networks

Correlator cycles addresses

Interim Database

+ Scaleable

+ Decouples correlator & DPP design issues

+ Can trade hardware for code optimization

Digital Packaging Processor

~450 MB/s

~1 MB/s

~1 MB/s

40 GB/day

OVSA Software Task OrganizationDGAdvisory Committee

Data Base Management

JM

Data Analysis Software

GH

DPP definition GH

Hardware-embedded SW

NJIT

Array control & real time display

NJITRoutine

Calibration & Analysis

GH

Non-solar Analysis

CIT

Special calibration Analysis NJIT

DPP implementation

NJIT

Housekeeping data NJIT

QL / metadata

JM

Pipelined database creation

JM

IDL/Miriad shell

JM

User interface & displays

JM

Miriad Scripting

JMStephen White

Testing

NJIT

Assisted by NJIT

11-Feb-2010

Implementation Philosophy (1)

• Implementation is VERY manpower-limited Prioritization is vitalMaximize use of existing packages

Adaptation of RHESSI IDL-based user interface, display & database systems

Miriad analysis package

Implementation Philosophy (2)

• Highest priority goals:

– To have documented software enabling external users to conveniently do some science with OVSA observations by Sept 30, 2013

– To have basic software tools in place to support hardware development

Implementation Philosophy (3)

• Highest priority goal:To have documented software enabling external users to

conveniently do some science with OVSA observations by Sept 30, 2013

• Examples of lower priority goals:– Fine-scale RFI excision– Processing speed– Ability to analyze special cases or compromised events– Implementation of calibration refinements (e.g. polarization) – Integration of high-level analysis tools– Limited support for non-solar observations

Interim Software Milestones (1)

• Phase 1: 15 months - Dec 31, 2011

• Objective:

Primitive end-to-end capability to support hardware development & enable demonstration science– Place-holder (pass-through) DPP – Offline processing into Miriad-compatible format– Testing with legacy data– Primitive data base to support test data– Miriad IDL shell – basic development – Some support for calibration analysis– Miriad scripts for calibration and mapping– Limited-feature interface using RHESSI GUI

Interim Software Milestones (2)

• Phase 2: 15 months - Mar 31, 2013

• Objective:

Fairly complete but manually-oriented analysis package– DPP supports real-time data packaging and pre-

calibration without RFI excision– Improved data base provisions– Improved calibration analysis, application and mapping

software– Improved user interfacing.– Preliminary user documentation

Interim Software Milestones (3)• Phase 3: 6 months - Sept 30, 2013

• Objective: Fully-featured with automated data processing & quick-look generation.– DPP support for RFI excision and flexible frequency averaging– Flare identification and application database generation – Scripts to generate quick look / catalog data.– Scripts to automate data management– Additional bells and whistles.– Basic support for non-solar observations– Improved user documentation..

Next Steps

• Consensus on overall approach to OVSA software

• Consensus on calibration strategy

• Coordination between:– Hardware and software development schedule / needs– NSF- and NASA-funded tasks– OVSA and FASR software

• Early Definition of selected interfaces – Correlator DPP – Offline calibration input DPP– Housekeeping DPP– Interim database format

extras

DPP Architecture Options

Frequency Averaging

Baseline Processing

Correlator

Frequency Averaging

By polarization

Correlator

Frequency Averaging

Frequency Averaging

Baseline Processing

By baseline

Correlator

Frequency Averaging

Frequency Averaging

Baseline Processing

switched

- Not scaleable

- Well suited to correlator processing?

- Prevents polarization calibration by subchannel

+ Scaleable

-Couples correlator software and/or hardware to DPP speed

- Discourages use of ‘canary’ antennas for RFI identification

+ Fully scaleable

+ Decouples correlator and DPP performance/design

Need speed tests and correlator architecture input