Data Analysis Plan

GBT IDL Workshop

October 15, 2004

Meeting Outline

• Morning activities– Generate a list of discussion items

• Demonstrations

• Lunch

• Discussion forum

• Dinner

• Summary

Science Drivers

• Interactive package for basic single-dish analysis• Available at NRAO and home institutions• Import/Export data formats• Well-documented data model (data structure and data

flow)• Real-time data checks / post-processing• Handle our diverse observing modes, frequencies,

backends, ….• Handle new instruments and analysis techniques

– Pipelining, cameras, etc.

Expected Customers

• Black belts– Unique experiments– Explore every step in the analysis– Can develop new algorithms– Can share these algorithms

• Novices– Use canned procedures developed by others– Pipelining

Immediate Goals of the Data-Handling Project

• Interactive package for BASIC analysis• Leverage observer’s past experiences• Allow observer’s to contribute to the

development of enhancements• Sharing of contributions• Short time scale for a significant improvement• Limited goals for what NRAO contributes• Rely on other systems for other analysis needs

Options

• Resurrect UniPOPS (argh!)

• Reinvest in AIPS++/DISH or AIPS

• Go with a community standard– Invest in CLASS, IRAF, …– Invest in a commercial package

• IDL• MATLAB• Mathematica

Why IDL? – Resources/Familiarity

• IDL brings…– Large user base– Extensive built-in libraries, algorithms, graphics– Rapid development environment

• External users– Tom Bania – 3He package– Robishaw – Polarization– Kondratko – H2O Spectroscopy

• Other observatories– FUSE, Spitzer, ….– IDL Astronomy User’s Library– Arecibo’s IDL routines

• Internal resources– Langston – IDL Real-Time data display– O’Neil – IDL experience

Why a UniPOPS philosophy?

• Familiarity• Simple syntax

– Interactive, efficient, ….

• Emphasizes a ‘toolbox’ of single-purpose analysis steps– Exploratory analysis, new algorithms

• Scripting for generating complex analysis from these simple commands– User extendable

• Data export to move data to other systems for more complicated analysis

• Online and Offline analysis

Extent of the IDL Project

• Won’t Include (initially):– Advanced imaging– Pulsar, Radar, ….– Adding VLA data to GBT data– Bolometer cameras– Pipelining– ….

• Will include: Infrastructure• First priorities

– Data Model– I/O, Data Base– Toolbox of commands available from two different

user levels

Available Resources & Timeline

• Details from Nicole

IDL – An integral part of the observatory’s plans

• CLASS / AIPS– FITS for data import/export

• ALMA– Resource sharing -- Significant common infrastructure

• E2E – Data pipelining using AIPS++ technology– Learn from our IDL experiences and IDL Data models.

• OBIT– Prototype analysis for the Penn Array

• AIPS++/DISH– Based on appropriate data model from IDL experience, OBIT,

and ALMA– Python instead of GLISH, CASA– Combining VLA data with GBT data– Advanced GBT imaging