Multidimensional Data in the

Virtual Observatory

Jose Enrique Ruiz del Mazo

Tutora: Dr. Lourdes Verdes-Montenegro

IAA - CSIC

Máster FISYMAT

Trabajo de Investigación Tutelada

Universidad de Granada

Diciembre 2010

Context

• The AMIGA project

• The Virtual Observatory

• Multidimensional Data in Astronomy

• AMIGA VO Contributions

Generic Datasets

• Data Discovery

• Data Access

Generic Dataset Discovery Service

• Input Parameters

• Query Response

• Implementation

Conclusions and Future Work

SUMMARY

AMIGA

Analysis of the interstellar Medium of Isolated GAlaxies

PI : Dr. Lourdes Verdes-Montenegro

IAA-CSIC, IRAM

http://amiga.iaa.es

Obs. Marseille, Obs. Paris, CfA, ASIAA, MPIfA, IAC,

Univ. Alabama, Mc Donald Observatory, Arcetri, UNAM,

Kapteyn Astronomical Institute

Multiλ analysis ~1000 galaxias

+

Need of a statistically significant sample of isolated galaxies, in order to provide a baseline to compare with the behaviour of galaxies in

denser environments

Need of intensive and complex analysis of 3D data

2D spatial + 1 Velocity

VIRTUAL OBSERVATORY

The Virtual Observatory is an infrastructure of interoperable data and services.

IVOA provides technical standards for :

• Providers to share data and services

• Developers of applications to discover the services, find and access the data

The final goal is that astronomers use this data infrastructure in a seamless way

VIRTUAL OBSERVATORY

The Virtual Observatory is an infrastructure of interoperable data and services.

IVOA provides technical standards for :

• Providers to share data and services

• Developers of applications to discover the services, find and access the data

The final goal is that astronomers use this data infrastructure in a seamless way

VIRTUAL OBSERVATORY

The Virtual Observatory is an infrastructure of interoperable data and services.

IVOA provides technical standards for :

• Providers to share data and services

• Developers of applications to discover the services, find and access the data

The final goal is that astronomers use this data infrastructure in a seamless way

VIRTUAL OBSERVATORY

The Virtual Observatory is an infrastructure of interoperable data and services.

IVOA provides technical standards for :

• Providers to share data and services

• Developers of applications to discover the services, find and access the data

The final goal is that astronomers use this data infrastructure in a seamless way

MULTIDIMENSIONAL DATA

Credit

Stephen Todd and Douglas Pierce-Price

Observational Techniques

• Radiointerferometry

• Integral Field Spectroscopy

• Multi Object Spectroscopy

• Fabry-Perot Instruments

• OTF Imaging

Credit

M. Westmoquette

AMIGA VO CONTRIBUTIONS

AMIGA Catalog

• ConeSearch Service

• Web Interface

RADAMS

Radio Astronomy Data Model for Single-dish telescopesJuan de Dios Santader-Vela

Robledo DSS-63 VO Archive

• ConeSearch Service

• SSA Service

• Web Interface

TAPAS

Telescope Archive for Public Access System

IRAM-30m VO Archive

• ConeSearch Service

• Web Interface

GENERIC DATASETS

Typed Datasets

• Tabular Data

• 1D Spectra

• 2D Images

• 3D Cubes

• Time Series

SIMPLE ACCESS PROTOCOLS

GENERIC DATASETS

Typed Datasets

• Tabular Data

• 1D Spectra

• 2D Images

• 3D Cubes

• Time Series

Generic / MultiTyped DatasetsComplex data associations of different individual types

Survey Field

• Spectral data cube

• 2D projections/extractions of the cube

• Source catalog computed from the 2-D continuum

• Some extracted spectra of objects in the field

SIMPLE ACCESS PROTOCOLS

GENERIC DATASETS

Typed Datasets

• Tabular Data

• 1D Spectra

• 2D Images

• 3D Cubes

• Time Series

Generic / MultiTyped DatasetsComplex data associations of different individual types

Survey Field

• Spectral data cube

• 2D projections/extractions of the cube

• Source catalog computed from the 2-D continuum

• Some extracted spectra of objects in the field

SIMPLE ACCESS PROTOCOLS

DATA DISCOVERY

Associated Data Collections

• Simple Access Protocols perform discovery of associated data

• Present IVOA models provide full description

• REF-ID mechanism in VOTable allow association of data

• RESOURCE mechanism in VOTable allow metadata extension

DATA DISCOVERY

Associated Data Collections

• Simple Access Protocols perform discovery of associated data

• Present IVOA models provide full description

• REF-ID mechanism in VOTable allow association of data

• RESOURCE mechanism in VOTable allow metadata extension

Issues

• Original data products can be very large, worsening transfer rates and latency

• Clients applications do not support all native observatory-dependent formats

• Users are often not interested in the whole product but in a smaller portion

DATA DISCOVERY

Associated Data Collections

• Simple Access Protocols perform discovery of associated data

• Present IVOA models provide full description

• REF-ID mechanism in VOTable allow association of data

• RESOURCE mechanism in VOTable allow metadata extension

Issues

• Original data products can be very large, worsening transfer rates and latency

• Clients applications do not support all native observatory-dependent formats

• Users are often not interested in the whole product but in a smaller portion

Virtual Data from Uniformly Sampled Datasets

• Data generated on-the-fly at access time based on user demands

• Discovery implies negotiation with the service for access methods

• WCS metadata needed for most virtual data generation

DATA ACCESS

Virtual Data generation may require

asynchronous services deployed on

distributed GRID architectures

Whole dataset

Filtering/Flagging

Spectrum extraction

2D slices extraction

Dimensional reduction

Cutout 3D sub-cube

General 2D projection

General 3D projection

General 2D slices through a 3D cube

Complex transformations

GDS DISCOVERY INPUTS

REQUEST=queryData

POS

SIZE

BAND

TIME

POL

FORMAT

REDSHIFT

REGION

INTERSECT

TARGETNAME

TARGETCLASS

TYPE

SPECRES

SPATRES

TIMERES

FLUXLIMIT

SNR

VARAMPL

FLUXCALIB

WAVECALIB

ASTCALIB

PUBID

CREATORID

COLLECTION

MTIME

TOP

MAXREC

COMPRESS

RUNID

COVERAGE

TARGET

RESOLUTION

PRECISION

SENSITIVITY

PUBLISHER

SERVICE

GDS DISCOVERY OUTPUT

Query

Association

Access

DataSet: Declaration of Spatial, Time and Polarization Axis

DataID

Provenance: BeamMajorAxis, BeamMinorAxis, BeamPositionAngle

Curation

Target: Velocity

Derived: DerivedVelocity, VelocityStatError, VelocityConfidence

VarAmplStatError, VarAmplConfidence

CoordSys: RedshiftFrameUcd

Char.SpatialAxis

Char.SpectralAxis

Char.TimeAxis

Char.FluxAxis: FluxAverage, FluxMin, FluxSaturation, FluxSupportExtent

Char.PolarizationAxis

CONSISTENCY

INSTRUMENTAL

PHYSICS

CONSISTENCY

PHYSICS

GDS IMPLEMENTATION

GDS IMPLEMENTATION

GDS IMPLEMENTATION

accessData

getCapabilities

• Study of the state of the art of both MultiD data in Astronomy and Protocols in the VO

• Determine the best strategy for discovery and access of complex MultiD datasets

• Propose Discovery Method for a GDS conceived in the less possible intrusive way

• Reuse of existing VO Data Models and VO Protocols with minor modifications

• Implementation of the proposed Discovery Method for a GDS

CONCLUSIONS

• Virtual Data generation and accesData standards needed

• Achieve final IVOA recommendation for MultiD discovery and access VO protocols

• Upcoming facilities will provide 3D datacubes and services to access and use them

• getCapabilities method is key for interoperability among services

• Conception and development of VO Scientific Workflows for 3D Data Analysis

• EU funded project Wf4Ever

Advanced Workflow Preservation Technologies for Enhanced Sience

FUTURE WORK