Download - Multidimensional Data in the VO
Multidimensional Data in the
Virtual Observatory
Jose Enrique Ruiz del Mazo
Tutora: Dr. Lourdes Verdes-Montenegro
IAA - CSIC
Máster FISYMAT
Trabajo de Investigación Tutelada
Universidad de Granada
Diciembre 2010
Context
• The AMIGA project
• The Virtual Observatory
• Multidimensional Data in Astronomy
• AMIGA VO Contributions
Generic Datasets
• Data Discovery
• Data Access
Generic Dataset Discovery Service
• Input Parameters
• Query Response
• Implementation
Conclusions and Future Work
SUMMARY
AMIGA
Analysis of the interstellar Medium of Isolated GAlaxies
PI : Dr. Lourdes Verdes-Montenegro
IAA-CSIC, IRAM
http://amiga.iaa.es
Obs. Marseille, Obs. Paris, CfA, ASIAA, MPIfA, IAC,
Univ. Alabama, Mc Donald Observatory, Arcetri, UNAM,
Kapteyn Astronomical Institute
Multiλ analysis ~1000 galaxias
+
Need of a statistically significant sample of isolated galaxies, in order to provide a baseline to compare with the behaviour of galaxies in
denser environments
Need of intensive and complex analysis of 3D data
2D spatial + 1 Velocity
VIRTUAL OBSERVATORY
The Virtual Observatory is an infrastructure of interoperable data and services.
IVOA provides technical standards for :
• Providers to share data and services
• Developers of applications to discover the services, find and access the data
The final goal is that astronomers use this data infrastructure in a seamless way
VIRTUAL OBSERVATORY
The Virtual Observatory is an infrastructure of interoperable data and services.
IVOA provides technical standards for :
• Providers to share data and services
• Developers of applications to discover the services, find and access the data
The final goal is that astronomers use this data infrastructure in a seamless way
VIRTUAL OBSERVATORY
The Virtual Observatory is an infrastructure of interoperable data and services.
IVOA provides technical standards for :
• Providers to share data and services
• Developers of applications to discover the services, find and access the data
The final goal is that astronomers use this data infrastructure in a seamless way
VIRTUAL OBSERVATORY
The Virtual Observatory is an infrastructure of interoperable data and services.
IVOA provides technical standards for :
• Providers to share data and services
• Developers of applications to discover the services, find and access the data
The final goal is that astronomers use this data infrastructure in a seamless way
MULTIDIMENSIONAL DATA
Credit
Stephen Todd and Douglas Pierce-Price
Observational Techniques
• Radiointerferometry
• Integral Field Spectroscopy
• Multi Object Spectroscopy
• Fabry-Perot Instruments
• OTF Imaging
Credit
M. Westmoquette
AMIGA VO CONTRIBUTIONS
AMIGA Catalog
• ConeSearch Service
• Web Interface
RADAMS
Radio Astronomy Data Model for Single-dish telescopesJuan de Dios Santader-Vela
Robledo DSS-63 VO Archive
• ConeSearch Service
• SSA Service
• Web Interface
TAPAS
Telescope Archive for Public Access System
IRAM-30m VO Archive
• ConeSearch Service
• Web Interface
GENERIC DATASETS
Typed Datasets
• Tabular Data
• 1D Spectra
• 2D Images
• 3D Cubes
• Time Series
SIMPLE ACCESS PROTOCOLS
GENERIC DATASETS
Typed Datasets
• Tabular Data
• 1D Spectra
• 2D Images
• 3D Cubes
• Time Series
Generic / MultiTyped DatasetsComplex data associations of different individual types
Survey Field
• Spectral data cube
• 2D projections/extractions of the cube
• Source catalog computed from the 2-D continuum
• Some extracted spectra of objects in the field
SIMPLE ACCESS PROTOCOLS
GENERIC DATASETS
Typed Datasets
• Tabular Data
• 1D Spectra
• 2D Images
• 3D Cubes
• Time Series
Generic / MultiTyped DatasetsComplex data associations of different individual types
Survey Field
• Spectral data cube
• 2D projections/extractions of the cube
• Source catalog computed from the 2-D continuum
• Some extracted spectra of objects in the field
SIMPLE ACCESS PROTOCOLS
DATA DISCOVERY
Associated Data Collections
• Simple Access Protocols perform discovery of associated data
• Present IVOA models provide full description
• REF-ID mechanism in VOTable allow association of data
• RESOURCE mechanism in VOTable allow metadata extension
DATA DISCOVERY
Associated Data Collections
• Simple Access Protocols perform discovery of associated data
• Present IVOA models provide full description
• REF-ID mechanism in VOTable allow association of data
• RESOURCE mechanism in VOTable allow metadata extension
Issues
• Original data products can be very large, worsening transfer rates and latency
• Clients applications do not support all native observatory-dependent formats
• Users are often not interested in the whole product but in a smaller portion
DATA DISCOVERY
Associated Data Collections
• Simple Access Protocols perform discovery of associated data
• Present IVOA models provide full description
• REF-ID mechanism in VOTable allow association of data
• RESOURCE mechanism in VOTable allow metadata extension
Issues
• Original data products can be very large, worsening transfer rates and latency
• Clients applications do not support all native observatory-dependent formats
• Users are often not interested in the whole product but in a smaller portion
Virtual Data from Uniformly Sampled Datasets
• Data generated on-the-fly at access time based on user demands
• Discovery implies negotiation with the service for access methods
• WCS metadata needed for most virtual data generation
DATA ACCESS
Virtual Data generation may require
asynchronous services deployed on
distributed GRID architectures
Whole dataset
Filtering/Flagging
Spectrum extraction
2D slices extraction
Dimensional reduction
Cutout 3D sub-cube
General 2D projection
General 3D projection
General 2D slices through a 3D cube
Complex transformations
GDS DISCOVERY INPUTS
REQUEST=queryData
POS
SIZE
BAND
TIME
POL
FORMAT
REDSHIFT
REGION
INTERSECT
TARGETNAME
TARGETCLASS
TYPE
SPECRES
SPATRES
TIMERES
FLUXLIMIT
SNR
VARAMPL
FLUXCALIB
WAVECALIB
ASTCALIB
PUBID
CREATORID
COLLECTION
MTIME
TOP
MAXREC
COMPRESS
RUNID
COVERAGE
TARGET
RESOLUTION
PRECISION
SENSITIVITY
PUBLISHER
SERVICE
GDS DISCOVERY OUTPUT
Query
Association
Access
DataSet: Declaration of Spatial, Time and Polarization Axis
DataID
Provenance: BeamMajorAxis, BeamMinorAxis, BeamPositionAngle
Curation
Target: Velocity
Derived: DerivedVelocity, VelocityStatError, VelocityConfidence
VarAmplStatError, VarAmplConfidence
CoordSys: RedshiftFrameUcd
Char.SpatialAxis
Char.SpectralAxis
Char.TimeAxis
Char.FluxAxis: FluxAverage, FluxMin, FluxSaturation, FluxSupportExtent
Char.PolarizationAxis
CONSISTENCY
INSTRUMENTAL
PHYSICS
CONSISTENCY
PHYSICS
GDS IMPLEMENTATION
GDS IMPLEMENTATION
GDS IMPLEMENTATION
accessData
getCapabilities
• Study of the state of the art of both MultiD data in Astronomy and Protocols in the VO
• Determine the best strategy for discovery and access of complex MultiD datasets
• Propose Discovery Method for a GDS conceived in the less possible intrusive way
• Reuse of existing VO Data Models and VO Protocols with minor modifications
• Implementation of the proposed Discovery Method for a GDS
CONCLUSIONS
• Virtual Data generation and accesData standards needed
• Achieve final IVOA recommendation for MultiD discovery and access VO protocols
• Upcoming facilities will provide 3D datacubes and services to access and use them
• getCapabilities method is key for interoperability among services
• Conception and development of VO Scientific Workflows for 3D Data Analysis
• EU funded project Wf4Ever
Advanced Workflow Preservation Technologies for Enhanced Sience
FUTURE WORK