may 17, 2005maria nieto-santisteban, jhu / ivoa - kyoto1 vo applications @ jhu open skyquery and...
TRANSCRIPT
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 1
VO Applications @ JHUOpen SkyQuery and more
…T. Budavari, S. Carliles, L. Dobos, G. Fekete, J. Gupchup, V. Haridas, N. Li,
T. Malik, M. A. Nieto-Santisteban, W. O’Mullane, J. Raddick, A. Szalay, & A. Thakar
The Johns Hopkins University
Baltimore, MD, USA.
THE US NATIONAL VIRTUAL OBSERVATORY
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 2
NVO Core Applications
• NVO Registry (STScI & JHU)– Search and Registration of astronomical
resources (Organizations, Catalogs, Images, Services, etc.)
• NVO Spectrum Services– Search, Plotting, and Spectra upload
• Open SkyQuery – Query and Cross-match of catalog data
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 3
Related to NVO
• CasJobs– Personal database (MyDB) on the server
side– Scheduling and tracking of long SQL Jobs
• VO Enabled Mirage– Wrapper to Mirage (Bell Laboratories)– Takes input from existing VO services (Cone
and SIAP) in the form of VOTable
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 4
NVO Registry
• Distributed database of astronomical resources
• Registries share data by harvesting
• Searching & Publishing
• SOAP WSCaltech
NCSA
STScI/JHUHEASARC
Astrogrid
CDS
JapanVO
ESOCfA
NOAOESACNRAO
Registries as of April 2005
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 5
NVO Spectrum Services • Search and access > 500k spectra
• Plots, normalizes, combines spectra
• Export data in VOTable
• SOAP WS• Mirrored and
extended by MAST (STScI)
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 6
Open SkyQuery
• SkyNode individual and cross-match queries
• Import/Export VOTable
• Plots with VOPlot
• Automatic Query Builder
• SOAP WS
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 7
Virtual Observatory Query Language Layers
VOQL2 Federation
VOQL1 Web services
Portals working across nodes –SQL-like query language and federation system i.e. Open SkyQuery
SkyNodes using Astronomical Data Query Language (ADQL) and VOTABLE
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 8
Open SkyQuery Architecture
Users
OSQ Portal
Uses Registry and full SkyNodes.
May use services at any level
VOQL2
ADQL
VO
VORegistryQueryable
Archives
ExecPlan
FULL
SkyNode
SkyNode
SkyNode
SkyNode
SkyNode
register
lookup
lookupBASIC
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 9
Data Flow – Execution plan
SkyNode 1
Open SkyQuery
SkyNode 2
SkyNode 3
query
http://www.openskyquery.net
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 10
Limitations
• Intended to work with small result sets constrained by Region or other attributes– Currently nodes limited to 5k rows– Intention to remove restriction – requires
change in operation
• Upload (MyData as a SkyNode) – Size limit is ~10k rows– No Region support on MyData at the moment
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 11
Contributors & Adaptors
• Contributors– Tamas Budavari and Tanu Malik (original prototype)– Nolan Li – main .Net development (Basic & Full)– Ramon Williamson (Java Basic SkyNode)– Tom McGlynn (Java Full SkyNode)– Iñaki Ortiz ESA/ESAC (Java “Full” SkyNode)
• Adaptors– NOAO
• Open SkyQuery portal for proprietary data using a private registry
– NDWFS, Galex, GOODS, HST, UDF, Phoenix, POSSUM …
– JVO (Subaro XMM), ESAC (XMM & ISO), India VO
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 12
VO Enabled Mirage
• IVOA Client Package (Sam Carliles @JHU)– VOTable parser based on JAVOT and SAVOT
• IVOA FITS Package (Tom McGlynn @Hearsac)
• Wrapper to Mirage (Bell Labs by Tin Kam Ho)
• Simultaneous visualization of tabular and image data
• Connects to Cone and SIAP services
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 13
CasJobs• Batch system• Personal DB on
server side: Tables, Procedures, ...
• Sharing env.• Long queries• Import/Export
VOTable• will(/is) support
VOStore• Adopted by
Galex• Mirrored by VO
India (SDSSDR2)
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 14
Future Work
• Reach the big scale– Parallel Open SkyQuery: framework capable
of processing cross-match queries involving catalogs with billion of rows in less than one hour
• Integrate CasJobs and Open SkyQuery
• CasJobs systems federation
May 17, 2005Maria Nieto-Santisteban, JHU / IVOA - Kyoto 15
Links of interest
• Registry– http://voservices.net/registry/– http://nvo.stsci.edu/voregistry/
• Spectrum Services– http://voservices.net/spectrum/
• Open SkyQuery– http://openskyquery.net
• CasJobs– http://casjobs.sdss.org & http://galex.stsci.edu/casjobs &
http://skynode.iucca.ernet.in/casjobs• VO Enable Mirage & IVOA Client Package
– http://skyservice.pha.jhu.edu/develop/vo/mirage/• IVOA Client & FITS Packages
– http://skyservice.pha.jhu.edu/develop/vo/ivoa/– http://skyservice.pha.jhu.edu/develop/vo/ivoafits/
• First NVO Summer School– http://www.us-vo.org/summer-school/2004/proceedings/