roy williams us national virtual observatory project california institute of technology
DESCRIPTION
Cyberinfrastructure for Astronomy. Roy Williams US National Virtual Observatory Project California Institute of Technology. Toward a “new astronomy”. Past: Observations of small, carefully selected samples. Trends. Future dominated by detector improvements. - PowerPoint PPT PresentationTRANSCRIPT
Cyberinfrastructurefor
Astronomy
Roy Williams
US National Virtual Observatory ProjectCalifornia Institute of Technology
Toward a “new astronomy”
• Past: Observations of small, carefully selected samples
Trends
• Future dominated by detector improvements
Total area of 3m+ telescopes in the world in m2, total number of CCD pixels in Megapixels, as a function of time. Growth over 25 years is a factor of 30 in glass, 3000 in pixels.
• Moore’s Law growth in CCD capabilities
• Gigapixel arrays on the horizon
• New Detector Technologies (e.g., STJ)
•Improvements in computing and storage will track growth in data volume
• Investment in software is critical, and growing
New astronomy: Data Federation
New Astronomy: Data Mining
• Future: Multi-wavelength data for millions of objects, allowing us to:– Discover significant patterns from the analysis of
statistically rich and unbiased image/catalog databases
– Understand complex astrophysical systems via confrontation between data and sophisticated numerical simulation
New Astronomy: New Views
• Simultaneous access to – multi-wavelength archives, – advanced visualization and – statistical analysis tools
New Astronomy: Multispectral Imagery
Crab Nebula.3 channels: X-ray in blue, optical in green, and radio in red.
Moffet Field California. 224 channels from 400 nm to 2500 nm
The Virtual Observatory
International VOhttp://ivoa.net
US VOhttp://us-vo.org
Interoperability
VizieR: Contains more than 4000 astronomical catalogues consisting of one or several tables.
“Give me all tables containing the V magnitude in the Johnson system.”
144 different names for Johnson V.
But first, what can you do now?
Science applications
– Resource discovery: Registry– Data discovery and integration: DataScope– Database queries and cross-correlation:
OpenSkyQuery– Spectrum browsing and analysis: Spectrum
Services– Source extraction / cross-correlation: WESEX,
WCS fixer– Sky coverage of catalogs: Catalog Coverage
Maps– On-demand mosaicking: Mosaic Service
NVO Registry Portal
Find source catalogs, image archives, and other astronomical resources registered with the NVO
A Registry is a distributed database of Virtual Observatory resources: primarily access services for catalog, image, and spectral data, but also descriptions of organizations and data collections. There are several coordinated registry implementations that share information by harvesting each other's resources. This registry is at STScI in Baltimore, MD.Searches for resources can be done by keyword, or advanced queries can be expressed in the SQL language. The registry is open for humans through web forms, or machines through SOAP web services.
DataScope
Using the NVO DataScope scientists can discover and explore hundreds of data resources available in the Virtual Observatory. DataScope uses the VO registry and VO access protocols to link to archives and catalogs around the world. Users can immediately discover what is known about a given region of the sky: they can view survey images from the radio through the X-ray, explore archived observations from multiple archives, find recent articles describing analysis of data in the region,
Discover and explore data in the Virtual Observatory
find known interesting or peculiar objects and survey datasets that cover the region. A summary page provides a quick précis of all of the available data. Users can download images and tables for further analysis on their local machines, or they can go directly to a growing set of VO enabled analysis tools, including Aladin, OASIS, VOPlot and VOStat.
The “view-data” link goes directly to DataScope.
See the event context from all major surveys.
See archival images corresponding to the latest gamma-ray bursts
DataScope with VOEvent
The results have already been collected from across the VO as soon as the GRB happens.
OpenSkyQuery
Cross-match your data with numerous catalogs
OpenSkyQuery allows you to cross-match astronomical catalogs and select subsets of catalogs with a general and powerful query language. You can also import a personal catalog of objects and cross-match it against selected databases.
Spectrum Services
Search, plot, and retrieve SDSS, 2dF, and other spectraThe Spectrum Services web site is dedicated to spectrum related VO services. On this site you will find tools and tutorials on how to access close to 500,000 spectra from the Sloan Digital Sky Survey (SDSS DR1) and the 2 degree Field redshift survey (2dFGRS). The services are open to everyone to publish their own spectra in the same framework. Reading the tutorials on XML Web Services, you can learn how to integrate the 45 GB spectrum and passband database with your programs with few lines of code.
Web Enabled Source Identification with Cross-Matching (WESIX)
Upload images to SExtractor and cross-correlate the objects found with selected survey catalogs.
This NVO service does source extraction and cross-matching for any astrometric FITS image. The user uploads a FITS image, and the remote service runs the SExtractor software for source extraction. The resulting catalog can be cross-matched with any of several major surveys, and the results returned as a VOTable. The web page also allows use of Aladin or VOPlot to visualize results.
An essential part of an astronomical image is the mapping between pixels and sky — the so-called WCS information. This NVO service takes an image with approximate WCS and computes the exact WCS, using major catalogs such as USNO-B and SDSS.
USNO-B1USNO-A2SDSSNOMAD
Input image
• Source detect• Match to catalog• Fit parameters
Exact WCS
Client Remote NVO Service
Refine an image with approximate pixel mapping to the exact WCS solution
WCS Fixer
Catalogue of Stellar Spectral Classifications (Skiff, 2005)
Find catalogs and source density for the position or object name of interest
An all-sky map showing distribution of sources across the sky. Dynamically generate a custom map for specific region and selection of catalogs. Composite all-sky overlay maps show which regions are covered by all of the selected catalogs.
Catalog Coverag Maps
Try it out!
• Start at http://us-vo.org• Develop your own VO-enabled software
with the NVO Summer School software kit
• For developers: come to the 2006 NVO Summer School in Aspen
DataScope
NVO: Spectrum Services
OpenSkyQuery
VOPlot
Under the hood…
… standards
Regis
try
Vie
ws
Serv
ices
View:Client’s Frame of Reference
File/Blob TableMIME type
SemanticsFITS
XMLS
pect
ral L
ines
Sour
ce C
atal
og
Spec
trum
Imag
e
CharacterizationR
egio
nE
vent
Dat
acub
e
Services, Filesystems, VOSpace
Space/Time
Portals, Applications, Interfaces
VOTable
Edu
cati
on
Service: Interacting with remote resources
Grid& Globus
Protocols (SOAP, REST, GET)Accounts & Security
AsyncC
one
Imag
e
Certificates
Dat
acub
e
WWW
Spe
ctru
m
Cat
alog
Reg
istr
y
VOSpace
Cus
tom
....
Contracts (WSDL)
Service Requests
Query Language
Files and Databases
internet
Example Web Service
name= M51x= 10y= 10sky_survey=DSS2_redmime_type=download_gif
http
://arc
hiv
e.e
so.o
rg/d
ss/d
ss/im
ag
eClient interface
Request: Keyword/value
Response: image/gif
Serviceendpoint
Web Services
• Remote execution
• Structures in and out
• Self-defined (WSDL)
• Click or code
Mosaicking enginefrom services
Logical SIAP
NVO Registry
Physical SIAP
Computing
Portal
Trust
Request:survey,region,[login]
Sandbox
web access
Image Data
Service
Registry:Publish, Find & Use
IVO identifiers
Con
e
Org
aniz
atio
n
Eve
nt p
ubli
sher
Harvest
Prov
enan
ce
Imag
e
Com
pute
Ser
vice
Registry Curation
Spec
trum
App
lica
tion
Etc
etc
....
Service
Cus
tom
....
Skyn
ode
Registryof
Registries
Distributed Registry
Caltech
NCSA
STScI/JHU
HEASARC
Astrogrid
CDS
JapanVO
Ongoing harvesting May 05
ESO
CfA
NOAO
DataScope
• Federates multiple cone, SIAP services
Registry
OAIPublish
Registry
OAI QueryRegistry
OAIPublish
DataScope
1
2
3
4Caltech
NCSA
JHU/StSci
Goddard
Palomar-QuestA repeating sky surveyCaltech-Yale
Palomar-Quest SurveyDjorgovski, Baltey, Drake, Graham, Mahabal, Williams
P48 Telescope
Caltech Yale
NCSA
Transient pipeline computing reservation at sunrise for immediate followup of transients
Synoptic survey massive resampling for ultrafaint detection of hi-Z quasars
NCSA and Caltech and Yale run different pipelines on the same data
50 Gbyte/night ALERT
Griffith Observatory“big picture”
158 feet
Show tile b14
158 feet
158 feet
Secret Sauce is Software
VOEventInstant Messaging for Astronomy
• Automated architecture• Structured packets• Trust
1604 image of Cas A
Transient Sources in the Sky
• Asteroids• Variables and binaries• Supernova• Gamma-ray burst• Lensing events• Gravitational waves• etc etc and:
• NOT YET THOUGHT OF ....?
• http://voeventnet.caltech.edu/transients/
Gamma-ray Burst Satellite Feed
subscribersubscriber
author/publisher
socket
subscriber
emailtelephone
Single provider Structured, for machine understandingFixed structuresStandard transports
Caltech
Exeter, UK
Los Alamos
Tucson
SDSS SNe U Washington/Stanford
PairitelBerkeley
Palomar P60Caltech
Palomar-Quest Caltech
RaptorLANL
Liverpool TelescopeLa Palma
UKIRTHawaii
AuthorPublisherRepositoryRelaySubscriber
FaulkesHawaii/Australia
OGLE IIIPoland
GCNNASA/GSFC
Microlensing Survey Exeter
CTIO/KPNO surveys
Sources of Events
• GCN, (Swift, Integral, etc)• Palomar-Quest (Caltech), Raptor (LANL),
etc• Supernova factories• Variable star observers (AAVSO)• non-photon:
– LIGO, neutrinos, cosmic rays, etc
• Future– PannStarrs, LSST
VOEvent Structure• Who
• Publisher, Contact, etc• Subscribers will use PublisherID to select
• WhereWhen (== STC)• Can be simple eg RA, Dec, eg UTC• Can be sophisticated, eg multiple frames, near objects, etc
• What• Hierarchy of named parameters• Units, Semantic type (UCD)• References, Descriptions
• How• How was the evidence gathered: camera, telescope, etc
• Why• probability list of interpretation
– supernova, comet, asteroid, .....
• Citation• Link to other VOEvent: Followup, Supercede, Retraction, • Link to support data
Google-like interfaces (coming)
• Prototype pan/zoom at Pittsburgh
QuickTime™ and aYUV420 codec decompressor
are needed to see this picture.
Questions?