us-vo.org/vim vim visual integration and mining a tool to explore your sources roy williams caltech...
TRANSCRIPT
us-vo.org/vim
VIMVisual Integration and MiningA Tool to Explore Your Sources
Roy WilliamsCaltech
THE US NATIONAL VIRTUAL OBSERVATORY
us-vo.org/vim
Start with your sources(currently <1000)
• Multiple named sky positions– Simple text (RA,Dec,ID)– Text wizard (NVO table importer, today 17.10)
– VOTable of positions (local or url)– Push positions from NVO portal
us-vo.org/vim
Getting Data about your sources
us-vo.org/vim
hover mouse to enlarge
Skyview cutouts in many wavelengths
Cen
A @
840
MH
z
us-vo.org/vim
Skyview cutouts in many wavelengths
Abell clusters in optical and X-ray
us-vo.org/vim
Spectral Collections from JHU brokered by NRAO:
• Arecibo Maser Catalog
• 2dFGRS
• SDSS DR5
Hover mouse on thumbnail to see larger image
Spectra from VO SSAP services
us-vo.org/vim
Getting Data about your sources
Proximity search
us-vo.org/vim
MulticoneProximity Search =user-supplied sources + radius
returns VOTable
radius
us-vo.org/vim
VIM tables structure– Multiple matches per source– Can join to sources (outer or nearest)
RA Dec ID
sources Cat1 Cat2
us-vo.org/vim
Shakbazian galaxy groupsprox search in Simbad
us-vo.org/vim
Table column metadata
NED metadata
us-vo.org/vim
us-vo.org/vim
Compute new columns, eg joint IR-optical colors
Choose which columns to
display
us-vo.org/vim
Visualization by download & use Topcat
us-vo.org/vim
Architecture
browser Vim
personal persistent storage
upload sourcesHTML + AJAX
Catalogs (cone)Images (Skyview)Spectra (SSAP)
All the relevant information about your sources-- mashups from the VO-- kept for you in persistent storage-- view, mine, download
python api
Registry
Inventory
NVO portal
us-vo.org/vim
• Auto-script:• Clicking builds script• Can edit script and rerun
us-vo.org/vim
# got some sdss sources from Vizier, a 12 minute radius with rmag<21# about 773 sources foundsources.run(bench, open(‘mysources.votable').read())
# run 2mass prox search at 2 arc secondsproxsearch.run(bench, survey='2MASS-PSC', sr=0.00055556, tablename='_2mass', verb='1', maxthreads='10')
# merge with the sdssjoin.run(bench, tablename='_2mass', match='best')
# select rows with non-null j magnitude (i.e. thos in 2mass)# we get abouyt 148 remainingselectrows.run(bench, tablename='sources', criterion='! NULL__2mass_j_m')
# compute 2mass-J - SDSS-raddcol.run(bench, tablename='sources', colname='JminusR', expression='_2mass_j_m - rmag')
# compute 2mass-H - SDSS-zaddcol.run(bench, tablename='sources', colname='Hminusz', expression='_2mass_h_m - zmag')
# then download sources.votable from Utility/ViewAllFiles, and plot with topcat
Vim auto-scripting
us-vo.org/vim
• WebServer or Laptop install• Pure python webserver (CherryPy)
• Scalable• Auto-script, edit, rerun paradigm• Powered by Stilts (2,000,000,000 rows !)
• Simple sharing• Share your workbench by emailing a URL
http://envoy5.cacr.caltech.edu:8888/?benchID=7462948574635185948473627485958433
• Can write-protect with password
VIM Features
us-vo.org/vim
• Content– Any cone search (all 1000’s in VOregistry)
– Cutouts from Skyview– NVO Inventory services– Spectra via SSAP (from NRAO)
• Thumbnails and images and FITS
• Display– Column selection, Row sort/select– Images small-hover-large– Tools and metadata choice– AJAX progress for data fetching …but not Safari :-(
VIM Features
us-vo.org/vim
NVO Portal:Using the registry to get catalogs
us-vo.org/vim
NVO Portal:Finding catalogs by spatial correlation
us-vo.org/vim
my sources
Catalog A:Perfect correlation
Catalog C:No correlation
Catalog B:Weak correlation
NVO Portal:Inventory service
(1000’s of catalogs ingested)
us-vo.org/vim
Questions?