![Page 1: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/1.jpg)
Connec&ng Bayesian Networks and Seman&cs (as a step towards complete classifica&on)
Ashish Mahabal, Caltech
aam at astro.caltech.edu AstroInforma8cs 2012, MSR, Redmond
With Alex Ball, George Djorgovski, Ciro Donalek, Andrew Drake, MaEhew Graham, Baback Moghaddam, Mike Turmon, Roy Williams, NS Philip, …
Ashish Mahabal
![Page 2: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/2.jpg)
Catalina Real-‐&me Transient Survey 2,500 deg2 / night (total 30000+)
LSST
Astrosat
PTF
Skymapper
Pan-‐STARRS
Recent, current and future mul8epoch surveys
Data capaci8es orders of magnitudes different.
GALEX, Spitzer, FIRST, …
LIGO + IndIGO …
Transient astronomy has been gaining trac8on
Ashish Mahabal
![Page 3: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/3.jpg)
CRTS has discovered over 6000 transients
Ashish Mahabal
Associated with the transients are several types of parameters.
![Page 4: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/4.jpg)
But we have a Variety of parameters that can be used • Discovery: magnitudes, delta-‐magnitudes • Contextual:
– distance to nearest star
– Magnitude of the star
– color of that star
– normalized distance to nearest galaxy
– Distance to nearest radio source
– Flux of nearest radio source
– Galac&c la&tude
• Follow-‐up
– Colors (g-‐r, r-‐I, i-‐z etc.)
• Prior classifica&ons (event type)
• Characteris&cs from light-‐curve
– amplitude
– Median buffer range percentage
– Standard devia&on
– Stetson k
– Flux percen&le ra&o mid80
– Prior outburst sta&s&c
Not all parameters are always present leading to swiss-‐cheese like data sets
hEp://ki-‐media.blogspot.com/
Bayesian Networks best to deal with such datasets as they can deal with missing data and the structure can be learnt from the data – at least in principle
Ashish Mahabal
![Page 5: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/5.jpg)
We are interested in all types of Transients AGN Subtypes
SN Subtypes
![Page 6: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/6.jpg)
GPRs can be useful for this Given several epochs and corresponding magnitudes, es&mate the likelihood of a par&cular magnitude for a new epoch (using some covariance func&on)
hyperparameters are “free” and are varied (all 3)
B Moghaddam
Ashish Mahabal
![Page 7: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/7.jpg)
dm/dt Classifier Architecture
SN not SN
SN Ia SN II
CV / Blz RR / Mira
CV Blazar RR Lyrae Mira
Decision Tree decomposes this mul&-‐class classifier into a series of binary discrimina&on tasks.
This specific DT follows the stra&fica&on that seems natural to astronomers.
All nodes shown were Implemented via “jump histogram” binary classifiers.
Cataclysmic Periodic
Collapsing
Ashish Mahabal
98%
73%
B Moghaddam, Y Chen
With GPRs and dm/dt, the more epochs you have the beEer the results are
![Page 8: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/8.jpg)
Combiner
Ensemble of kNNs
Ensembles of Decision Trees
Bayesian Network
Neural Network
Input Data
“Supervised” SOM
Combiner
Final Classifica&on
Lightcurves Features Archival Data
Ext. Kn.
Model BoK, CV, feature selec&on, etc
There are difficul8es with that and we will not be going into that.
![Page 9: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/9.jpg)
• Many bits of informa&on can be used towards that – Broadly speaking SNe rise only once (not strictly, but we are talking large delta-‐mag in small delta-‐&me here), allowing for early characteriza&on
As part of the general theme of classifying transients and variables, it is important to separate SNe from non-‐SNe
Ashish Mahabal
Just three points can be used in principle to say if an object is a supernova or not
![Page 10: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/10.jpg)
• Many surveys are primarily ager SNe – Ogen using image subtrac&on (and thus targe&ng known galaxies)
– One problem is that dwarf galaxies are missed – And there are many such
SN Hunt 7
Ashish Mahabal
![Page 11: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/11.jpg)
Naïve Bayes
• x: feature vector of event parameters • y: object class that gives rise to x (1<y<k) • Certain features of x known: (posi&on, flux) • Others will be unknown: (color, delta-‐mag) • Assump&on: based on y, x is decomposable into B
dis&nct independent classes (labeled xb) • This helps with the curse of dimensionality • Also allows us to deal with missing values
Ashish Mahabal Radio, for instance
BN with P60 follow-‐up colors: CV/SN classifica&on ~80% with single epoch
![Page 12: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/12.jpg)
Very broadly speaking 5 flavors of BNs possible
– Naïve – TAN – constructed (seman&cs, expert knowledge etc.) – single winner from several naïve
– Fully learned from data
Naive Bayesian Network is good because its easier Learning from data is 8me consuming Missing values complicate maNers TAN oOen “learns” wrong connec8ons
Ashish Mahabal
As number of parameters grow, search space increases super-‐exponen&ally
BNs allow us to state dependencies and independencies between variables (nodes)
![Page 13: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/13.jpg)
Output table from Naïve BN
A Ball Ashish Mahabal
![Page 14: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/14.jpg)
Wrongly connected
Tree Augmented Network
Ashish Mahabal
TAN can some8mes “learn” wrong connec8ons
![Page 15: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/15.jpg)
Output table for TAN BN (clearly worse than naïve)
Ashish Mahabal
![Page 16: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/16.jpg)
The aim …
Ashish Mahabal
![Page 17: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/17.jpg)
Simple parameters that can be used:
• Distance to nearest star (and perhaps color) • Galaxy proximity (normalized)
• Archival lightcurve (including with upper limits)
As part of the general theme of classifying transients and variables, it is important to separate SNe from non-‐SNe
Ashish Mahabal
![Page 18: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/18.jpg)
Proximity to a galaxy is useful in marking SNe – Ogen there is confusion when more than one galaxy is present nearby
– As a result distance has to be normalized • Which radius to consider is also an important considera&on
– Misclassifica&on of S/G possible, so look at nearest stars
– Coincident stars will be a direct NO – Dwarf galaxies (low SNR – not catalogued – have to be considered)
Only a small frac&on have radio-‐counterparts – so that too is a NO (but if yes, its likely to be very interes&ng)
Ashish Mahabal
![Page 19: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/19.jpg)
Definite SN – but in which galaxy?
Ashish Mahabal
The transient (not seen) is at the center)
![Page 20: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/20.jpg)
Ashish Mahabal
The corresponding schema&c
![Page 21: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/21.jpg)
Ashish Mahabal
De Vaucouleurs’ radius (and associated ellip&city) ogen misleading for some types of galaxies.
![Page 22: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/22.jpg)
Ashish Mahabal
Petrosian radius ogen more representa&ve
![Page 23: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/23.jpg)
Possible SN in a cluster
Ashish Mahabal
No obvious galaxy near the center – can be easily missed
![Page 24: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/24.jpg)
Ashish Mahabal
Schema&c for the cluster field
![Page 25: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/25.jpg)
lower sigma detec&ons have to be considered for full treatment
Alex Ball Ashish Mahabal
![Page 26: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/26.jpg)
Using archival lightcurves, including upper-‐limits, one can separate SNe from sources like CVs (due to specific high thresholds, earlier outbursts could have been missed making one mistake a CV for a supernova)
Exploi&ng sta&s&cs from peaks useful
(1) Number of peaks (2) Points in each peak
(3) dt between peaks (4) Ra&o of their significance
CV SN
Ashish Mahabal
![Page 27: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/27.jpg)
Ashish Mahabal
Boxplot with dt-‐ra&o calculated from 1-‐sigma peaks
![Page 28: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/28.jpg)
Ashish Mahabal
N-‐sigma sta&s&cs for different types of transients
![Page 29: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/29.jpg)
Separa&on of SNe and non-‐SNe
normalized Based on peaks
80-‐90% completeness
Ashish Mahabal
![Page 30: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/30.jpg)
Non-‐SNe (1) SNe (2)
1
2
2
2
1
1
Using 900 non-‐SNe and 600 SNe
80-‐90% completeness using just these parameters
Ashish Mahabal
![Page 31: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/31.jpg)
Ashish Mahabal
![Page 32: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/32.jpg)
Alex Ball Ashish Mahabal
![Page 33: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/33.jpg)
The new classifica8ons feed back in to the por_olio of the object – to be combined with other bits
Ashish Mahabal
![Page 34: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/34.jpg)
CRTS follow-‐up through various telescopes (Gaia network, IGO, P60, SAAO)
hEp://gaia020.ast.cam.ac.uk:5000/ hEp://www.astro.caltech.edu/P60FollowUp/ Ashish Mahabal
S. Barway, V. Mohan, L Wyrzykowski, S Koposov, S Hodgkin, V Mohan, S Barway, L Eyer, R Anderson, G Altavilla, G Clemen&ni, …
![Page 35: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/35.jpg)
Definite steps Towards Automated Event Classifica&on
Event parameters: m1(t), m2(t),… α, δ, µ, … image shape…
colors lightcurves
etc.
Expert and ML generated priors
contextual informa&on
Event Classifica8on
Engine
P(SN Ia) = … P(SN II) = … P(AGN) = … P(CV) = … P(dM) = …
….
A necessity for large synop&c surveys
Classifica&on probabili&es (evolving, iterated)
IVOA
Ashish Mahabal
![Page 36: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/36.jpg)
Automa&ng the Op&mal Follow-‐Up What type of follow-up data has the greatest potential to discriminate among the competing models (event classes)? Request follow-up observations from the optimal available facility
B. Moghaddam, M. Turmon (JPL)
![Page 37: Connec&ng(Bayesian(Networks(and(Seman&cs · CatalinaRealL&me(TransientSurvey(2,500 deg2 / night (total 30000+)( LSST Astrosat(PTF(Skymapper(PanLSTARRS(Recent,%current%and% futuremulepoch](https://reader033.vdocument.in/reader033/viewer/2022051800/5ad36cde7f8b9a72118e567d/html5/thumbnails/37.jpg)
Summary
Ashish Mahabal
Bayesian networks are powerful tools for classifica&on • Naïve networks ogen work best • Learning from data most principled but also resource intensive (even prohibi&ve) • Combining expert knowledge and seman&cs with combina&ons of naïve networks likely to be the best compromise
Just a few parameters enough to separate some classes. Such small networks can be combined with others for a comprehensive classifica8on u8lity that can be run in real-‐8me for surveys like CRTS and more follow-‐up worthy objects chosen thereof.
Concrete ideas about incorpora8ng more seman8cs are welcome.