![Page 1: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/1.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and Exploitation
Valueof
Information
1st year review. UCLA 2012
Numerical computation of Non-Comm. VoI Metrics & Spectra of
Random Graphs
Co-PI Raj Rao NadakuditiUniversity of Michigan
![Page 2: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/2.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information
Mission Informationand
Objectives
Non-commutativeInfo Theory
Info-geometric learning
Consensuslearning
Info theoreticsurrogates
Information-driven Learning. Jordan (Lead); Ertin, Fisher,
Hero, Nadakuditi
Bounds, models
and
learning algorith
ms
Scalable, Actionable
VoI measures
Research programInfo-driven learning
![Page 3: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/3.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information
• Principal component analysis • Direction-finding (e.g. sniper localization) • Pre-processing/Denoising to SVM-based classification (e.g. pattern, gait & face recognition) • Regression, Matched subspace detectors• Community/Anomaly detection in networks/graphs
• Canonical Correlation Analysis• PCA-extension for fusing multiple correlated sources
• LDA, MDS, LSI, Kernel(.) ++, MissingData(.)++
• Eigen-analysis Spectral Dim. Red. Subspace methods• Technical challenge:
• Quantify eigen-VoI (Thrust 1) and Exploit quantified uncertainty (Thrust 2) for eigen-analysis based sensor fusion and learning
Eigen-analysis methods & apps.
![Page 4: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/4.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationRole of Non-Comm. Info theory
• For noisy, estimated subspaces, quantify:• Fundamental limits and phase transitions• Estimates of accuracy possibly, data-driven• Rates of convergence, learning rates • P-values • Impact of adversarial noise models
• “Classical” info. measures in low-dim.-large sample regime• e.g. f-divergence, Shannon mutual info., Sanov’s thm.
vs.• Non-comm. info. measures in high-dim.-relatively-small-
sample regime• Non-commutative analogs of above
![Page 5: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/5.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationAnalytical signal-plus-noise model
• Low dimensional (= k) latent signal model
• Xn is n x m noise-only Gaussian matrix
• c = n/m = # Sensors / # Samples
• Theta ~ SNR
![Page 6: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/6.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationEmpirical subspaces are unequal
• c = n/m = # Sensors / # Samples
• Theta ~ SNR, X is Gaussian
• Insight: Subspace estimates are biased!• “Large-n-large-m” versus “Small-n-large-m”
![Page 7: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/7.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information
A non-commutative VoI metric (beyond Gaussians)
• Xn is n x m unitarily-invariant noise-only random matrix• Theorem [N. and Benaych-Georges, 2011]:
• μ = Spectral measure of noise singular values• D = D-transform of μ “log-Fourier” transform in NCI
![Page 8: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/8.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationNumerically computing D-transform
• Desired:• Allow continuous and discrete valued inputs• O(n log n) where n is number of singular values• Numerically stable
![Page 9: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/9.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationEmpirical VoI quantification
• Based on an eigen-gap based segment, compute non-comm VoI subspaces
![Page 10: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/10.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationAccomplishment - I
• Uk are Chebyshev polynomials• Series coefficients computed via DCT in O(n log n)• Closed-form G transform (and hence D transform)
series expansion!• “Numerical computation of convolutions in free probability theory” (with Sheehan
Olver)
![Page 11: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/11.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information
• For noisy, estimated subspaces, quantify:• Fundamental limits and phase transitions• Estimates of accuracy possibly, data-driven• Rates of convergence, learning rates • P-values • Impact of adversarial noise models• Impact of finite training data
• Facilitate fast, accurate performance prediction for eigen-methods!
• Transition: MATLAB toolbox
Broader Impact
![Page 12: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/12.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationSpectra of Networks
• Role of spectra of social and related networks:• Community structure discovery• Dynamics • Stability
Open problem: Predict graph spectra given degree sequence
Broader Impact: ARL CTA & ITA, ARO MURI
![Page 13: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/13.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationNon. Comm. Prob. for Network Science
• Role of spectra of social and related networks:• Community structure discovery• Dynamics • Stability
Open Solved problem: Predict spectra of a graph given expected degree sequence
Answer: Free multiplicative convolution of degree sequence with semi-circle
“Spectra of graphs with expected degree sequence” (with Mark Newman)
![Page 14: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/14.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationAccomplishment - II
• Predicting spectra (numerical free convolution – Accomplishment I)
• “When is a hub not a hub (spectrally)?” • New phenomena, new VoI analytics
![Page 15: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/15.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationPhase transition in comm. detection
• Unidentifable: If cin – cout < 2 • cin = Avg. degree “within”; cout = Avg. degree “without”
![Page 16: Value of Information 1 st year review. UCLA 2012 Kickoff ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and](https://reader035.vdocument.in/reader035/viewer/2022062321/56649ebc5503460f94bc47d7/html5/thumbnails/16.jpg)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationYear 2 plans
• Accomplishments– Numerical computation of Non-Comm convolutions– Predicting spectra of complicated networks
• Impact– Information fusion
• Numerical computation of Non-Comm. Metrics• Performance prediction• New VoI analytics for networks• Predicting graph spectra from degree sequence
– Information exploitation• Selective fusion of subspace information