numerical computation of non-comm. voi metrics & spectra of random graphs

Valueof

Information

1st year review. UCLA 2012

Kickoff

ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and Exploitation

Valueof

Information


Numerical computation of Non-Comm. VoI Metrics & Spectra of

Random Graphs

Co-PI Raj Rao NadakuditiUniversity of Michigan

Valueof

Information


Kickoff


Valueof

Information

Mission Informationand

Objectives

Non-commutativeInfo Theory

Info-geometric learning

Consensuslearning

Info theoreticsurrogates

Information-driven Learning. Jordan (Lead); Ertin, Fisher,

Hero, Nadakuditi

Bounds, models

and

learning algorithms

Scalable, Actionable

VoI measures

Research programInfo-driven learning

Valueof

Information


Kickoff


Valueof

Information

• Principal component analysis • Direction-finding (e.g. sniper localization) • Pre-processing/Denoising to SVM-based classification (e.g. pattern, gait & face recognition) • Regression, Matched subspace detectors• Community/Anomaly detection in networks/graphs

• Canonical Correlation Analysis• PCA-extension for fusing multiple correlated sources

• LDA, MDS, LSI, Kernel(.) ++, MissingData(.)++

• Eigen-analysis Spectral Dim. Red. Subspace methods• Technical challenge:

• Quantify eigen-VoI (Thrust 1) and Exploit quantified uncertainty (Thrust 2) for eigen-analysis based sensor fusion and learning

Eigen-analysis methods & apps.

Valueof

Information


Kickoff


Valueof

Information Role of Non-Comm. Info theory

• For noisy, estimated subspaces, quantify:• Fundamental limits and phase transitions• Estimates of accuracy possibly, data-driven• Rates of convergence, learning rates • P-values • Impact of adversarial noise models

• “Classical” info. measures in low-dim.-large sample regime• e.g. f-divergence, Shannon mutual info., Sanov’s thm.

vs.• Non-comm. info. measures in high-dim.-relatively-small-

sample regime• Non-commutative analogs of above

Valueof

Information


Kickoff


Valueof

Information Analytical signal-plus-noise model

• Low dimensional (= k) latent signal model

• Xn is n x m noise-only Gaussian matrix

• c = n/m = # Sensors / # Samples

• Theta ~ SNR

Valueof

Information


Kickoff


Valueof

Information Empirical subspaces are unequal

• c = n/m = # Sensors / # Samples

• Theta ~ SNR, X is Gaussian

• Insight: Subspace estimates are biased!• “Large-n-large-m” versus “Small-n-large-m”

Valueof

Information


Kickoff


Valueof

InformationA non-commutative VoI metric

(beyond Gaussians)

• Xn is n x m unitarily-invariant noise-only random matrix• Theorem [N. and Benaych-Georges, 2011]:

• μ = Spectral measure of noise singular values• D = D-transform of μ “log-Fourier” transform in NCI

Valueof

Information


Kickoff


Valueof

Information Numerically computing D-transform

• Desired:• Allow continuous and discrete valued inputs• O(n log n) where n is number of singular values• Numerically stable

Valueof

Information


Kickoff


Valueof

Information Empirical VoI quantification

• Based on an eigen-gap based segment, compute non-comm VoI subspaces

Valueof

Information


Kickoff


Valueof

Information Accomplishment - I

• Uk are Chebyshev polynomials• Series coefficients computed via DCT in O(n log n)• Closed-form G transform (and hence D transform)

series expansion!• “Numerical computation of convolutions in free probability theory” (with Sheehan

Olver)

Valueof

Information


Kickoff


Valueof

Information

• For noisy, estimated subspaces, quantify:• Fundamental limits and phase transitions• Estimates of accuracy possibly, data-driven• Rates of convergence, learning rates • P-values • Impact of adversarial noise models• Impact of finite training data

• Facilitate fast, accurate performance prediction for eigen-methods!

• Transition: MATLAB toolbox

Broader Impact

Valueof

Information


Kickoff


Valueof

Information Spectra of Networks

• Role of spectra of social and related networks:• Community structure discovery• Dynamics • Stability

Open problem: Predict graph spectra given degree sequence

Broader Impact: ARL CTA & ITA, ARO MURI

Valueof

Information


Kickoff


Valueof

Information Non. Comm. Prob. for Network Science

• Role of spectra of social and related networks:• Community structure discovery• Dynamics • Stability

Open Solved problem: Predict spectra of a graph given expected degree sequence

Answer: Free multiplicative convolution of degree sequence with semi-circle

“Spectra of graphs with expected degree sequence” (with Mark Newman)

Valueof

Information


Kickoff


Valueof

Information Accomplishment - II

• Predicting spectra (numerical free convolution – Accomplishment I)

• “When is a hub not a hub (spectrally)?” • New phenomena, new VoI analytics

Valueof

Information


Kickoff


Valueof

Information Phase transition in comm. detection

• Unidentifable: If cin – cout < 2 • cin = Avg. degree “within”; cout = Avg. degree “without”

Valueof

Information


Kickoff


Valueof

InformationRelation to other research thrusts

• Accomplishments– Numerical computation of Non-Comm convolutions– Predicting spectra of complicated networks

• Impact– Information fusion

• Numerical computation of Non-Comm. Metrics• Performance prediction• New VoI analytics for networks• Predicting graph spectra from degree sequence

– Information exploitation• Selective fusion of subspace information

numerical computation of non-comm. voi metrics & spectra of random graphs

Documents