significance: gives for the first time exact inference results in closed-form efficiency is cubic in...
Post on 19-Dec-2015
212 views
TRANSCRIPT
• Significance: gives for the first time exact inference results in closed-form
• Efficiency is cubic in the number of variables
• Derived Stable-Jacobi approximate inference algorithm.• Significance: when converging, converges to the exact result, while typically more efficient• We analyze its convergence and give two sufficient conditions for
convergence.
• Detection: given the channel transformation A, observation vector y, and the stable parameters of the noise z, compute the most probable transmission x
• Sample CDMA problem setup borrowed from [Yener-Tran-Comm.-2002]
• Exact inference: more accurate detection than methods designed for the AWGN (additive white Gaussian noise channel)
• Approximate inference: converges, as predicted to the exact conditional posterior marginals
• First time exact inference in linear-stable model
• Faster, more accurate, reduces memory consumption and conveniently computed in closed-form
• Future work: • Investigate other families of distributions like Wishart and geometric stable
distributions• Other transforms
Closed to scalar multiplication
• We use the linear model Y=AX+Z • X,Z are i.i.d. hidden variables drawn from a stable distribution,
Y are the observations
• Inference is computed by
• The problem: stable distribution has no closed-form cdf nor pdf (thus Copulas or CFG can not be used)
• Solution: perform inference in the characteristic function (Fourier) domain
Inference with Heavy-Tails in Linear ModelsDanny Bickson and Carlos Guestrin
• Network flows are linear• Total flow at a node composed of sums of distinct flows
• The challenge: how to model heavy tailed network traffic?
Motivation: Large Scale Network modeling• Huge amounts of data.• Daily stats collected from the PlanetLab network using
PlanetFlow:• 662 PlanetLab nodes spread over the world• 19,096,954,897 packets were transmitted • 10,410,216,514,054 bytes where transmitted• 24,012,123 unique IP addresses observed
Bandwidth distribution is heavy tailed: 1% of the top flows are 19% of the total traffic
Bandwidth/port number distribution is heavy tailed
Heavy-tailed traffic distribution
• Use linear multivariate statistical methods for network modeling, monitoring, performance analysis and intrusion detection.
• Typically can not be computed in closed-form. Various approximations: Mixtures of distributions [Chen-Infocom07] , Histograms [Lakhina-Sigcomm05], Sketches [Li-IMC06], Entropy [Lakhina-Sigcomm05], Sampled moments [Nguyen-IMC07], Etc.
Previous approaches for computing inference in heavy-tailed linear models
Output: Posterior marginal
Exact inference
Input: Prior marginal
Quantization Fitting Resampling NBP output
Main contribution• First to compute exact inference in linear-stable model
conveniently in closed-form.• Efficient iterative approximate inference.• Our solution is:
• More efficient• More accurate• Requires less memory/ storage
Stable distribution
Characteristic exponent
Skew Scale Shift
),,,( S
• A family of heavy tailed distributions.• Used in different problem domains: economics, physics, geology etc.• Example: Cauchy, Gaussian and Levy distributions are stable.
Closed to addition
• Related work on linear models: • Convolutional factor graphs (CFG) – [Mao-Tran-Info-Theory-03].
Assumes pdf factorizes as a convolution of factors (shows this is possible for any linear model)
• Copula method – handles linear model in the cdf domain• Independent components analysis (ICA) - learns linear models
and tries to reconstruct X. Can be used as a complimentary method, since we assume that A is given.
Non-parametric BP (NBP) [Sudderth-CVPR03]
Linear characteristic graphical models (LCM)
• Given a linear model, we define LCM as the product of the joint characteristic functions for the probability distribution
• Motivation: LCM is the dual model to the convolution representation of the linear model
• Unlike CFG, LCM is always defined, for any distribution
• CFG shows that Any linear model can be represented as a convolution
Linearity of stable distribution
Modeling network flows using stable distributions
• Our goal is to compute the posterior marginal p(x|y)
• Because stable distribution have no closed-form pdf, we have to compute marginalization in the Fourier domain.
• The dual operation to marginalization is slicing.
• The projection-slice theorems allows us to compute inference in the Fourier domain:
Inference in the Fourier domain
Slicing operation
2D Fourier transform
Marginalization Difficult!
Our goal
InverseFourier
Posterior marginal
2D Characteristic function
• LCM-Elimination: Exact inference algorithm for a general linear model
• Variable elimination algorithm in the Fourier domain
• Borrows ideas from belief propagation to compute approximate inference in the Fourier domain
• Uses distributivity of the slice and product operations
• Algorithm is exact on trees
Exact inference in LCM
Main result 1: exact inference in LCM with stable distributions
Main result 2: approximate inference in LCM with stable distributions
Approximate inference in LCM
Application: network monitoring
• We model PlanetLab networks flows using a LCM with stable distributions.• Extracted traffic flows from 25 Jan 2010: Total of 247,192,372 flows (non-zero entries of the matrix A)• Fitted flows for each node (vector b) total of 16,741,746 unique nodes
• Computing the posterior marginal p(x|y)
• Cost of elimination is too high O(16M^3)
• Solution: USE Stable Jacobi with GRAPHLAB!
Stable-Jacobi approximate inference algorithm
Speedup Accuracy
Marginal characteristic function
Acknowledgements
This research was supported by:• ARO MURI W911NF0710287• ARO MURI W911NF0810242 • NSF Mundo IIS-0803333 • NSF Nets-NBD CNS-0721591.
Application: multiuser detection
Lower BER (bit error rate) is better
Conclusion
Number of packets is heavy tailed [Lakhina– Sigcomm 2005]
Num
ber o
f pac
kets
Slicing operation
Running time
Difficulties in previous approximations