Probabilistic Graphical Models

Andreas Geiger

Autonomous Vision GroupMPI Tubingen

Computer Vision and Geometry LabETH Zurich

January 13, 2017

Autonomous Vision GroupResearch

Dr. Andreas GeigerGroup Leader

Dr. Osman UlusoyPostdoc

Fatma GüneyPhD Student

Aseem BehlPhD Student

Lars MeschederPhD Student

Joel JanaiPhD Student

Benjamin CoorsPhD Student

Yiyi LiaoVisiting PhD Student

Gernot RieglerVisiting PhD Student

2

Autonomous Vision GroupResearch

Computer Vision Robotics

Machine LearningGraphical Models Deep Learning

2

Class Requirements & Materials

Requirements:

� Linear Algebra (matrices, vectors)

� Probability Theory (marginals, conditionals)

Materials: www.cvlibs.net/learn

3

Machine Learning

“Field of study that gives computers the ability to learn without beingexplicitly programmed.” Arthur Samuel, 1959

4

Machine Learning

5

Machine Learning

5

Machine Learning

5

Machine Learning

5

Graphical Models

6

Supervised Learning

fθInput Output

Model

� Learning: Estimate parameters θ from training data

� Inference: Make novel predictions fθ(·)

7

Classification

fθ

Input

"Beach"

Model Output

7

Structured Prediction

fθ

Input Model Outputt=1

t=2

t=3t=1 t=2 t=3x1 x2 x3

7

Vehicle Localization

lane 1lane 2lane 3

x1=? x2=? x3=? x4=? x5=? x6=? x7=? x8=? x9=? x10=?

� Goal: Estimate vehicle location at time t = 1, . . . , 10

� Variables: x = {x1, . . . , x10} xi ∈ {1, 2, 3}� Observations: y = {y1, . . . , y10} yi ∈ R3

8

Markov Random Field

f1 f2 f3g gx1 x2 x3

f10gx10

g

9

Markov Random Field

f1 f2 f3g gx1 x2 x3

f10gx10

g

pθ(x|y) =1

Z

10∏i=1

fi (xi )9∏

i=1

gθ(xi , xi+1)

9

Markov Random Field

f1 f2 f3g gx1 x2 x3

f10gx10

g

pθ(x|y) =1

Z

10∏i=1

fi (xi )9∏

i=1

gθ(xi , xi+1)

Unary Factors:

� f1(x1) =

0.70.20.1

, f2(x2) =

0.70.10.2

, f3(x3) =

0.20.10.7

, . . .

9

Markov Random Field

f1 f2 f3g gx1 x2 x3

f10gx10

g

pθ(x|y) =1

Z

10∏i=1

fi (xi )9∏

i=1

gθ(xi , xi+1)

Pairwise Factors:

� gθ(xi , xi+1) =

θ11 θ12 θ13θ21 θ22 θ23θ31 θ32 θ33

9

Markov Random Field

f1 f2 f3g gx1 x2 x3

f10gx10

g

pθ(x|y) =1

Z

10∏i=1

fi (xi )9∏

i=1

gθ(xi , xi+1)

Pairwise Factors:

� gθ(xi , xi+1) =

θ11 θ12 θ13θ21 θ22 θ23θ31 θ32 θ33

� Learning Problem:

θ∗ = argmaxθ

∏n

pθ(xn|yn)

9

Markov Random Field

f1 f2 f3g gx1 x2 x3

f10gx10

g

pθ(x|y) =1

Z

10∏i=1

fi (xi )9∏

i=1

gθ(xi , xi+1)

Pairwise Factors:

� gθ(xi , xi+1) =

0.8 0.2 0.00.2 0.6 0.20.0 0.2 0.8

� Change 2 lanes: 0%

� Change 1 lane: 20%

� Otherwise: stay on lane

9

Inference

f1 f2 f3g gx1 x2 x3

f10gx10

g

� Maximum-A-Posteriori State:

x1, . . . , x10 = argmaxx1,...,x10

pθ(x1, . . . , x10|y)

� Marginal Distribution:

p(x1) =∑x2

∑x3

· · ·∑x10

pθ(x1, . . . , x10|y)

� Complexity?

10

Inference

f1 f2 f3g gx1 x2 x3

f10gx10

g

� Maximum-A-Posteriori State:

x1, . . . , x10 = argmaxx1,...,x10

pθ(x1, . . . , x10|y)

� Marginal Distribution:

p(x1) =∑x2

∑x3

· · ·∑x10

pθ(x1, . . . , x10|y)

� Complexity?

10

Inference

f1 f2 f3g gx1 x2 x3

f10gx10

g

� Maximum-A-Posteriori State:

x1, . . . , x10 = argmaxx1,...,x10

pθ(x1, . . . , x10|y)

� Marginal Distribution:

p(x1) =∑x2

∑x3

· · ·∑x10

pθ(x1, . . . , x10|y)

� Complexity?

10

Inference

f1 f2 f3g gx1 x2 x3

f10gx10

g

� Maximum-A-Posteriori State:

x1, . . . , x10 = argmaxx1,...,x10

pθ(x1, . . . , x10|y)

� Marginal Distribution:

p(x1) =∑x2

∑x3

· · ·∑x10

pθ(x1, . . . , x10|y)

� Complexity? O(#states#nodes) Here: 310 = 59049

10

Inference

f1 f2 f3g gx1 x2 x3

p(x1, x2, x3) =1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

11

Inference

f1 f2 f3g gx1 x2 x3

p(x1, x2, x3) =1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

p(x1) =∑x2,x3

1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

11

Inference

f1 f2 f3g gx1 x2 x3

p(x1, x2, x3) =1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

p(x1) =∑x2,x3

1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

=1

Zf1(x1)

∑x2

f2(x2) g(x1, x2)∑x3

f3(x3) g(x2, x3)

11

Inference

f1 f2 f3g gx1 x2 x3

μ(x2)

p(x1, x2, x3) =1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

p(x1) =∑x2,x3

1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

=1

Zf1(x1)

∑x2

f2(x2) g(x1, x2)∑x3

f3(x3) g(x2, x3)︸ ︷︷ ︸µ(x2)

11

Inference

f1 f2 f3g gx1 x2 x3

μ(x2)

p(x1, x2, x3) =1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

p(x1) =∑x2,x3

1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

=1

Zf1(x1)

∑x2

f2(x2) g(x1, x2)µ(x2)

11

Inference

f1 f2 f3g gx1 x2 x3

μ'(x1)

p(x1, x2, x3) =1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

p(x1) =∑x2,x3

1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

=1

Zf1(x1)

∑x2

f2(x2) g(x1, x2)µ(x2)︸ ︷︷ ︸µ′(x1)

11

Inference

f1 f2 f3g gx1 x2 x3

μ'(x1)

p(x1, x2, x3) =1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

p(x1) =∑x2,x3

1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

=1

Zf1(x1)µ

′(x1)

11

Inference

f1 f2 f3g gx1 x2 x3

μ'(x1)

p(x1, x2, x3) =1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

p(x1) =∑x2,x3

1

Zf1(x1) f2(x2) f3(x3) g(x1, x2) g(x2, x3)

=1

Zf1(x1)µ

′(x1)

Marginal = Product of Factor & Incoming Messages

11

J. Pearl: Reverend Bayes on inference engines:A distributed hierarchical approach. AAAI, 1982.

12

Inference

Observations

lane 1lane 2lane 3

Try yourself: www.cvlibs.net/learn

13

Inference

Marginal Distributions

lane 1lane 2lane 3

Try yourself: www.cvlibs.net/learn

13

Inference

Maximum-A-Posteriori State

lane 1lane 2lane 3

Try yourself: www.cvlibs.net/learn

13