structured models for image segmentationpeople.inf.ethz.ch/alucchi/talks/berkeley_130213.pdf ·...
TRANSCRIPT
![Page 1: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/1.jpg)
1
Structured Models for Image Segmentation
Aurelien Lucchi
Wednesday February 13th, 2013
Joint work with Yunpeng Li, Kevin Smith, Raphael Sznitman, Radhakrishna Achanta, Bohumil Maco,
Graham Knott, Pascal Fua.
![Page 2: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/2.jpg)
2
Image Segmentation
● Goal: partition an image into meaningful regions with respect to a particular application.
![Page 3: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/3.jpg)
3
Image Segmentation
● Goal: partition an image into meaningful regions with respect to a particular application.
![Page 4: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/4.jpg)
4
Understanding the Brain
![Page 5: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/5.jpg)
5
Electron Microscopy Data
5 × 5 × 5 μm section taken from the CA1 hippocampus, corresponding to a 1024 × 1024 × 1000 volume (N ≈ 109 total voxels)
● Human brain contains ~100 billion (1011) neurons and 100 trillion (1014) synapses.
![Page 6: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/6.jpg)
6
Image Segmentation
Statistics
![Page 7: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/7.jpg)
7
Image Segmentation
Featureextraction
ClassificationStructuredprediction
ground-truth
![Page 8: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/8.jpg)
8
Image Segmentation
Featureextraction
ClassificationStructuredprediction
ground-truth
![Page 9: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/9.jpg)
9
Outline
1. CRF for Image segmentation
2. Maximum Margin Training for CRFs - Cutting Plane (Structured SVM)
3. Maximum Margin Training of CRFs - Online Subgradient Descent (SGD)
4. SLIC superpixels/supervoxels
![Page 10: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/10.jpg)
10
1. CRF for Image Segmentation
![Page 11: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/11.jpg)
11
Structured Prediction
● Non structured output
● inputs X can be any kind of objects● output y is a real number
● Prediction of complex outputs
● Structured output y is complex (images, text, audio...)● Ad hoc definition of structured data: data that consists of several parts, and not
only the parts themselves contain information, but also the way in which the parts belong together
Slide courtesy: Christoph Lampert
![Page 12: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/12.jpg)
12
Structured Prediction for Images
Histograms, Filter responses, ...
![Page 13: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/13.jpg)
13
CRF for Image Segmentation
Pair-wise Terms MAP SolutionUnary likelihoodData (D)
Maximum-a-posteriori (MAP) solution :
Boykov and Jolly [ICCV 2001], Blake et al. [ECCV 2004]Slide courtesy : Pushmeet Kohli
![Page 14: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/14.jpg)
14
CRF for Image Segmentation
Pair-wise Terms MAP SolutionUnary likelihoodData (D)
Maximum-a-posteriori (MAP) solution :
Boykov and Jolly [ICCV 2001], Blake et al. [ECCV 2004]Slide courtesy : Pushmeet Kohli
![Page 15: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/15.jpg)
15
CRF for Image Segmentation
Pair-wise Terms
Favors the same label for neighboring nodes.
![Page 16: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/16.jpg)
16
CRF for Image Segmentation
Pair-wise Terms MAP SolutionUnary likelihoodData (D)
Maximum-a-posteriori (MAP) solution :
Boykov and Jolly [ICCV 2001], Blake et al. [ECCV 2004]Slide courtesy : Pushmeet Kohli
![Page 17: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/17.jpg)
17
Energy Minimization
● MAP inference for discrete graphical models:
● Dynamic programming– Exact on non loopy graphs
● Graph-cuts (Boykov, 2001)– Optimal solution if energy function is submodular
● Belief propagation (Pearl, 1982)– No theoretical guarantees on loopy graphs but seems to work well in
practice.● Mean field (root in statistical physics)● ...
![Page 18: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/18.jpg)
18
Training a Structured Model
● First rewrite the energy function as:
where w is a vector of parameters to be learned from training data and is a joint feature map to map the input-output pair into a linear feature space.
Log-linear model
![Page 19: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/19.jpg)
19
Training a Structured Model
● Energy function is parametrized by vector w
-1 1
-1 ? ?
1 ? ?+
![Page 20: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/20.jpg)
20
Training a Structured Model
● Energy function is parametrized by vector w
-1 1
-1 0 1
1 1 0+
Low energy
High energy
![Page 21: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/21.jpg)
21
Training a Structured Model
● Maximum likelihood● Pseudo-likelihood● Variational approximation● Contrastive divergence● Maximum-margin framework
![Page 22: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/22.jpg)
22
2. Maximum Margin Training for CRFs - Cutting Plane (Structured SVM)
![Page 23: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/23.jpg)
23
Structured SVM
● Given a set of N training examples with ground truth labels , we can write
≡
Energy for the correct labeling at least as low as energy of any incorrect labeling.
![Page 24: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/24.jpg)
24
Structured SVM
E( )
E( )
E( )
E( )
<ground-truth
![Page 25: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/25.jpg)
25
Structured SVM
● Given a set of N training examples with ground truth labellings we optimize :
![Page 26: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/26.jpg)
26
Structured SVM
● In order to deal with the exponential number of constraints, Tsochantaridis* proposed a cutting plane algorithm.● Iteratively finds the most violated constraint and
adds it to the working set of constraints.
* I. Tsochantaridis et al., Support vector machine learning for interdependent and structured output spaces, ICML, 2004.
Loss-augmented inference
![Page 27: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/27.jpg)
27
Illustrative Example
SSVM Problem● Exponential constraints● Most are dominated by a small set of
“important” constraints
Cutting plane approach● Repeatedly finds the next most
violated constraint…● …until set of constraints is a good
approximation.
An Introduction to Structured Output Learning Using Support Vector MachinesYisong Yue, Thorsten Joachims
![Page 28: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/28.jpg)
28
Illustrative Example
SSVM Problem● Exponential constraints● Most are dominated by a small set of
“important” constraints
Cutting plane approach● Repeatedly finds the next most
violated constraint…● …until set of constraints is a good
approximation.
An Introduction to Structured Output Learning Using Support Vector MachinesYisong Yue, Thorsten Joachims
![Page 29: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/29.jpg)
29
Illustrative Example
SSVM Problem● Exponential constraints● Most are dominated by a small set of
“important” constraints
Cutting plane approach● Repeatedly finds the next most
violated constraint…● …until set of constraints is a good
approximation.
An Introduction to Structured Output Learning Using Support Vector MachinesYisong Yue, Thorsten Joachims
![Page 30: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/30.jpg)
30
Illustrative Example
SSVM Problem● Exponential constraints● Most are dominated by a small set of
“important” constraints
Cutting plane approach● Repeatedly finds the next most
violated constraint…● …until set of constraints is a good
approximation.
An Introduction to Structured Output Learning Using Support Vector MachinesYisong Yue, Thorsten Joachims
![Page 31: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/31.jpg)
31
Drawbacks of SSVM
● Finding the most violated constraint at each iteration of the cutting plane is intractable in loopy graphical models.
● Approximations can sometimes be imprecise enough to have a major impact on learning● An unsatisfactory constraint can cause the cutting
plane algorithm to prematurely terminate.
![Page 32: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/32.jpg)
32
3. Maximum Margin Training of Structured Models: Online Subgradient
Descent (SGD)
![Page 33: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/33.jpg)
33
SGD Approach
● Reformulate the problem as an unconstrained optimization:
● SGD approach : compute and step in the negative direction of a sub-gradient of
● See N. Ratliff et al., (Online) Subgradient Methods for Structured Prediction. In AISTATS, 2007.
![Page 34: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/34.jpg)
34
Subgradient
● A subgradient for the convex loss function L at w is defined as a vector g, such that:
w
● Set of all subgradients is called the subdifferential.
● If L is differentiable at w, then g is the gradient .
![Page 35: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/35.jpg)
35
Subgradient
● How to compute a subgradient?
● Subgradient can be obtained by computing
at
Loss-augmented inference
![Page 36: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/36.jpg)
36
SGD approach
![Page 37: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/37.jpg)
37
SGD Approach
● Convergence guarantees:
Goes to 0 with appropriate step size
![Page 38: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/38.jpg)
38
What can we say about Approximate Subgradients ?
● Epsilon subgradients:
w
ε
![Page 39: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/39.jpg)
39
What can we say about Approximate Subgradients ?
● Convergence guarantees (see S. M. Robinson. Linear convergence of epsilon-subgradient descent methods for a class of convex functions. Mathematical Programming, 86:41–50, 1999):
Goes to 0 with appropriate step size
![Page 40: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/40.jpg)
40
Proposed Algorithm
● Goal: better estimate the subgradient by using working sets of constraints, denoted .
● Algorithm:● First solves the loss-augmented inference to find a
constraint and add it to the working set .● It then steps in the opposite direction of the
subgradient computed as an average over the set of violated constraints belonging to .
![Page 41: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/41.jpg)
41
Proposed Algorithm
![Page 42: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/42.jpg)
42
EM Segmentation ResultsSegmentation performance measured with the Jaccard index.
[Lucchi] Supervoxel-Based Segmentation of Mitochondria in EM Image Stacks with Learned Shape Features, A. Lucchi et al.[SGD+inference] (Online) Subgradient Methods for Structured Prediction, N. Ratliff et al.[SSVM] Support Vector Machine Learning for Interdependent and Structured Output Spaces, I. Tsochantaridis et al.[Samplerank] Samplerank: Training Factor Graphs with Atomic Gradients, M. Wick et al.
![Page 43: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/43.jpg)
43
MSRC Segmentation Results
[Ladicky] What, Where and How Many? Combining Object Detectors and Crfs, L. Ladicky et al.[SGD+inference] (Online) Subgradient Methods for Structured Prediction, N. Ratliff et al.[SSVM] Support Vector Machine Learning for Interdependent and Structured Output Spaces, I. Tsochantaridis et al.[Yao] Describing the Scene as a Whole: Joint Object Detection, Scene Classification and Semantic Segmentation, J. Yao et al.
All the methods in the top part of the table were optimized for the average score.
![Page 44: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/44.jpg)
44
MSRC Segmentation Results
[SGD+inference] (Online) Subgradient Methods for Structured Prediction, N. Ratliff et al.
![Page 45: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/45.jpg)
45
Time Analysis
● Sampling can replace the more expensive inference step without much performance loss, leading to significantly lower learning time.
[SGD+inference] (Online) Subgradient Methods for Structured Prediction, N. Ratliff et al.[Samplerank] Samplerank: Training Factor Graphs with Atomic Gradients, M. Wick et al.
Running time for T = 1000 iterations.
Computational overhead = increase in time resulting from the working set.
![Page 46: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/46.jpg)
46
Summary
● Structured learning makes it possible to jointly learn the model parameters.
● Used a working sets of constraints to produce better subgradient estimates and higher-quality solutions
![Page 47: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/47.jpg)
47
Future Challenges
● Better energy functions, fully connected CRFs...● High order potentials
Increasing connectivity
![Page 48: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/48.jpg)
48
4. SLIC superpixels/supervoxels
SLIC Superpixels Compared to State-of-the-art Superpixel Methods, IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2012R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua and S. Süsstrunk.
![Page 49: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/49.jpg)
49
SLIC Superpixels
● Clusters pixels in the combined five-dimensional color and image plane space.
● Efficiently generate compact, nearly uniform superpixels.
SLIC Superpixels Compared to State-of-the-art Superpixel Methods, IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2012.Source code available online.
![Page 50: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/50.jpg)
50
SLIC Supervoxels
Supervoxel-Based Segmentation of Mitochondria in EM Image Stacks with Learned Shape Features,IEEE Transactions on Medical Imaging, 2011, A. Lucchi, K.Smith, R. Achanta, G. Knott, P. Fua.
![Page 51: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/51.jpg)
51
SLIC Superpixels
Under-segmentation error = error with respect to a known ground truth.
Boundary recall = fraction of ground truth edges fall within one pixel of a least one superpixel boundary.
● GS04: Efficient Graph-Based Image Segmentation, P. Felzenszwalb and D. Huttenlocher.● NC05: Normalized Cuts and Image Segmentation, J. Shi and J. Malik.● QS09: Quick Shift and Kernel Methods for Mode Seeking, A. Vedaldi et al.● TP09: Turbopixels: Fast Superpixels Using Geometric Flows, A. Levinshtein et al.
SLIC
SLIC
![Page 52: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/52.jpg)
52
SLIC Superpixels
● GKM: 10 iterations of k-means.● GS04: Efficient Graph-Based Image Segmentation, P. Felzenszwalb and D. Huttenlocher.● NC05: Normalized Cuts and Image Segmentation, J. Shi and J. Malik.● QS09: Quick Shift and Kernel Methods for Mode Seeking, A. Vedaldi et al.● TP09: Turbopixels: Fast Superpixels Using Geometric Flows, A. Levinshtein et al.
![Page 53: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/53.jpg)
53
SLIC Superpixels
● Simple approach but successful: more than 100 citations, re-implemented in vlfeat and scikit + GPU implementation.
● Code: http://cvlab.epfl.ch/~lucchi/code.php
![Page 54: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/54.jpg)
54
Questions
![Page 55: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/55.jpg)
55
Resources
● http://cvlab.epfl.ch/research/medical/em/mitochondria/● http://cvlab.epfl.ch/research/medical/em/synapses/● http://cvlab.epfl.ch/~lucchi/
![Page 56: Structured Models for Image Segmentationpeople.inf.ethz.ch/alucchi/talks/Berkeley_130213.pdf · 2015. 5. 12. · Structured Models for Image Segmentation Aurelien Lucchi Wednesday](https://reader035.vdocument.in/reader035/viewer/2022071404/60f8ce28f0bb4431b0210499/html5/thumbnails/56.jpg)
56
Credits
● Slides courtesy :● Christoph Lampert (Learning with Structured Inputs
and Outputs)● Pushmeet Kohli (Efficiently Solving Dynamic
Markov Random Fields using Graph Cuts)● Ben Taskar (Structured Prediction: A Large Margin
Approach, NIPS tutorial)● Yisong Yue, Thorsten Joachims (An Introduction to
Structured Output Learning Using Support Vector Machines)