closing remarks: what can we do with multiple diverse...
TRANSCRIPT
![Page 1: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/1.jpg)
CVPR 2013 Diversity Tutorial
Closing Remarks: What can we do with multiple
diverse solutions?
Dhruv Batra Virginia Tech
![Page 2: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/2.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 2
CRF
. . .
Diverse Segmentations
Example Result
Now what?
![Page 3: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/3.jpg)
CVPR 2013 Diversity Tutorial
Your Options • Nothing
– User in the loop
• (Approximate) Min Bayes Risk – Use solutions to estimate the distribution and optimize Bayes
Risk
• Re-ranking – Pick a good solution from the list
(C) Dhruv Batra 3
Increasing Side Information
![Page 4: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/4.jpg)
CVPR 2013 Diversity Tutorial
Interactive Segmentation • Setup
– Model: Color/Texture + Potts Grid CRF – Inference: Graph-cuts – Dataset: 50 train/val/test images
(C) Dhruv Batra 4
Image + Scribbles Diverse 2nd Best 2nd Best MAP MAP
1-2 Nodes Flipped 100-500 Nodes Flipped
![Page 5: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/5.jpg)
CVPR 2013 Diversity Tutorial
Interactive Segmentation
(C) Dhruv Batra 5
89%
90%
91%
92%
93%
94%
95%
96%
MAP M-Best-MAP Confidence DivMBest
+0.05%
+1.61%
+3.62%
(Oracle) (Oracle) (Oracle)
M=6
Seg
men
tatio
n A
ccur
acy
Better
![Page 6: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/6.jpg)
CVPR 2013 Diversity Tutorial
Your Options • Nothing
– User in the loop
• (Approximate) Min Bayes Risk – Use solutions to estimate the distribution and optimize Bayes
Risk
• Re-ranking – Pick a good solution from the list
(C) Dhruv Batra 6
![Page 7: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/7.jpg)
CVPR 2013 Diversity Tutorial
Statistics 101 • Loss
– PCP, Pascal Loss, etc
• “True” Distribution
• Expected Loss:
• Min Bayes Risk
(C) Dhruv Batra 7
XMAP
X
P (X )P (ygt | x) =
L(ygt, y)
BR(y) = EP
�L(ygt, y)
�
�
ygt∈Y
L(ygt, y)P (ygt)miny∈Y
![Page 8: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/8.jpg)
CVPR 2013 Diversity Tutorial
Structured Output Problems • Min Bayes Risk
• Two Problems
• Approximate MBR:
(C) Dhruv Batra 8
Intractable Intractable
XMAP
X
P (X )
�
ygt∈Y
L(ygt, y)P (ygt)miny∈Y
miny∈DivMBest
�
ygt∈DivMBest
L(ygt, y)P (ygt)
![Page 9: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/9.jpg)
CVPR 2013 Diversity Tutorial
Semantic Segmentation • Setup
– Models: • Hierarchical CRF [Ladicky et al. ECCV ’10, ICCV ‘09]
• Second-Order Pooling [Carreira ECCV ‘12]
– Inference: • Alpha-expansion • Greedy
– Dataset: Pascal Segmentation Challenge (VOC 2012) • 20 categories + background; ~1500 train/val/test images
(C) Dhruv Batra 9
![Page 10: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/10.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 10
CRFDiverse Segmentations
Large-Margin Re-ranking
![Page 11: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/11.jpg)
CVPR 2013 Diversity Tutorial
Semantic Segmentation
(C) Dhruv Batra 11
Input MAP Best of 10-Div
![Page 12: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/12.jpg)
CVPR 2013 Diversity Tutorial
Semantic Segmentation
(C) Dhruv Batra 12
PAC
AL
Acc
urac
y
Better
#Solutions / Image
44%
47%
50%
53%
56%
59%
1 2 3 4 5 6 7 8 9 10
MAP [State-of-art circa 2012]
15%-gain possible
Same Features Same Model
DivMBest (Oracle)
Rand (Re-rank)
MBR
![Page 13: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/13.jpg)
CVPR 2013 Diversity Tutorial
Your Options • Nothing
– User in the loop
• (Approximate) Min Bayes Risk – Use solutions to estimate the distribution and optimize Bayes
Risk
• Re-ranking – Pick a good solution from the list
(C) Dhruv Batra 13
![Page 14: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/14.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 14
CRFDiverse Segmentations
Large-Margin Re-ranking
![Page 15: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/15.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 15
CRFDiverse Segmentations
Large-Margin Re-ranking
ψ( , )
![Page 16: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/16.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 16
CRFDiverse Segmentations
Large-Margin Re-ranking
ψ( , )α� −α�ψ( , )
![Page 17: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/17.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 17
CRFDiverse Segmentations
Large-Margin Re-ranking
ψ( , )α� −α�ψ( , ) ≥ 1− ξilossi
minα,ξi
||α||2 + C�
i
ξiDiscriminative Re-ranking of Diverse Segmentation
[Yadollahpour et al., CVPR13, Wednesday Poster]
![Page 18: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/18.jpg)
CVPR 2013 Diversity Tutorial
Semantic Segmentation
(C) Dhruv Batra 18
PAC
AL
Acc
urac
y
Better
#Solutions / Image
44%
47%
50%
53%
56%
59%
1 2 3 4 5 6 7 8 9 10
MAP [State-of-art circa 2012]
DivMBest (Oracle)
Rand (Re-rank)
DivMBest (Re-ranked) [Y.B.S., CVPR ‘13]
MBR
![Page 19: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/19.jpg)
CVPR 2013 Diversity Tutorial
Qualitative Results: Success
(C) Dhruv Batra 19
![Page 20: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/20.jpg)
CVPR 2013 Diversity Tutorial
Qualitative Results: Success
(C) Dhruv Batra 20
![Page 21: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/21.jpg)
CVPR 2013 Diversity Tutorial
Qualitative Results: Success
(C) Dhruv Batra 21
![Page 22: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/22.jpg)
CVPR 2013 Diversity Tutorial
Qualitative Results: Failures
(C) Dhruv Batra 22
![Page 23: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/23.jpg)
CVPR 2013 Diversity Tutorial
Qualitative Results: Failures
(C) Dhruv Batra 23
![Page 24: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/24.jpg)
CVPR 2013 Diversity Tutorial
Qualitative Results: Failures
(C) Dhruv Batra 24
![Page 25: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/25.jpg)
CVPR 2013 Diversity Tutorial
Summary • All models are wrong
• Some beliefs are useful
• Diverse Multiple Solutions – A way to get useful beliefs out.
• DivMBest + Reranking – Big impact possible on many applications!
(C) Dhruv Batra 25
![Page 26: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/26.jpg)
CVPR 2013 Diversity Tutorial
Summary
• What does my model believe?
(C) Dhruv Batra 26
�
Posterior Summary
![Page 27: Closing Remarks: What can we do with multiple diverse ...cc.gatech.edu/.../06_closing_remarks_dhruv_batra.pdfClosing Remarks: What can we do with multiple diverse solutions? Dhruv](https://reader035.vdocument.in/reader035/viewer/2022070108/602b698d97a0de5b6d798f32/html5/thumbnails/27.jpg)
CVPR 2013 Diversity Tutorial
Thanks!
(C) Dhruv Batra 27