active detection - eth z · receiving observation earlier (i.e., at an ancestor) only increases its...
TRANSCRIPT
![Page 1: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/1.jpg)
Active Detection via Adaptive Submodularity
Yuxin Chen†, Hiroaki Shioi
‡, Cesar Antonio Fuentes Montesinos
†
! Lian Pin Koh
†, Serge Wich
¶ and Andreas Krause
†
!| ICML Beijing | June 23, 2014 |
† ‡ !¶
![Page 2: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/2.jpg)
Motivating Example: Biodiversity Monitoring
Application: Detecting Orangutan nests
![Page 3: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/3.jpg)
![Page 4: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/4.jpg)
���4
Automatic, Open-loop Computer Vision System
85% recall 10% precision ☹
0.5 0.6 0.7 0.8 0.9 10
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
recall
prec
isio
n
![Page 5: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/5.jpg)
���5
✔
✗
Interactive Detection
Is there a nest at location (x,y)?
YesNo
No YesNo Yes
No Yes
The adaptive policy
✗
How can human experts best help the detection task?
![Page 6: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/6.jpg)
���6
Open-loop (Passive) System
✔✗ ✗ ✗ ✗ ✔ ✗ ✔
✗
Closed-loop (Active) System
✔ ✔ ✗ ✔ ✗ ✗ ✗
![Page 7: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/7.jpg)
Evidence for Detection
Train classifier (e.g., SVM; conv. Neural Network, etc.) on 45 positive and 148 negative examples Use sliding window to produce “response images” Which detection should be proposed next? ���7
![Page 8: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/8.jpg)
Votes and Hypotheses
���8
Voting elements
Hypotheses
Interactions between voting elements and hypotheses: G = (V,H, E)
1 2 3 4 5 6 7 8
1 2 3 4H = {h1, . . . , h4}
V = {v1, . . . , v8}
Barinova
Evidence for Detection
Train classifier (e.g., SVM; conv. Neural Network, etc.) on 45 positive and 148 negative examples!Use sliding window to produce “response images”!Which detection should be proposed next?
Evidence for Detection
Train classifier (e.g., SVM; conv. Neural Network, etc.) on 45 positive and 148 negative examples!Use sliding window to produce “response images”!Which detection should be proposed next?
Evidence for Detection
Train classifier (e.g., SVM; conv. Neural Network, etc.) on 45 positive and 148 negative examples!Use sliding window to produce “response images”!Which detection should be proposed next?
Evidence for Detection
Train classifier (e.g., SVM; conv. Neural Network, etc.) on 45 positive and 148 negative examples!Use sliding window to produce “response images”!Which detection should be proposed next?
Evidence for Detection
Train classifier (e.g., SVM; conv. Neural Network, etc.) on 45 positive and 148 negative examples!Use sliding window to produce “response images”!Which detection should be proposed next?
Evidence for Detection
Train classifier (e.g., SVM; conv. Neural Network, etc.) on 45 positive and 148 negative examples!Use sliding window to produce “response images”!Which detection should be proposed next?
Evidence for Detection
Train classifier (e.g., SVM; conv. Neural Network, etc.) on 45 positive and 148 negative examples!Use sliding window to produce “response images”!Which detection should be proposed next?
Evidence for Detection
Train classifier (e.g., SVM; conv. Neural Network, etc.) on 45 positive and 148 negative examples!Use sliding window to produce “response images”!Which detection should be proposed next?
(x2, y2) (x3, y3) (x4, y4)(x1, y1)
[Hough ’59; Gall et al, ’09; Barinova et al, ’11]
![Page 9: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/9.jpg)
Active Detection as an Adaptive Optimization Problem
���9
Positive coverage: Votes can be fully explained /covered by a true hypotheses.
1 2 3 4 5 6 7 8
1 2 3 4
1 2 3 4 5 6 7 8
1 2 3 4
1 2 3 4 5 6 7 8
1 2 3 4
1 2 3 4 5 6 7 8
1 2 3 4
1 2 3 4 5 6 7 8
1 2 3 4
1 2 3 4 5 6 7 8
1 2 3 4
1 2 3 4 5 6 7 8
1 2 3 4
Assume that each vote carries unit weight
Now we observe that hypotheses 3 is true.
Then we observe that hypotheses 1 is true.
Then we observe that hypotheses 4 is true.
![Page 10: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/10.jpg)
Active Detection as an Adaptive Optimization Problem
���10
Negative coverage: Votes that are similar with false votes should be discounted.
1 2 3 4 5 6 7 8
1 2 3 4
1 2 3 4 5 6 7 8
1 2 3 4
1 2 3 4 5 6 7 8
1 2 3 4
Assume that each vote carries unit weight
Now we observe that hypotheses 3 is false.
We also need to discount the votes that are similar
with the false votes!!
Suppose that we can cluster similar votes.
1 2 3 4 5 6 7 8
1 2 3 4
1 2 3 4 5 6 7 8
1 2 3 4
![Page 11: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/11.jpg)
The general case: Real-votes setting
���11
1 2 3 4 5 6 7 8
1 2 3 4
122 6
814
97 2
128 82
1 2 3 4 5 6 7 8
1 2 3 4
62 6
414
97 1
64 42
Voting elements with real-value votes
Now observe that hypothesis 3 is false
![Page 12: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/12.jpg)
Active Detection in a Nutshell
���12
1 2 3 4 5 6 7 8
1 2 3 4Positive observations: Votes can be fully explained /covered by a true hypotheses.
Negative observations: Votes that are similar with false votes should be discounted.
real-value
The Objective
Coverage for edge (v,h) Coverage due to negative observations+Coverage due to
positive observations=
Coverage of G = (V,H, E) = Coverage for edge (v,h)X
(v,h)
![Page 13: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/13.jpg)
13
Diminishing Evidence in Detection
Positive observations explain “response” in local areas Negative observations explain “response” in similar areas
Adaptive submodular objective can capture this diminishing returns effect
Say how can we find a good policy??
![Page 14: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/14.jpg)
���14
s
s
Gain less
Gain more
selected hypotheses
stochastic outcome
Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit.
taken over its outcome
Adaptive Submodularity [Golovin & Krause, 2011]
![Page 15: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/15.jpg)
Greedy vs. Optimal
���15
Cost of the Greedy algorithm w.r.t. F
Cost of optimal policy
. . .. . .
. . .
�����
�����
�����
�����⇣ln
Q
�+ 1
⌘·
Assume that:
‣ The optimal policy achieves a maximum coverage of Q
‣ The greedy policy achieves a maximum coverage of Q-β
Don’t flip between average and worst-case? More intuition
![Page 16: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/16.jpg)
16
[Submodular, Barinova et al‘12]
Active detection improves precision and recall
Detection Results
100 200 300 400 500 600 70030
32
34
36
38
40
42
44
Labeling effort
#Det
ectio
ns
100 200 300 400 500 600 70030
32
34
36
38
40
42
44
Passive
100 200 300 400 500 600 70030
32
34
36
38
40
42
44
Active
![Page 17: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/17.jpg)
���17
“complicated base detectors”
TUD-pedestrian: Pedestrian Detection
![Page 18: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/18.jpg)
���18
Votes and Hypotheses Hough-forest Based Detector
1 5 6
4
3 2
h1 h2
Original ImageResponse Image
[Hough-forest, Gall et al, CVPR’09]
![Page 19: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/19.jpg)
���19
Cyan box: current detection. Red boxes: ground-truth labels of pedestrians. Green boxes: detections made by the active detector.
0.4 0.6 0.8 1
0.5
0.6
0.7
0.8
0.9
1
recall
prec
isio
n
Passive (Barinova et al. ’12)
Active
TUD-pedestrian: Detection Results
![Page 20: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/20.jpg)
���20
!
PASCAL 2008 - Person Category Deformable Parts Model (DPM)
State of the art??
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
recall
precision
Passive (Felzenszwalb et al. ’10)
Passive
Active
![Page 21: Active Detection - ETH Z · Receiving observation earlier (i.e., at an ancestor) only increases its expected marginal benefit. taken over its outcome Adaptive Submodularity [Golovin](https://reader033.vdocument.in/reader033/viewer/2022052008/601d2637a8327676a4570732/html5/thumbnails/21.jpg)
Come to our poster on Tuesday for more details !
Conclusion
Conclusions; no thanks;
An active detection framework that enables turning existing base detectors into systems that intelligently interact with users. !We show that the objective function satisfies adaptive submodularity, allowing us to use efficient greedy algorithms, with strong theoretical guarantees. !We demonstrate the effectiveness of the active detection algorithm on three different real-world object detection tasks.