“secret” of object detection
DESCRIPTION
“Secret” of Object Detection. Zheng Wu (Summer intern in MSRNE) Sep. 3, 2010 Joint work with Ce Liu (MSRNE) William T. Freeman (MIT) Adam Kalai (MSRNE) Jianxiong Xiao (MIT). person. motorbike. Outline. Introduction Sliding-Window-based Object Detection Window Generation - PowerPoint PPT PresentationTRANSCRIPT
“Secret” of Object DetectionZheng Wu
(Summer intern in MSRNE)Sep. 3, 2010
Joint work with Ce Liu (MSRNE)
William T. Freeman (MIT)Adam Kalai (MSRNE)Jianxiong Xiao (MIT)
Intro Sliding Windows Features Cascade Classifier PASCAL Con 2
motorbike
person
Intro Sliding Windows Features Cascade Classifier PASCAL Con 3
Outline IntroductionSliding-Window-based Object Detection
Window GenerationFeature ExtractionCascade Classifier
PASCAL ChallengeConclusion
Intro Sliding Windows Features Cascade Classifier PASCAL Con 4
Related Topic Image Matching
Image Classification
Object Detection
Intro Sliding Windows Features Cascade Classifier PASCAL Con
Object Detection
• Single Pattern • Multiple Patterns
5
Viola & Jones Face Detector
Dalal & Triggs Pedestrian Detector
Felzenszwalb’s Part-based Detector
Intro Sliding Windows Features Cascade Classifier PASCAL Con 6
Outline IntroductionSliding-Window-based Object Detection
Window GenerationFeature ExtractionCascade Classifier
PASCAL ChallengeConclusion
Intro Sliding Windows Features Cascade Classifier PASCAL Con 7
Sliding Windows
Search over:• Location• Scale• Aspect Ratio
Millions of windows!
Intro Sliding Windows Features Cascade Classifier PASCAL Con 8
Sliding Windows Subsample on grid
- have to set “optimal” step size manually Fix Aspect Ratio
- assume single pattern detection Fix Scale
- assume object’s resolution does not change much between training and test sets.
Search with branch-and-bound method- have to use special scoring function
Intro Sliding Windows Features Cascade Classifier PASCAL Con 9
Sliding Windows We propose sliding windows from segmentation
Superpixel Segmentation[Levinshtein et al, PAMI09]
Region Segmentation[Felzenszwalb & Huttenlocher, IJCV03]
< 100,000 sliding windows / image on PASCAL Dataset
Intro Sliding Windows Features Cascade Classifier PASCAL Con 10
Outline IntroductionSliding-Window-based Object Detection
Window GenerationFeature ExtractionCascade Classifier
PASCAL ChallengeConclusion
Intro Sliding Windows Features Cascade Classifier PASCAL Con 11
Generic Feature “Objectness” features [Alexe et al, CVPR10]
Intro Sliding Windows Features Cascade Classifier PASCAL Con 12
Generic Feature Each type of generic feature is weak, but
combination is stronger Low dimensional feature (=8) Not suitable for objects with “concave” shape, i.e.
table, chair
Intro Sliding Windows Features Cascade Classifier PASCAL Con 13
Generic Feature
aerop
lane
bicyc
le bird
boat
bottle bu
s car
cat
chair co
w
dining
table do
gho
rse
motorbi
ke
perso
n
potte
dplan
t
shee
pso
fatra
in
tvmon
itor
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1re
call
Intro Sliding Windows Features Cascade Classifier PASCAL Con 14
Class-specific Feature Histogram of Orientated Gradients
Intro Sliding Windows Features Cascade Classifier PASCAL Con 15
Class-specific Feature Dense grid (>10*10) (secret 1) Overlapping cells Histogram bin size High dimensional feature (>1000)
- redundant or overfitting? Normalization No spatial relationship maintained
Intro Sliding Windows Features Cascade Classifier PASCAL Con 16
Outline IntroductionSliding-Window-based Object Detection
Window GenerationFeature ExtractionCascade Classifier
PASCAL ChallengeConclusion
Intro Sliding Windows Features Cascade Classifier PASCAL Con 17
Cascade Classifier Same type of classifier with different features
Viola & Jones Face Detector, IJCV01 Different types of classifier with same features
Harzallah et al, ICCV09 (INRIA) Vedaldi et al, ICCV09 (Oxford)
Intro Sliding Windows Features Cascade Classifier PASCAL Con 18
Cascade Classifier Training SVM is slow…
to train 20,000 examples with 4000 dimensions:>15min for Linear SVM
>3 hours for Nonlinear SVM Training SVM requires a lot of memory…
design matrix: 20,000*20,000 matrix Training with Imbalance data a few hundreds of positive examples billions of negative examples
Intro Sliding Windows Features Cascade Classifier PASCAL Con 19
Boosted SVM
Intro Sliding Windows Features Cascade Classifier PASCAL Con 20
42327 examples, half for training, half for testing Training error is 0.05 for all boosted classifiers
Boosted SVM
Intro Sliding Windows Features Cascade Classifier PASCAL Con 21
Positive Training Set
Intro Sliding Windows Features Cascade Classifier PASCAL Con 22
Negative Training Set
Random Samples
SVM ver. 1
Training Sample Pool
False Positives
SVM ver. 2
… … Secret 2
Intro Sliding Windows Features Cascade Classifier PASCAL Con 23
Outline IntroductionSliding-Window-based Object Detection
Window GenerationFeature ExtractionCascade Classifier
PASCAL ChallengeConclusion
Intro Sliding Windows Features Cascade Classifier PASCAL Con 24
PASCAL Dataset 2009
Intro Sliding Windows Features Cascade Classifier PASCAL Con 25
PASCAL Dataset 2009
Intro Sliding Windows Features Cascade Classifier PASCAL Con 26
PASCAL Dataset 2009
Intro Sliding Windows Features Cascade Classifier PASCAL Con 27
PASCAL Dataset 2009
Intro Sliding Windows Features Cascade Classifier PASCAL Con 28
PASCAL Dataset 2009
Intro Sliding Windows Features Cascade Classifier PASCAL Con 29
PASCAL Dataset 2009
Intro Sliding Windows Features Cascade Classifier PASCAL Con 30
PASCAL Dataset 2009
Intro Sliding Windows Features Cascade Classifier PASCAL Con 31
PASCAL Dataset 2009
Intro Sliding Windows Features Cascade Classifier PASCAL Con 32
PASCAL 2009 (trainval + test)
Intro Sliding Windows Features Cascade Classifier PASCAL Con 33
PASCAL 2009 (train+val)
aeroplane bicycle bus car horse motorbike train tvmonitor0
5
10
15
20
25
Aver
age
Prec
isio
n
Intro Sliding Windows Features Cascade Classifier PASCAL Con 34
PASCAL 2009 (train+val,1/image)
Intro Sliding Windows Features Cascade Classifier PASCAL Con 35
PASCAL 2009 (train+val,5/image)
Intro Sliding Windows Features Cascade Classifier PASCAL Con 36
PASCAL 2009 (train+val,10/image)
Intro Sliding Windows Features Cascade Classifier PASCAL Con 37
True Positives - aeroplane
Intro Sliding Windows Features Cascade Classifier PASCAL Con 38
False Positives - aeroplane
Intro Sliding Windows Features Cascade Classifier PASCAL Con 39
True Positives - bicycle
Intro Sliding Windows Features Cascade Classifier PASCAL Con 40
False Positives - bicycle
Intro Sliding Windows Features Cascade Classifier PASCAL Con 41
True Positives - horse
Intro Sliding Windows Features Cascade Classifier PASCAL Con 42
False Positives - horse
Intro Sliding Windows Features Cascade Classifier PASCAL Con 43
Conclusion Proposing sliding windows without fixing scale or
aspect ratio is possible. Simple feature (saliency, contrast, etc) is only
useful for certain object classes. Histogram-based feature is not sufficient to
represent real world object, no matter what learning algorithm is used.
Boosting is helpful to speed up SVM-training and reduce the memory usage.
Digging out “hard” negative examples. Throwing away “hard” positive examples.
Intro Sliding Windows Features Cascade Classifier PASCAL Con 44
Future Work It is time to go beyond the histogram-of-X feature
- not every pixel within bounding box is informative - the appearance of object’s part is more robust
Evolve the classifier- even PASCAL dataset is too small
- the right decision boundary is still far away… - Active learning? Online learning?
Thank You !
45