generic object detection using feature maps oscar danielsson ([email protected]) stefan carlsson...
Post on 20-Dec-2015
228 views
TRANSCRIPT
Generic Object Detection using Feature Maps
Oscar Danielsson ([email protected])Stefan Carlsson ([email protected])
Outline
Detect all Instances of an Object Class
The classifier needs to be fast (on average). This is typically accomplished by:1. Using image features that can be computed quickly2. Using a cascade of increasingly complex classifiers (Viola and Jones IJCV 2004)
Outline
Famous Object Detectors (1)
Dalal and Triggs (CVPR 05) use a dense Histogram of Oriented Gradients (HOG) representation - the window is tiled into (overlapping) sub-regions and gradient
orientation histograms from all sub-regions are concatenated. A linear SVM is used for classification.
€
H(I) = wT ⋅
Mhi1M
hin
⎡
⎣
⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥
M
⎡
⎣
⎢ ⎢ ⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥ ⎥ ⎥
+b
Famous Object Detectors (2)Felzenszwalb et. al. (PAMI 10, CVPR 10) extend the Dalal and Triggs model to include
high resolution parts with flexible location.
€
H(I) = wT ⋅
Mhi1M
hin
⎡
⎣
⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥
M
⎡
⎣
⎢ ⎢ ⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥ ⎥ ⎥
+b
Famous Object Detectors (3)
€
pi f i I( ) < piθ i
Viola and Jones (IJCV 2004) construct a weak classifier by thresholding the response of a Haar filter (computed using integral images). Weak classifiers are combined into a
strong classifier using AdaBoost.
€
H(I) = α ii p i f i (I )< p iθ i{ }
∑
Outline
Motivation
Corners Corners + Blobs Regions Edges
Different object classes are characterized by different features. So we want to leave the choice of features up to the user.
Therefore we construct an object detector based on feature maps. Any feature detectors in any combination can be used to generate feature maps.
Our Object Detector
€
pidi I( ) < piθ i
We use AdaBoost to build a strong classifier. We construct a weak classifier by thresholding the distance from a measurement point to the closest occurrence of a
given feature.
€
H(I) = α ii p i d i (I )< p iθ i{ }
∑
Outline
Extraction of Training Data
€
f =
d1
M
dn
⎡
⎣
⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥
1. Feature maps are extracted by some external feature detectors2. Distance transforms are computed for each feature map3. (For each training window) Distances from each measurement point to the closest
occurrence of the corresponding feature are concatenated into a vector
Training
Cascade
Strong Learner
Weak Learner
Decision Stump Learner
1. Require positive training examples and background images
2. Randomly sample background images to extract negative training examples
3. Loop:1. Train strong classifier2. Append strong classifier to
current cascade3. Run cascade on background
images to harvest false positives4. If number of false positives
sufficiently few, stop
{ fi }, { Ij }
{ fi }, { ci }, T
{ fi }, { ci }, { di }
{ fi }, { ci }, { di }
Viola-Jones Cascade Construction
Training
Cascade
Strong Learner
Weak Learner
Decision Stump Learner
1. Require positive training examples and background images
2. Randomly sample background images to extract negative training examples
3. Loop:1. Train strong classifier2. Append strong classifier to
current cascade3. Run cascade on background
images to harvest false positives4. If number of false positives
sufficiently few, stop
{ fi }, { Ij }
{ fi }, { ci }, T
{ fi }, { ci }, { di }
{ fi }, { ci }, { di }
Viola-Jones Cascade Construction
Training
Cascade
Strong Learner
Weak Learner
Decision Stump Learner
1. Require labeled training examples and number of rounds
2. Init. weights of training examples 3. For each round
1. Train weak classifier2. Compute weight of weak
classifier3. Update weights of training
examples
{ fi }, { ci }, T
{ fi }, { Ij }
{ fi }, { ci }, { di }
{ fi }, { ci }, { di }
AdaBoost
Training
Cascade
Strong Learner
Weak Learner
Decision Stump Learner
1. Require labeled training examples and number of rounds
2. Init. weights of training examples 3. For each round
1. Train weak classifier2. Compute weight of weak
classifier3. Update weights of training
examples
{ fi }, { ci }, T
{ fi }, { Ij }
{ fi }, { ci }, { di }
{ fi }, { ci }, { di }
AdaBoost
Training
Cascade
Strong Learner
Weak Learner
Decision Stump Learner
1. Require labeled and weighted training examples
2. Compute node output3. Train decision stump4. Split training examples using decision
stump5. Evaluate stopping conditions6. Train decision tree on left subset of
training examples7. Train decision tree on right subset of
training examples
{ fi }, { ci }, { di }
{ fi }, { Ij }
{ fi }, { ci }, T
{ fi }, { ci }, { di }
Decision Tree Learner
Training
Cascade
Strong Learner
Weak Learner
Decision Stump Learner
1. Require labeled and weighted training examples
2. Compute node output3. Train decision stump4. Split training examples using decision
stump5. Evaluate stopping conditions6. Train decision tree on left subset of
training examples7. Train decision tree on right subset of
training examples
{ fi }, { ci }, { di }
{ fi }, { Ij }
{ fi }, { ci }, T
{ fi }, { ci }, { di }
Decision Tree Learner
Training
Cascade
Strong Learner
Weak Learner
Decision Stump Learner
1. Require labeled and weighted training examples
2. For each measurement point1. Compute a threshold by
assuming exponentially distributed distances
2. Compute classification error after split
3. If error lower than previous errors, store threshold and measurement point
{ fi }, { ci }, { di }
{ fi }, { Ij }
{ fi }, { ci }, T
{ fi }, { ci }, { di }
Feature and Threshold Selection
Outline
Hierarchical DetectionEvaluate an “optimistic” classifier on regions in search space. Split positive regions
recursively.
Hierarchical DetectionEvaluate an “optimistic” classifier on regions in search space. Split positive regions
recursively.
Hierarchical DetectionEvaluate an “optimistic” classifier on regions in search space. Split positive regions
recursively.
Hierarchical DetectionEvaluate an “optimistic” classifier on regions in search space. Split positive regions
recursively.
Hierarchical DetectionEvaluate an “optimistic” classifier on regions in search space. Split positive regions
recursively.
Hierarchical DetectionEvaluate an “optimistic” classifier on regions in search space. Split positive regions
recursively.
Hierarchical Detection
x
y
s
x
y
s
Search space Image space
Each point in search space corresponds to a window in the image. In the image is a measurement point.
Hierarchical Detection
x
y
s
x
ySearch space Image space
A region in search space corresponds to a set of windows in the image. This translates to a set of locations for the measurement point.
Hierarchical Detection
x
y
s
x
ySearch space Image space
We can then compute upper and lower bounds for the distance to the closest occurrence of the corresponding feature. Based on these bounds we construct an
optimistic classifier.
€
M
dmin
M
⎡
⎣
⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥≤ f ≤
M
dmax
M
⎡
⎣
⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥
Outline
Experiments
• Detection results obtained on the ETHZ Shape Classes dataset, which was used for testing only
• Training data downloaded from Google images : 106 applelogos, 128 bottles, 270 giraffes, 233 mugs and 165 swans
• Detections counted as correct if Aintersect / Aunion ≥ 0.2
• Features used: edges, corners, blobs, Kadir-Brady + SIFT + quantization
ResultsReal AdaBoost slightly better than Dirscrete and Gentle AdaBoost
ResultsDecision tree weak classifiers should be shallow
ResultsUsing all features are better than using only edges
ResultsUsing the asymmetric weighting scheme of Viola and Jones yields a slight
improvement
ResultsApplelogos Bottles Mugs Swans
ResultsHierarchical search yields a significant speed-up
Outline
Conclusion
• Proposed object detection scheme based on feature maps• Used distances from measurement points to nearest
feature occurrence in image to construct weak classifiers for boosting
• Showed promising detection performance on the ETHZ Shape Classes dataset
• Showed that a hierarchical detection scheme can yield significant speed-ups
– Thanks for listening!
Famous Object Detectors (4)
€
piwiTfi I( ) < piθ i
Laptev (IVC 09) construct a weak classifier using a linear discriminant on a histogram of oriented gradients (HOG – computed by integral histograms) from a sub-region of the window. Again, weak classifiers are combined into a strong classifier using AdaBoost.
€
H(I) = α ii p iw i
Tf i (I )< p iθ i{ }
∑