Download - Poselets
![Page 1: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/1.jpg)
Poselets
Michael KraininCSE 590V
Oct 18, 2011
![Page 2: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/2.jpg)
Person Detection
• Dalal and Triggs ‘05– Learn to classify
pedestrians vs. background
– HOG + linear SVM– Doesn’t account for
variations in body pose and viewpoint
![Page 3: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/3.jpg)
Poselets
Poselets capture part of the pose from a given viewpoint [Bourdev & Malik, ICCV09]
![Page 4: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/4.jpg)
Poselets
But how are we going to create training examples of poselets? [Bourdev & Malik, ICCV09]
![Page 5: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/5.jpg)
How do we train a poselet for a given pose configuration?
[Bourdev & Malik, ICCV09]
![Page 6: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/6.jpg)
Finding correspondences at training time
Given part of a human pose
How do we find a similar pose configuration in the training set?
[Bourdev & Malik, ICCV09]
![Page 7: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/7.jpg)
We use keypoints to annotate the joints, eyes, nose, etc. of people
Left Hip
Left Shoulder
Finding correspondences at training time
[Bourdev & Malik, ICCV09]
![Page 8: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/8.jpg)
Residual Error
Finding correspondences at training time
[Bourdev & Malik, ICCV09]
![Page 9: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/9.jpg)
Training poselet classifiers
Residual Error:
0.15
0.20
0.10
0.35
0.15
0.85
1. Given a seed patch2. Find the closest patch for every
other person3. Sort them by residual error4. Threshold them
[Bourdev & Malik, ICCV09]
![Page 10: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/10.jpg)
Training poselet classifiers
1. Given a seed patch2. Find the closest patch for every
other person3. Sort them by residual error4. Threshold them5. Use them as positive training
examples to train a linear SVM with HOG features
[Bourdev & Malik, ICCV09]
![Page 11: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/11.jpg)
Which poselets should we train?
• Choose thousands of random windows, generate poselet candidates, train linear SVMs
• Select a small set of poselets that are:– Individually effective–Complementary
[Bourdev & Malik, ICCV09]
![Page 12: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/12.jpg)
Person Detection Using Poselets
• [12] Felzenszwalb et al. Object Detection with Discriminatively Trained Part Based Models. PAMI 2010.
![Page 13: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/13.jpg)
Other Uses of Poselets
• Object Segmentation (CVPR 2011)– Predict area using “soft masks”– Deformation to match image edges– Extends poselets to other objects
• Activity Recognition (CVPR 2011)– Recognition from a single image– Use which poselets fired and to
what extent to predict the activity
![Page 14: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/14.jpg)
Describing People: A Poselet-Based Approach to Attribute Classification
L. Bourdev, S. Maji, and J. MalikICCV 2011
![Page 15: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/15.jpg)
Random 1¼% of the test set
[Bourdev et al., ICCV11]
![Page 16: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/16.jpg)
How poselets help in high-level vision
The image is a complex function of the viewpoint,
pose, appearance, etc.
Poselets decouple pose and camera view from
appearance[Bourdev et al., ICCV11]
![Page 17: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/17.jpg)
Gender recognition with poselets
[Bourdev et al., ICCV11]
![Page 18: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/18.jpg)
Attribute Classification Overview
PoseletActivations
Given a test image
[Bourdev et al., ICCV11]
![Page 19: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/19.jpg)
Features
Features
Pyramid HOG
LAB histogram
Skin features Hands-skin Legs-skin
Poseletpatch
B .* CSkinmask
Armsmask
Features
PoseletActivations
[Bourdev et al., ICCV11]
![Page 20: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/20.jpg)
PoseletActivations
Features
Poselet-level Classifiers
Poselet Level Classification
• Linear SVM + Sigmoid• Some attributes are associated with a
particular body part
[Bourdev et al., ICCV11]
![Page 21: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/21.jpg)
PoseletActivations
Features
Poselet-levelClassifiers
Person-level Classifiers
Person-level Classification• Another Linear SVM + Sigmoid
[Bourdev et al., ICCV11]
![Page 22: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/22.jpg)
PoseletActivations
Features
Poselet-level Classifiers
Person-level Classifiers
Context-level Classifiers
Context-Level Classification
[Bourdev et al., ICCV11]
![Page 23: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/23.jpg)
Results
![Page 24: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/24.jpg)
Random 1¼% of the test set
[Bourdev et al., ICCV11]
![Page 25: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/25.jpg)
Label Frequency SPM Poselets-No-context Poselets-Full
Precision/recall on our test set
[Bourdev et al., ICCV11]
![Page 26: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/26.jpg)
Gender Recognition vs. Cognitec
• Cognitec is one of the leading face recognition companies (according to latest NIST test)
Cognitec PoseletsScope Frontal faces
onlyAny view
Features Proprietary biometric
Standard HOG
Min resolution
60pixel face width
Any resolution
Performance 75.0% AP 82.4% AP• Upper bound for any gender
recognizer based on frontal faces: Max AP=80.5% vs. 82.4% [Bourdev et al., ICCV11]
![Page 27: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/27.jpg)
Highest-scoring “is male”
Lowest-scoring “is male”[Bourdev et al., ICCV11]
![Page 28: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/28.jpg)
Highest-scoring “has long hair”
Lowest-scoring “has long hair” [Bourdev et al., ICCV11]
![Page 29: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/29.jpg)
Highest-scoring “wears a hat”
Lowest-scoring “wears a hat” [Bourdev et al., ICCV11]
![Page 30: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/30.jpg)
Highest-scoring “wears glasses”
Lowest-scoring “wears glasses” [Bourdev et al., ICCV11]
![Page 31: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/31.jpg)
Highest-scoring “long sleeves”
Lowest-scoring “long sleeves” [Bourdev et al., ICCV11]
![Page 32: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/32.jpg)
[Bourdev et al., ICCV11]
![Page 33: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/33.jpg)
“100 Special Moments” by Jason Salavon
![Page 34: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/34.jpg)
Highly-Weighted Poselets
is-male
has-long-hair
has-glasses
![Page 35: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/35.jpg)
Which poselets are discriminative for gender?
Preferred by human subjects
Preferred by our system
best
best
worst
worst
[Bourdev et al., ICCV11]
![Page 36: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/36.jpg)
“A person with long pants”
“A man with short hair and long sleeves”
“A woman with long hair, glasses and long pants”(??)
Describing people
[Bourdev et al., ICCV11]
![Page 37: Poselets](https://reader036.vdocument.in/reader036/viewer/2022062310/568165ef550346895dd91367/html5/thumbnails/37.jpg)
Poselets website
http://eecs.berkeley.edu/~lbourdev/poselets
The set of published poselet papers H3D data set + Matlab tools Java3D annotation tool + video
tutorial Matlab code to detect people using
poselets Latest trained poselets
[Bourdev et al., ICCV11]