large-scale, real-world face recognition in movie trailers alan wright
TRANSCRIPT
![Page 1: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/1.jpg)
Large-Scale, Real-World Face Recognition in Movie
Trailers
Alan Wright
![Page 2: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/2.jpg)
Plan of Attack
• Extract Facial Tracks from videos• Extract the features from the facial tracks.• Build framework to load and test data.• Begin with baseline testing (Sparse, min,
mean, etc)• Algorithm development…
![Page 3: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/3.jpg)
Dataset
• 2010 Movie Trailerso 305 videoso Avg. of 625 faces per trailero Avg. of 23 face trackers per trailer
• PubFigo 58,787 photoso 200 people o Avg. of 295 photos per person.
![Page 4: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/4.jpg)
Dataset
Facetrack: Dictionary:
![Page 5: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/5.jpg)
Face Tracks
• We have:
• Instead of:
![Page 6: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/6.jpg)
Linear Combination Approach
+ x2 + x3
+ x4 + x5 + x6
+ x7 + x8 + x9
Test Image
= x1
Training Images
![Page 7: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/7.jpg)
Linear Combination Approach
y
Testing
A=
=
Training
x
Coefficients
![Page 8: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/8.jpg)
Linear Combination & Face Tracks
![Page 9: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/9.jpg)
Linear Combination & Face Tracks
![Page 10: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/10.jpg)
Linear Combination & Face Tracks
![Page 11: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/11.jpg)
Linear Combination & Face Tracks
![Page 12: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/12.jpg)
Linear Combination & Face Tracks
![Page 13: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/13.jpg)
Linear Combination & Face Tracks
![Page 14: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/14.jpg)
Linear Combination & Face tracks
![Page 15: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/15.jpg)
Linear Combination & Face Tracks
![Page 16: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/16.jpg)
Linear Combination & Face Tracks
![Page 17: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/17.jpg)
Linear Combination & Face Tracks
Sylvester Stallone
![Page 18: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/18.jpg)
Variations
• Blurry images within face tracks
• Unknowns
• Facial Expressions
![Page 19: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/19.jpg)
Variations
• Pose
• Occlusions
![Page 20: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/20.jpg)
Preliminary Testing
• Ran SRC facial recognition on extracted facial tracks from Date Night.
• Choose the name that has the most “votes” in the track.
• Looking strictly at Steve Carell and Tina Fey tracks…
![Page 21: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/21.jpg)
Results
• Correctly assessed 15 out of 34 face tracks correctly (44.12% accuracy).
• 6 out of 18 Tina Fey face tracks (33.33% accuracy).
• 9 out of 16 Steve Carell face tracks (56.25% accuracy).
![Page 22: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/22.jpg)
Correct Result
• Track 7- IDed CORRECTLY as: Tina Feyo Alyssa Milano had 2 voteso Drew Barrymore had 4 voteso Jennifer Love Hewitt had 1 voteso Salma Hayek had 3 voteso Tina Fey had 22 votes
• 32 frames in this track
• 68.75% identified as Tina Fey
![Page 23: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/23.jpg)
Correct Result
![Page 24: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/24.jpg)
Correct Result
• Good facial crop.
• Many frames.
• Good lighting
• Non blurry
• Generally straightforward pose
![Page 25: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/25.jpg)
Incorrect Result (1)
• Track 9- IDed as: Gael Garcia Bernalo Clive Owen had 2 voteso Cristiano Ronaldo had 1 voteso Gael Garcia Bernal had 3 voteso Steve Carell had 2 votes
![Page 26: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/26.jpg)
Incorrect Result (1)
![Page 27: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/27.jpg)
Incorrect Result (1)
• Blurry
• Not a great crop
• Only 7 frames
![Page 28: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/28.jpg)
Incorrect Result (2)
• Track 26 - IDed as: Nicole Kidmano David Duchovny had 2 voteso Lucy Liu had 1 voteso Michael Bloomberg had 1 voteso Nicole Kidman had 3 voteso Salma Hayek had 1 votes
![Page 29: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/29.jpg)
Incorrect Result (2)
![Page 30: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/30.jpg)
Incorrect Result (2)
• Profile view
• Poor crop
• Only 8 frames.
![Page 31: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/31.jpg)
Incorrect Result (3)
• track 40 - IDed as: Keira Knightleyo Adriana Lima had 6 voteso Cindy Crawford had 8 voteso Eliot Spitzer had 1 voteso Eva Mendes had 1 voteso Gillian Anderson had 1 voteso Keira Knightley had 18 voteso Meg Ryan had 3 voteso Minnie Driver had 1 voteso Salma Hayek had 3 voteso Tina Fey had 15 votes
![Page 32: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/32.jpg)
Incorrect Result (3)
![Page 33: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/33.jpg)
Incorrect Result (3)
• Dark lighting
• Pose variation
• Motion blur toward the end
• 57 frames
![Page 34: Large-Scale, Real-World Face Recognition in Movie Trailers Alan Wright](https://reader035.vdocument.in/reader035/viewer/2022062619/551827a45503469d318b4cc1/html5/thumbnails/34.jpg)
What’s next?
• Continue to look at the coefficient vector.
• Look into confidence of each choice (SCI and residual error)
• Apply different facial recognition algorithms.
• Best way to choose a name for each track? (Mode, strongest confidence, etc.)