cs4670: computer vision - cornell university...crs baseline lig mrim-fusion (71 alcala avw alcala...
TRANSCRIPT
![Page 1: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/1.jpg)
Lecture 29: Recent work in recognition
CS4670: Computer VisionNoah Snavely
![Page 2: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/2.jpg)
Object recognition
• Category recognition has been the focus of extensive research in the past decade
• Extensive use and development of machine learning techniques, better features
• Moderate-scale datasets derived from the Web– PASCAL VOC: 20 object categories, > 10K images,
> 25K instances, hand-labeled ground truth, annual competitions
![Page 3: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/3.jpg)
• Twenty object categories (aeroplane to TV/monitor)
• Three challenges:
– Classification challenge (is there an X in this image?)
– Detection challenge (draw a box around every X)
– Segmentation challenge
![Page 4: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/4.jpg)
![Page 5: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/5.jpg)
![Page 6: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/6.jpg)
![Page 7: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/7.jpg)
![Page 8: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/8.jpg)
is
is there a cat?
![Page 9: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/9.jpg)
![Page 10: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/10.jpg)
![Page 11: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/11.jpg)
![Page 12: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/12.jpg)
![Page 13: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/13.jpg)
![Page 14: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/14.jpg)
![Page 15: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/15.jpg)
![Page 16: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/16.jpg)
![Page 17: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/17.jpg)
![Page 18: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/18.jpg)
Chance essentially 0
![Page 19: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/19.jpg)
![Page 20: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/20.jpg)
![Page 21: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/21.jpg)
![Page 22: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/22.jpg)
![Page 23: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/23.jpg)
![Page 24: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/24.jpg)
![Page 25: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/25.jpg)
![Page 26: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/26.jpg)
Best localization methods
• Sliding window-style classifiers
– SVM, Adaboost
– Flexible spatial template: “star model”
• Separate classifiers by viewpoint
• Use of context in classifiers
• Local features
– HoG, SIFT, local histograms of gradient orientations
![Page 27: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/27.jpg)
HoG features
• Image partitioned into 8x8 blocks
• In each block, compute histogram of gradient orientations
![Page 28: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/28.jpg)
Flexible Spatial Template (UofC-TTI)
• Hierarchical model [Felzenszwalb et al 2008]
– Coarse template for finding the root part
– Fine-scale templates connected by springs
– Learning automatically from labeled bounding boxes
• Separate models per viewpoint
![Page 29: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/29.jpg)
Six-component car model
root filters (coarse) part filters (fine) deformation models
side view
frontal view
![Page 30: CS4670: Computer Vision - Cornell University...CRS BASELINE LIG MRIM-FUSION (71 ALCALA AVW ALCALA LAVW CRS SOFT-EER (703) _e_ SOFT-BASELINE (700) _e— CIC3M GEN-DIS (69 g) LIG MRIM-COLORSIFT](https://reader035.vdocument.in/reader035/viewer/2022071408/60fffee37f680f038d17973d/html5/thumbnails/30.jpg)
Six-component person model