introduction - hacettepepinar/courses/vbm686/... · 2018. 10. 9. · source: rob fergus and antonio...
TRANSCRIPT
![Page 1: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/1.jpg)
Introduction
VBM686 – Computer Vision
Pinar Duygulu
Hacettepe University
![Page 2: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/2.jpg)
Source: Svetlana Lazebnik, UIUC
Why study computer vision?
![Page 3: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/3.jpg)
Why study computer vision?
Source: Fei Fei Li, Stanford University
An image is worth 1000 words
Images and movies are everywhere
![Page 4: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/4.jpg)
http://www.youtube.com/yt/press/statistics.html
For YouTube alone
More than 1 billion unique users
Hundreds of millions of hours are watched every day
300 hours of video are uploaded every minute
Massive amounts of visual data
4
![Page 5: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/5.jpg)
What do you see in the picture?
Source: Martial Hebert, CMU
![Page 6: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/6.jpg)
What do you see in the picture?Black backgroundTwo objects
One teapotOne toy
There is a light coming from rightOne object is shiny the other is not
Toy:Consists of 5 layers, in different colorsThere is a text : Fisher PriceThe layers are in donut shapeLayers are plasticBottom is wood
Teapot:Consists of body and handleBody is metalHandle is ceramicHandle: Dark blue on whiteBody : golden Reflection of toy on the body
Source: Martial Hebert, CMU
![Page 7: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/7.jpg)
Challenge – What do you see in the picture?
Source: Octavia Camps, Penn State
![Page 8: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/8.jpg)
Challenge – What do you see in the picture?
A hand holding a man
A hand holding a shiny sphere
An Escher drawing
Source: Octavia Camps, Penn State
![Page 9: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/9.jpg)
Source: Aykut Erdem and Erkut Erdem
![Page 10: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/10.jpg)
![Page 11: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/11.jpg)
20 35 90 45 75
25 40 70 40 70
20 35 90 45 75
25 40 70 40 70
20 35 90 45 75
![Page 12: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/12.jpg)
1923
Max Wertheimer (1880 – 1943)
“I stand at the window and see a house, trees, sky.
Theoretically I might say there were 327 brightnesses and
nuances of colour. Do I have “327”? No. I have
sky, house, and trees.”
Source: Aykut Erdem and Erkut Erdem
![Page 13: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/13.jpg)
Nikos K. Logothetis
Nearly half of the
cerebral cortex in
humans is devoted to
processing visual
information.
![Page 14: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/14.jpg)
Source: Michael Black, Brown University
![Page 15: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/15.jpg)
We are easily deceived
by our visual system.
![Page 16: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/16.jpg)
Shading
Source: Michael Black, Brown University
![Page 17: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/17.jpg)
Perception and grouping
Subjective contours
![Page 18: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/18.jpg)
Occlusion
Source: Michael Black, Brown University
![Page 19: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/19.jpg)
Parts and relations
Source: Michael Black, Brown University
![Page 20: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/20.jpg)
How good are our models?
Source: Michael Black, Brown University
![Page 21: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/21.jpg)
How good are our models?
Source: Michael Black, Brown University
![Page 22: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/22.jpg)
Is it only about matching?
Source: Michael Black, Brown University
![Page 23: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/23.jpg)
Is it only about matching?
Source: Michael Black, Brown University
![Page 24: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/24.jpg)
Context
![Page 25: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/25.jpg)
Context
a person?
![Page 26: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/26.jpg)
Context
the blob is identical to the one on the previous slide after a 90deg rotation
a person?
![Page 27: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/27.jpg)
Prior Expectations
Source: Michael Black, Brown University
![Page 28: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/28.jpg)
The goal of computer vision
• To extract “meaning” from pixels
Source: “80 million tiny images” by Torralba et al.
Humans are remarkably good at this…
![Page 29: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/29.jpg)
We are trying to develop automatic
algorithms that would “see”.
Source: Aykut Erdem and Erkut Erdem
![Page 30: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/30.jpg)
The goal of computer vision• To extract “meaning” from pixels
What What we see we see What a computer seesSource: S. Narasimhan
![Page 31: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/31.jpg)
Template matching
Slide credit: Fei-fei LiPinar Duygulu, ENLG 2015 31
![Page 32: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/32.jpg)
Scene categorization• outdoor/indoor
•city/forest/factory/etc.
Slide credit: Fei-fei Li and Sevetlana LazebnikPinar Duygulu, ENLG 2015 32
![Page 33: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/33.jpg)
Image annotation/tagging• street
• people
• building
• mountain
• …
Slide credit: Fei-fei Li and Sevetlana Lazebnik
Pinar Duygulu, ENLG 2015 33
![Page 34: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/34.jpg)
Object Detectionfind pedestrians
Slide credit: Fei-fei Li and Sevetlana LazebnikPinar Duygulu, ENLG 2015 34
![Page 35: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/35.jpg)
Activity Recognition• walking
• shopping
• rolling a cart
• sitting
• talking
• …
Slide credit: Fei-fei Li and Sevetlana LazebnikPinar Duygulu, ENLG 2015 35
![Page 36: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/36.jpg)
Object recognition
mountain
building
tree
banner
marketpeople
street lamp
sky
building
Slide credit: Fei-fei Li and Sevetlana LazebnikPinar Duygulu, ENLG 2015 36
![Page 37: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/37.jpg)
woman
holding a
watermelon
What actions are taking
place?
person
riding a
motorcycle
woman
looking at
apples
woman
walking
Action
Recognition
![Page 38: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/38.jpg)
Object
RelationsUnderstand where things are
in the world
person on
a
motorcycle
woman
behind
a stand
woman
near to
another
woman
woman in
front of a
person
![Page 39: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/39.jpg)
How vision relates to language?Image
Captioning
・ a street scene with a person on a motorcycle.
・ a person on a motorcycle along a farmers market
・ a woman is showing a watermelon slice to a woman on a scooter.
・ a person on a motorcycle talking to a person with a watermelon.
・ people at a veggie and fruit market looking at the merchandise.
![Page 40: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/40.jpg)
Input Output
Joshua Drewe
![Page 41: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/41.jpg)
“Cat”
Joshua Drewe
![Page 42: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/42.jpg)
“Cat”
Joshua Drewe
20 35 90 45 75
25 40 70 40 70
20 35 90 45 75
25 40 70 40 70
20 35 90 45 75
![Page 43: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/43.jpg)
Source: Fei Fei Li
![Page 44: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/44.jpg)
Source: Fei Fei Li
![Page 45: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/45.jpg)
Source: Fei Fei Li
![Page 46: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/46.jpg)
Source: Fei Fei Li
![Page 47: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/47.jpg)
Applications
![Page 48: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/48.jpg)
Optical character recognition (OCR)
Source: S. Seitz, N. Snavely
Digit recognitionyann.lecun.com
License plate readershttp://en.wikipedia.org/wiki/Automatic_number_plate_recognition
Sudoku grabberhttp://sudokugrab.blogspot.com/
Automatic check processing
![Page 49: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/49.jpg)
Biometrics
Fingerprint scanners on many new laptops, other devices
Face recognition systems now beginning to appear more widelyhttp://www.sensiblevision.com/
Source: S. Seitz
![Page 50: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/50.jpg)
Face detection
• Many consumer digital cameras now detect faces
Source: S. Seitz
![Page 51: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/51.jpg)
Smile detection
Sony Cyber-shot® T70 Digital Still Camera Source: S. Seitz
![Page 52: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/52.jpg)
Face recognition: Apple iPhoto software
http://www.apple.com/ilife/iphoto/
Source: S. Lazebnik, UIUC
![Page 53: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/53.jpg)
Mobile visual search: Google Goggles
Source: S. Lazebnik, UIUC
![Page 54: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/54.jpg)
Automotive safety
• Mobileye: Vision systems in high-end BMW, GM, Volvo models
– Pedestrian collision warning– Forward collision warning– Lane departure warning– Headway monitoring and warning
Source: A. Shashua, S. Seitz
![Page 55: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/55.jpg)
Self-driving cars
Source: S. Lazebnik, UIUC
![Page 56: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/56.jpg)
Vision-based interaction: Xbox Kinect
Source: S. Lazebnik, UIUC
![Page 57: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/57.jpg)
3D Reconstruction: Kinect Fusion
YouTube Video
Source: S. Lazebnik, UIUC
![Page 58: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/58.jpg)
3D Reconstruction: Multi-View Stereo
YouTube VideoSource: S. Lazebnik, UIUC
![Page 59: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/59.jpg)
Photosynth
http://labs.live.com/photosynth/
Based on Photo Tourism technology developed by
Noah Snavely, Steve Seitz, and Rick Szeliski
Source: Szeliski, Seitz. Chen
![Page 61: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/61.jpg)
Earth viewers (3D modeling)
Image from Microsoft’s Virtual Earth
(see also: Google Earth)
Source: Szeliski, Seitz. Chen
![Page 62: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/62.jpg)
Object recognition (in supermarkets)
LaneHawk by EvolutionRobotics“A smart camera is flush-mounted in the checkout lane, continuously watching for items. When an item is detected and recognized, the cashier verifies the quantity of items that were found under the basket, and continues to close the transaction. The item can remain under the basket, and with LaneHawk,you are assured to get paid for it… “
Source: Szeliski, Seitz. Chen
![Page 63: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/63.jpg)
Object recognition (in mobile phones)
• This is becoming real:
– Microsoft Research
– Point & Find, Nokia
Source: Szeliski, Seitz. Chen
![Page 64: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/64.jpg)
Special effects: shape and motion capture
Source: Szeliski, Seitz. Chen
![Page 65: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/65.jpg)
Vision in space
Vision systems (JPL) used for several tasks• Panorama stitching
• 3D terrain modeling
• Obstacle detection, position tracking
• For more, read “Computer Vision on Mars” by Matthies et al.
NASA'S Mars Exploration Rover Spirit captured this westward view from atop
a low plateau where Spirit spent the closing months of 2007.
Source: Szeliski, Seitz. Chen
![Page 66: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/66.jpg)
Robotics
http://www.robocup.org/NASA’s Mars Spirit Rover
http://en.wikipedia.org/wiki/Spirit_rover
Source: Szeliski, Seitz. Chen
![Page 67: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/67.jpg)
Medical imaging
Image guided surgery
Grimson et al., MIT3D imaging
MRI, CT
Source: Szeliski, Seitz. Chen
![Page 68: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/68.jpg)
Why is computer vision difficult?
![Page 69: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/69.jpg)
Challenges: viewpoint variation
Michelangelo 1475-1564
Source: Fei-Fei, Fergus & Torralba
![Page 70: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/70.jpg)
Challenges: illumination
Source: J. Koenderink
![Page 71: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/71.jpg)
Challenges: scale
Source: Fei-Fei, Fergus & Torralba
![Page 72: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/72.jpg)
Challenges: deformation
Xu, Beihong 1943
Source: Fei-Fei, Fergus & Torralba
![Page 73: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/73.jpg)
Challenges: occlusion
Source: Fei Fei, Fergus, Torralba
![Page 74: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/74.jpg)
Challenges: Background Clutter
Source: Svetlana Lazebnik, UIUC
![Page 75: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/75.jpg)
Challenges: Motion
Source: Svetlana Lazebnik, UIUC
![Page 76: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/76.jpg)
Challenges: object intra-class variation
Source: Fei-Fei, Fergus & Torralba
![Page 77: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/77.jpg)
Challenges: local ambiguity
Source: Fei-Fei, Fergus & Torralba
![Page 78: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/78.jpg)
Challenges: local ambiguity
Source: Rob Fergus and Antonio Torralba
![Page 79: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/79.jpg)
Challenges: local ambiguity
Source: Rob Fergus and Antonio Torralba
![Page 80: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/80.jpg)
Context
Pinar Duygulu, ENLG 2015 81
Slide credit: Fei-fei Li
![Page 81: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/81.jpg)
Challenges: Inherent ambiguity• Many different 3D scenes could have given rise to a
particular 2D picture
Image source: Svetlana Lazebnik, UIUC
![Page 82: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/82.jpg)
Challenges or opportunities?• Images are confusing, but they also reveal the
structure of the world through numerous cues
• Our job is to interpret the cues!
Image source: J. Koenderink
![Page 83: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/83.jpg)
Depth cues: Linear perspective
Image source: Svetlana Lazebnik, UIUC
![Page 84: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/84.jpg)
Depth cues: Aerial perspective
Image source: Svetlana Lazebnik, UIUC
![Page 85: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/85.jpg)
Depth ordering cues: Occlusion
Source: J. Koenderink
![Page 86: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/86.jpg)
Shape cues: Texture gradient
Image source: Svetlana Lazebnik, UIUC
![Page 87: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/87.jpg)
Shape and lighting cues: Shading
Image source: Svetlana Lazebnik, UIUC
![Page 88: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/88.jpg)
Position and lighting cues: Cast shadows
Source: J. Koenderink
![Page 89: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/89.jpg)
Grouping cues: Similarity (color, texture,proximity)
Source: Svetlana Lazebnik, UIUC
![Page 90: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/90.jpg)
Grouping cues: “Common fate”
Source: Fei Fei Li, Stanford University
![Page 91: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/91.jpg)
Origins of computer vision
L. G. Roberts, Machine Perception of Three Dimensional Solids, Ph.D. thesis, MIT Department of Electrical Engineering, 1963.
Source: Svetlana Lazebnik, UIUC
![Page 92: Introduction - Hacettepepinar/courses/VBM686/... · 2018. 10. 9. · Source: Rob Fergus and Antonio Torralba. Context Pinar Duygulu, ENLG 2015 81 Slide credit: Fei-fei Li. Challenges:](https://reader031.vdocument.in/reader031/viewer/2022012011/613fbcc0b44ffa75b8046b2b/html5/thumbnails/92.jpg)
Connections to other disciplines
Computer Vision
Image Processing
Machine Learning
Artificial Intelligence
Robotics
Cognitive scienceNeuroscience
Computer Graphics
Source: Svetlana Lazebnik, UIUC