![Page 1: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/1.jpg)
1A-L1 Introduction
CS4495/6495 Introduction to Computer Vision
![Page 2: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/2.jpg)
Outline
•What is computer vision?
• State of the art
•Why is this hard?
•Course overview
• Software
![Page 3: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/3.jpg)
Why study Computer Vision? • Images (and movies) have become ubiquitous in
both production and consumption. • Therefore applications to manipulate images
(movies) are becoming core. • As are systems that extract information from
imagery • Surveillance • Building 3D representations • Motion capture assisted
![Page 4: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/4.jpg)
Why study Computer Vision? • But most of all…
It is a really deep and cool set of problems!
![Page 5: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/5.jpg)
Every picture tells a story
Goal of computer vision is to write computer programs that can interpret images
Steve Seitz
![Page 6: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/6.jpg)
Making sense of a picture
•We want to extract meaning out of an image/sequence of images
• This is different from image processing, which is mainly concerned with transforming images
• Image processing operations such as blurring, thresholding etc. are often used as part of CV algorithms
![Page 7: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/7.jpg)
Making sense of a picture
• Look at this scene carefully…
![Page 8: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/8.jpg)
![Page 9: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/9.jpg)
Making sense of a picture
•What items could you identify? How did you recognize them?
•What about other objects/spaces/time of day etc.?
![Page 10: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/10.jpg)
Current state of the art
•Can computers match (or beat) human vision?
• Yes and no (but mostly no!)
•Humans are much better at “hard” things
•Computers can be better at “easy” things
• Though getting really good at labeling using machine learning techniques. Only a little on that in this course.
Steve Seitz
![Page 11: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/11.jpg)
Current state of the art
• The next slides show some examples of what current vision systems can do
![Page 12: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/12.jpg)
Optical character recognition (OCR)
Handwritten Digit recognition
Technology to convert scanned docs to text
If you have a scanner, it probably came with OCR software
License plate readers http://en.wikipedia.org/wiki/Automatic_number_plate_recognition
Steve Seitz
![Page 13: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/13.jpg)
Face detection and more…
•Most digital cameras can detect faces…
![Page 14: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/14.jpg)
Face detection and more…
• Some can detect blinking or smiling…
SONY “Smile Shutter”
![Page 15: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/15.jpg)
Face detection and more…
•And some can even recognize you!
![Page 16: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/16.jpg)
Object recognition (in supermarkets)
• Evolution Robotics Retail developed LaneHawk™, a retail loss-prevention solution that helps turn bottom-of-basket (BOB) losses and in-cart losses into profits in real time.
• The company was acquired by Datalogic 5 years later!
![Page 17: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/17.jpg)
Object recognition (in mobile devices!)
![Page 18: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/18.jpg)
The Matrix movies, ESC Entertainment, XYZRGB, NRC
Special effects: shape capture
Steve Seitz
![Page 19: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/19.jpg)
Pirates of the Caribbean Industrial Light and Magic www.ilm.com
Special effects: motion capture
Steve Seitz
![Page 20: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/20.jpg)
Earth viewers (3D modeling)
Image from Microsoft’s Virtual Earth (see also: Google Earth) Steve Seitz
![Page 22: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/22.jpg)
Smart cars are here!
![Page 23: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/23.jpg)
Sports
Sportvision first down line Steve Seitz
![Page 24: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/24.jpg)
Vision-based interaction (and games)
Nintendo Wii has camera-based IR tracking built in. Steve Seitz
![Page 25: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/25.jpg)
But the game changer:
![Page 26: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/26.jpg)
Security and surveillance
![Page 27: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/27.jpg)
Medical imaging
Image guided surgery Grimson et al., MIT
3D imaging MRI, CT
Steve Seitz
![Page 28: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/28.jpg)
Current state of the art
• This is just a taste of the state of the art.
• Some of these are less than 5 years old, most less than 10
• This is a very active research area, and rapidly changing
• Many new apps in the next 5 years
![Page 29: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/29.jpg)
Why is this hard?
![Page 30: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/30.jpg)
Simple scene right?
Dark square
Light square
Edward Adelson
![Page 31: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/31.jpg)
Really?
Edward Adelson
![Page 32: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/32.jpg)
Really!
Edward Adelson
![Page 33: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/33.jpg)
Vision is NOT Image Processing
• In the previous example, the two squares have exactly the same measurement of intensity.
• So, seeing is not the same as measuring properties in the image.
•Rather, “seeing” is building a percept of what is in the world based upon the measurements made by an imaging sensor.
![Page 34: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/34.jpg)
Building models from change (1)
Michael Black
![Page 35: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/35.jpg)
Building models from change (1)
Left Image Michael Black
![Page 36: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/36.jpg)
Building models from change (1)
Right Image Michael Black
![Page 37: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/37.jpg)
Building models from change (2)
Dan Kersten http://vision.psych.umn.edu/users/kersten/kersten-lab/shadows.html
![Page 38: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/38.jpg)
Building models from change (3)
Dan Kersten http://vision.psych.umn.edu/users/kersten/kersten-lab/shadows.html
![Page 39: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/39.jpg)
Interpreting images
• The previous example is one where the human system is again “wrong” – nothing is moving upwards. But feels like the best interpretation.
•Our goal is to develop your understanding of some of what it takes to go from image to interpretation.
![Page 40: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/40.jpg)
Course overview
![Page 41: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/41.jpg)
A little bit of pedagogy…
![Page 42: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/42.jpg)
A little bit of pedagody… Computational Models (Math!)
![Page 43: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/43.jpg)
Computational Models (Math!)
A little bit of pedagody…
Algorithm
![Page 44: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/44.jpg)
Computational Models (Math!)
A little bit of pedagogy…
Real Images
Scene Ground truth
Algorithm
![Page 45: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/45.jpg)
Computational Models (Math!)
Real Images Algorithm
A little bit of pedagogy…
Introduction to Computer
Vision
![Page 46: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/46.jpg)
Topic outline 1. INTRODUCTION 2. IMAGE PROCESSING FOR COMPUTER VISION 3. CAMERA MODELS AND VIEWS 4. FEATURES AND MATCHING 5. LIGHTNESS AND BRIGHTNESS 6. IMAGE MOTION 7. MOTION AND TRACKING 8. CLASSIFICATION AND RECOGNITION 9. MISCELLANEOUS OPERATIONS 10. HUMAN VISION
![Page 47: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/47.jpg)
Problem sets
•8 problem sets (PS0 to PS7)
![Page 48: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/48.jpg)
Policies
•Blackboard-level conversations OK, esp. on forums
•Write your own code
•Ask questions on forum first, then contact TA/instructor
![Page 49: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/49.jpg)
Exam
• There will be a final exam.
• It’s not hard – it simply designed to require folks to go back over the slides (and text) and remember what we’ve learned.
![Page 50: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/50.jpg)
Grading
• The general rubric is 85% of the final grade is based upon the problem sets.
•15% is the final.
![Page 51: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/51.jpg)
Software
• Embedded programming exercises (in Octave)
•Matlab/Octave: Primary platform for exercises, problem sets
•Python + NumPy + OpenCV: You can submit your problem set solutions in Python, but there will be very limited support
![Page 52: CS4495/6495 Introduction to Computer Vision · Vision is NOT Image Processing •In the previous example, the two squares have exactly the same measurement of intensity. •So, seeing](https://reader034.vdocument.in/reader034/viewer/2022042222/5ec81b3020c2e94c905e9445/html5/thumbnails/52.jpg)
Learning goals
What do you expect to learn from this course?
• Note down somewhere and track your progress.
• In the end, you may not have learnt everything you expected.
• At the same time, you may have learnt some things you did not know about at all