![Page 1: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/1.jpg)
ECS 189G: Intro to Computer VisionMarch 31st, 2015
Yong Jae LeeAssistant Professor
CS, UC Davis
![Page 2: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/2.jpg)
Plan for today
• Topic overview • Introductions• Course overview:
– Logistics and requirements
2
![Page 3: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/3.jpg)
What is Computer Vision?
3
![Page 4: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/4.jpg)
Computer Vision
Enable machines to “see” the visual world as we do
![Page 5: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/5.jpg)
Computer Vision
• Automatic understanding of images and video1. Computing properties of the 3D world from visual data
(measurement)
Slide credit: Kristen Grauman5
![Page 6: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/6.jpg)
1. Vision for measurementReal-time stereo Structure from motion
NASA Mars Rover
Tracking
Demirdjian et al.Snavely et al.
Wang et al.
Slide credit: Kristen Grauman6
![Page 7: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/7.jpg)
Computer Vision
• Automatic understanding of images and video1. Computing properties of the 3D world from visual data
(measurement)
2. Algorithms and representations to allow a machine to recognize objects, people, scenes, and activities (perception and interpretation)
Slide credit: Kristen Grauman7
![Page 8: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/8.jpg)
sky
water
Ferris wheel
amusement park
Cedar Point
12 E
tree
tree
tree
carouseldeck
people waiting in line
ride
ride
ride
umbrellas
pedestrians
maxair
bench
tree
Lake Erie
people sitting on ride
ObjectsActivitiesScenesLocationsText / writingFacesGesturesMotionsEmotions…
The Wicked Twister
2. Vision for perception, interpretation
Slide credit: Kristen Grauman8
![Page 9: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/9.jpg)
Computer Vision
• Automatic understanding of images and video1. Computing properties of the 3D world from visual data
(measurement)
2. Algorithms and representations to allow a machine to recognize objects, people, scenes, and activities. (perception and interpretation)
3. Algorithms to mine, search, and interact with visual data (search and organization)
Slide credit: Kristen Grauman9
![Page 10: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/10.jpg)
3. Visual search, organization
Image or video archives
Query Relevant content
Slide credit: Kristen Grauman10
![Page 11: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/11.jpg)
Related disciplines
Cognitive science
Algorithms
Image processing
Artificial intelligence
GraphicsMachine learning
Computer vision
Slide credit: Kristen Grauman11
![Page 12: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/12.jpg)
Vision and graphics
ModelImages Vision
Graphics
Inverse problems: analysis and synthesis
Slide credit: Kristen Grauman12
![Page 13: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/13.jpg)
Why is vision difficult?
13
![Page 14: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/14.jpg)
What humans see
14
![Page 15: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/15.jpg)
Slide credit: Larry Zitnick
What computers see
15
243 239 240 225 206 185 188 218 211 206 216 225
242 239 218 110 67 31 34 152 213 206 208 221
243 242 123 58 94 82 132 77 108 208 208 215
235 217 115 212 243 236 247 139 91 209 208 211
233 208 131 222 219 226 196 114 74 208 213 214
232 217 131 116 77 150 69 56 52 201 228 223
232 232 182 186 184 179 159 123 93 232 235 235
232 236 201 154 216 133 129 81 175 252 241 240
235 238 230 128 172 138 65 63 234 249 241 245
237 236 247 143 59 78 10 94 255 248 247 251
234 237 245 193 55 33 115 144 213 255 253 251
248 245 161 128 149 109 138 65 47 156 239 255
190 107 39 102 94 73 114 58 17 7 51 137
23 32 33 148 168 203 179 43 27 17 12 8
17 26 12 160 255 255 109 22 26 19 35 24
![Page 16: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/16.jpg)
Why is vision difficult?
• Ill-posed problem: real world much more complex than what we can measure in images– 3D 2D
• Impossible to literally “invert” image formation process
Slide credit: Kristen Grauman16
![Page 17: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/17.jpg)
Challenges: ambiguity
• Many different 3D scenes could have given rise to a particular 2D picture
Slide credit: Svetlana Lazebnik
![Page 18: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/18.jpg)
Challenges: many nuisance parameters
Illumination Object pose Clutter
ViewpointIntra-class appearance
Occlusions
Slide credit: Kristen Grauman18
![Page 19: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/19.jpg)
Challenges: scale
slide credit: Fei-Fei, Fergus, Torralba
![Page 20: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/20.jpg)
Challenges: Motion
slide credit: Svetlana Lazebnik
![Page 21: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/21.jpg)
Challenges: occlusion, clutter
Image source: National Geographslide credit: Svetlana Lazebnik
![Page 22: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/22.jpg)
Challenges: object intra-class variation
slide credit: Fei-Fei, Fergus, Torralba
![Page 23: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/23.jpg)
Slide credit: Fei-Fei, Fergus, Torralba
Challenges: context and human experience
23
![Page 24: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/24.jpg)
Challenges: context and human experience
Fei Fei Li, Rob Fergus, Antonio Torralba
![Page 25: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/25.jpg)
Challenges: context and human experience
Fei Fei Li, Rob Fergus, Antonio Torralba
![Page 26: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/26.jpg)
Biederman 1987Slide credit: Fei-Fei, Fergus, Torralba
Challenges: complexity
How many object categories are there?
26
![Page 27: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/27.jpg)
6 billion images 70 billion images 1 billion images served daily
10 billion images
100 hours uploaded per minute
Almost 90% of web traffic is visual!
:From
Challenges: complexity
27
![Page 28: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/28.jpg)
Challenges: complexity• Thousands to millions of pixels in an image• 30+ degrees of freedom in the pose of articulated objects
(humans)• About half of the cerebral cortex in primates is devoted to
processing visual information [Felleman and van Essen 1991]
Slide credit: Kristen Grauman28
![Page 29: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/29.jpg)
What works well today?
29
![Page 30: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/30.jpg)
Optical character recognition (OCR)
Source: S. Seitz, N. Snavely
Digit recognitionyann.lecun.com
License plate readershttp://en.wikipedia.org/wiki/Automatic_number_plate_recognition
Sudoku grabberhttp://sudokugrab.blogspot.com/
Automatic check processing
![Page 31: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/31.jpg)
Biometrics
Fingerprint scannersFace recognition systems
![Page 32: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/32.jpg)
Face detection
• Many consumer digital cameras now detect faces
Source: S. Seitz
![Page 33: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/33.jpg)
Face detection for privacy protection
slide credit: Svetlana Lazebnik
![Page 34: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/34.jpg)
Technology gone wild…
slide credit: Svetlana Lazebnik
![Page 35: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/35.jpg)
Face recognition
Slide credit: Devi Parikh35
![Page 36: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/36.jpg)
Interactive systems
Shotton et al.
![Page 37: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/37.jpg)
Instance recognition
Slide credit: Devi Parikh37
![Page 38: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/38.jpg)
Pedestrian detection
Slide credit: Devi Parikh38
![Page 39: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/39.jpg)
Autonomous agents
Google self-driving car
Mars rover
![Page 40: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/40.jpg)
3D reconstruction from photo collections
YouTube Video
Q. Shan, R. Adams, B. Curless, Y. Furukawa, and S. Seitz, The Visual Turing Test for Scene Reconstruction, 3DV 2013
slide credit: Svetlana Lazebnik
![Page 41: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/41.jpg)
The Matrix movies, ESC Entertainment, XYZRGB, NRC
Special effects: shape capture
Source: S. Seitz
![Page 42: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/42.jpg)
Pirates of the Carribean, Industrial Light and Magic
Special effects: motion capture
Source: S. Seitz
![Page 43: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/43.jpg)
Medical imaging
Image guided surgeryGrimson et al., MIT
3D imagingMRI, CT
Source: S. Seitz
![Page 44: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/44.jpg)
L. G. Roberts, Machine Perception of Three Dimensional Solids, Ph.D. thesis, MIT Department of Electrical Engineering, 1963.
Visual data in 1963
Slide credit: Kristen Grauman44
![Page 45: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/45.jpg)
Personal photo albums
Surveillance and security
Movies, news, sports
Medical and scientific images
Visual data today
Svetlana Lazebnik
Understand and organize and index all this data!!
![Page 46: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/46.jpg)
Why vision?
• As image sources multiply, so do applications
– Relieve humans of boring, easy tasks
– Enhance human abilities
– Advance human-computer interaction, visualization
– Perception for robotics / autonomous agents
– Organize and give access to visual content
Slide credit: Kristen Grauman46
![Page 47: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/47.jpg)
Applications
• Law enforcement / Surveillance• Robotics• Autonomous driving• Medical imaging• Photo organization• Image search• E-commerce• … cell phone cameras, social media, Google Glass,
etc.
Slide adapted from Devi Parikh47
![Page 48: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/48.jpg)
Summary• Computer Vision is useful, interesting, and difficult• A growing and exciting field• Lots of cool and important applications• New teams in existing companies, startups, etc.
Slide adapted from Devi Parikh48
![Page 49: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/49.jpg)
Introductions
• Instructor– Yong Jae Lee– [email protected]– Assistant Professor in CS, UC Davis since July 2014
– Ph.D. from UT Austin in 2012– Post-doc at CMU and UC Berkeley for 2 years– Research area: Computer Vision
• Visual Recognition• Graphics Applications
49
![Page 50: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/50.jpg)
Introductions
• TAs:– Vivek Dubey– [email protected]– MS student in ECE
– Ahsan Abdullah– [email protected]– PhD student in CS
50
![Page 51: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/51.jpg)
This course• ECS 189G (4-units)
• Lecture: Tues & Thurs 6:10-7:30 pm, Everson Hall 176
• Discussion section: Mon 2:10-3pm, Wellman Hall 2
• Office hours: Academic Surge 1044– Yong Jae: Fri 4-6 pm – Vivek: Mon & Wed 6-8 pm– Ahsan: Tues & Thurs 4-6 pm
51
![Page 52: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/52.jpg)
This course• Course webpage
https://sites.google.com/a/ucdavis.edu/ecs-189g-intro-to-computer-vision/
• SmartSite (assignment submission, grades)https://smartsite.ucdavis.edu/portal/site/ecs189g-sp2015
• Piazzahttps://piazza.com/uc_davis/spring2015/ecs189g
52
![Page 53: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/53.jpg)
Goals of this course
• Introduction to primary topics in Computer Vision
• Basics and fundamentals• Practical experience through assignments• Views of computer vision as a research area
53
![Page 54: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/54.jpg)
Prerequisites
• Upper-division undergrad course
• Basic knowledge of probability and linear algebra• Data structures, algorithms• Programming experience
• Experience with image processing or Matlab will help but is not necessary
![Page 55: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/55.jpg)
Topics overview
• Features and filters • Grouping and fitting• Recognition and learning
Focus is on algorithms, rather than specific systems
55
![Page 56: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/56.jpg)
Features and filters
Transforming and describing images; textures, colors, edges
Slide credit: Kristen Grauman56
![Page 57: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/57.jpg)
Grouping and fitting
[fig from Shi et al]
Clustering, segmentation, fitting; what parts belong together?
Slide credit: Kristen Grauman57
![Page 58: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/58.jpg)
Recognition and learning
Recognizing objects and categories, learning techniques
Slide credit: Kristen Grauman58
![Page 59: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/59.jpg)
Additional topic (time permitting)
Deep learning59
![Page 60: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/60.jpg)
Not covered: Multiple views and motion
Hartley and Zisserman
Lowe
Multi-view geometry, stereo vision
Fei-Fei Li
Slide credit: Kristen Grauman60
![Page 61: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/61.jpg)
Not covered: Video processing
Tomas Izo
Tracking objects, video analysis, low level motion, optical flow
Slide credit: Kristen Grauman61
![Page 62: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/62.jpg)
Textbooks
By Rick Szeliskihttp://szeliski.org/Book/
By Kristen Grauman, Bastian LeibeVisual Object Recognition
62
![Page 63: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/63.jpg)
Requirements / Grading
• Problem sets (70%)
• Final exam (25%)– comprehensive (cover all topics learned in class)
• Class and Piazza participation, including attendance (5%)– Piazza: participation points for posting (sensible)
questions and answers
63
![Page 64: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/64.jpg)
Problem sets• Some short answer concept questions• Matlab programming problems
– Implementation– Explanation, results
• Follow instructions; points will be deducted if we can’t run your code out of the box
• Ask questions on Piazza first• Submit to SmartSite• The assignments will take significant time to do• Start early
• TAs will go over problem set during first discussion section after release (others will be used as extra office hours)
Slide adapted from Kristen Grauman64
![Page 65: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/65.jpg)
Matlab
• Built-in toolboxes for low-level image processing, visualization
• Compact programs
• Intuitive interactive debugging
• Widely used in engineering
Slide credit: Kristen Grauman65
![Page 66: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/66.jpg)
Matlab
• CSIF labs 67, 71, 75 (pc33-pc60)• Academic Surge 1044 and 1116• Lab schedule (reservations) and remote access
info found on class website
• Matlab (Simulink Student Suite) can be purchased for $99
66
![Page 67: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/67.jpg)
Problem Set 0
• Matlab warmup• Basic image manipulation• Out Thursday, due 4/10
67
![Page 68: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/68.jpg)
Digital imagesImages as matrices
Slide credit: Kristen Grauman68
![Page 69: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/69.jpg)
im[176][201] has value 164 im[194][203] has value 37
width 520j=1
500 height
i=1Intensity : [0,255]
Digital images
Slide credit: Kristen Grauman69
![Page 70: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/70.jpg)
R G B
Color images, RGB color space
Slide credit: Kristen Grauman70
![Page 71: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/71.jpg)
Preview of some problem sets
Slide credit: Devi Parikh
resize: castle squished
crop: castle cropped
content aware resizing:seam carving
71
![Page 72: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/72.jpg)
Preview of some problem sets
Grouping
Slide credit: Kristen Grauman72
![Page 73: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/73.jpg)
Preview of some problem sets
Object search and recognition
Slide credit: Kristen Grauman73
![Page 74: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/74.jpg)
Problem set deadlines
• Problem sets due 11:59 PM– Follow submission instructions given in assignment
– Submit to SmartSite; no hard copy submissions
– Deadlines are firm. We’ll use SmartSite timestamp. Even 1 minute late is late.
• 3 total free late days for the semester– Use them wisely: first couple assignments are easier than others
• If your program doesn’t work, clean up the code, comment it well, explain what you have, and still submit. Draw our attention to this in your answer sheet.
Slide adapted from Kristen Grauman, Devi Parikh74
![Page 75: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/75.jpg)
Collaboration policy
• Can discuss problem sets with peers, but all responses and code must be written individually
• Students submitting answers or code found to be identical or substantially similar (due to inappropriate collaboration) risk failing the course
• Read and follow UC Davis code of conduct
Slide adapted from Kristen Grauman, Devi Parikh75
![Page 76: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/76.jpg)
Miscellaneous
• Check class website regularly for assignment files, notes, announcements, etc.
• Come to lecture on time• No laptops, phones, tablets, etc. in class please• Please interrupt with questions at any time
76
![Page 77: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/77.jpg)
Coming up
• Read the class webpage carefully• Next class (Thurs): lecture on linear filters• PS0 out Thursday, due 4/10
77
![Page 78: ECS 189G: Intro to Computer Visionyjlee/teaching/ecs189g... · Computer Vision • Automatic understanding of images and video 1. Computing properties of the 3D world from visual](https://reader033.vdocument.in/reader033/viewer/2022060502/5f1c01b32bd50343e5321f8b/html5/thumbnails/78.jpg)
Questions?
See you Thursday!