introduction to computer and human vision
DESCRIPTION
Introduction to Computer and Human Vision. Shimon Ullman, Ronen Basri, Michal Irani. Assistants: Tal Hassner Eli Shechtman . Misc. Course website: www.wisdom.weizmann.ac.il/~hassner/cv0203 - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/1.jpg)
Introduction to Computer and Human Vision
Shimon Ullman, Ronen Basri, Michal Irani
Assistants:Tal Hassner <[email protected]>Eli Shechtman <elishe @wisdom.weizmann.ac.il>
![Page 2: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/2.jpg)
Misc...
Course website:
www.wisdom.weizmann.ac.il/~hassner/cv0203
To be added to course mailing-list:
send email to <hassner@wisdom….>
Other recommended courses (for credit):
- Basic Topics
- Statistical Machine Learning
Vision & Robotics Seminar (not for credit):
Thursdays at 11:00-12:00 (Ziskind 1)send email <leah@wisdom…>ask to be added to “seminar13” mailing list
![Page 3: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/3.jpg)
How is an image formed? (geometry and photometry)
What kind of operations can we apply to images?
What do images tell us about the world? (analysis & interpretation)
Applications:- Manufacturing and inspection; QA- Robot navigation- Autonomous vehicles- Guiding tools for blind- Security and monitoring- Object/face recognition; OCR.- Medical Applications- Visualization; NVS- Visual communication- Digital libraries and video search- Video manipulation and editing
![Page 4: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/4.jpg)
Tentative Schedule
Lessons 7-9 (Michal): Motion and video analysis
Lesson 10 (Ronen): Image Segmentation
Lesson 11 (Ronen): Photometry
Lesson 12 (Shimon): Object recognition
Lessons 13-14 (Shimon): Human Vision
3 programming exercises (MATLAB) -- CAN SUBMIT IN PAIRS
3-4 theoretical exercises -- MUST SUBMIT INDIVIDUALLY
EXAM
Lessons 1-3 (Michal): Basic Image Processing
Lessons 4-6 (Ronen): Stereo and Structure from Motion
![Page 5: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/5.jpg)
Digital Images
World Camera Digitizer DigitalImage
Image Formation:
(i) What determines where the image of a 3D point appears on the 2D image?
(ii) What determines how bright that image point is?
(iii) How is a digital image represented?
(iv) Some simple operations on 2D images?today
![Page 6: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/6.jpg)
Digital Images
World Camera Digitizer DigitalImage
0 10 10 15 50 70 80
0 0 100 120 125 130 130
0 35 100 150 150 80 50
0 15 70 100 10 20 20
0 15 70 0 0 0 15
5 15 50 120 110 130 110
5 10 20 50 50 20 250
PIXEL
Typically: 0 = black 255 = white
(picture element)
![Page 7: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/7.jpg)
64 60 69 100 149 151 176 182 17965 62 68 97 145 148 175 183 18165 66 70 95 142 146 176 185 18466 66 68 90 135 140 172 184 18466 64 64 84 129 134 168 181 18259 63 62 88 130 128 166 185 18060 62 60 85 127 125 163 183 17862 62 58 81 122 120 160 181 17663 64 58 78 118 117 159 180 176
![Page 8: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/8.jpg)
210 209 204 202 197 247 143 71 64 80 84 54 54 57 58 206 196 203 197 195 210 207 56 63 58 53 53 61 62 51 201 207 192 201 198 213 156 69 65 57 55 52 53 60 50 216 206 211 193 202 207 208 57 69 60 55 77 49 62 61 221 206 211 194 196 197 220 56 63 60 55 46 97 58 106 209 214 224 199 194 193 204 173 64 60 59 51 62 56 48 204 212 213 208 191 190 191 214 60 62 66 76 51 49 55 214 215 215 207 208 180 172 188 69 72 55 49 56 52 56 209 205 214 205 204 196 187 196 86 62 66 87 57 60 48 208 209 205 203 202 186 174 185 149 71 63 55 55 45 56 207 210 211 199 217 194 183 177 209 90 62 64 52 93 52 208 205 209 209 197 194 183 187 187 239 58 68 61 51 56 204 206 203 209 195 203 188 185 183 221 75 61 58 60 60 200 203 199 236 188 197 183 190 183 196 122 63 58 64 66 205 210 202 203 199 197 196 181 173 186 105 62 57 64 63
x = 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72
414243444546474849505152535455
Grayscale Image
y =
![Page 9: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/9.jpg)
Three types of images:
– Gray-scale images I(x,y) [0..255]
– Binary images I(x,y) {0 , 1}
– Color images IR(x,y) IG(x,y) IB(x,y)
![Page 10: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/10.jpg)
Color Image
![Page 11: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/11.jpg)
Effects of down-sampling (reducing number of pixels)
4 x 4
16 x 16
64 x 64
8 x 8
32 x 32
128 x 128
![Page 12: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/12.jpg)
2 gray levels(1 bit/pixel)
BINARY IMAGE
4 gray levels(2 bits/pixel)
8 gray levels(3 bits/pixel)
16 gray levels(4 bits/pixel)
256 gray levels(8 bits/pixel)
Effects of reducing number of gray levels
![Page 13: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/13.jpg)
Occurrence(# of pixels)
Gray Level
The Image Histogram
H(k) = #pixels with gray-level k
Normalized histogram: Hnorm(k)=H(k)/N (N = # pixels in the image)
Histogram = The gray-level distribution:
Continuous probability density function: 1)( dxxp
k
norm kH 1)(
![Page 14: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/14.jpg)
The Image Histogram (Cont.)
PI(k)
k
PI(k)
k
1
1
0.5
PI(k)
k
0.1
![Page 15: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/15.jpg)
PI(k)
k0.5
0.1PI(k)
k
0.1
Histogram Stretching
![Page 16: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/16.jpg)
k k
)( jAkAifjk ideal
)(khideal)(kh
k k
x
dkkhxA0
)()( x
idealideal dkkhxA0
)()(
Histogram Equalization
![Page 17: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/17.jpg)
0 50 100 150 200 2500
500
1000
1500
2000
2500
3000
3500
0 50 100 150 200 2500
500
1000
1500
2000
2500
3000
3500
Original Equalized
Histogram Equalization
![Page 18: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/18.jpg)
0 50 100 150 200 2500
500
1000
1500
2000
2500
3000
0 50 100 150 200 2500
500
1000
1500
2000
2500
3000
Histogram Equalization
Original Equalized
![Page 19: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/19.jpg)
k
Aa
k
Ab
)( jAkAifjk ba
Histogram SpecificationTransforms an image so that its histogram matches that of another
image
(e.g., for comparing two images of the same scene acquired under different lighting condition)
aI
bI
![Page 20: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/20.jpg)
Image Enhancement
1) Gray value (histogram) Domain
2) Spatial Domain
3) Frequency Domain
- Histogram stretching
- Histogram equalization
- Histogram specification
- Gamma correction
etc...
noisy image(salt & pepper noise)
![Page 21: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/21.jpg)
Spatial Operations
g(x,y) = 1/M f(n,m)(n,m) inS
S = neighborhood of pixel (x,y)M = number of pixels in neighborhood Se.g.,
Examples of neighborhoods S:
3 x 3
5 x 5
Replace center pixel with average/median level:
(averaging mask; weighted mask; median filter…)
![Page 22: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/22.jpg)
Salt & Pepper Noise
3 X 3 Average 5 X 5 Average
7 X 7 Average Median
Noise Cleaning
![Page 23: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/23.jpg)
Salt & Pepper Noise
3 X 3 Average 5 X 5 Average
7 X 7 Average Median
Noise Cleaning
![Page 24: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/24.jpg)
Other spatial filters
Are strong brightness variations always noise…?
![Page 25: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/25.jpg)
Edge Detection
![Page 26: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/26.jpg)
![Page 27: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/27.jpg)
![Page 28: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/28.jpg)
Edge TypesLine Edge
gray
val
ue
xedge edge
gray
val
ue
xedge edge
Step Edge
![Page 29: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/29.jpg)
Edge Detection by Differentiation
gray
val
ue
x
1D image f(x)
1st derivative f'(x)
threshold|f'(x)|
Edge Pixels:|f'(x)| > Threshold
![Page 30: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/30.jpg)
Original image
x derivative Gradient magnitudey derivative
![Page 31: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/31.jpg)
Vertical edges Horizontal edges
Edge Detection
x
f
y
f
Image f
![Page 32: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/32.jpg)
Edge Detection
Image f f
![Page 33: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/33.jpg)
Image Sharpening
Blurry Image Laplacian Sharpened Image
Also Laplacian; Zero-crossings; Edge sharpening; etc….
![Page 34: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/34.jpg)
The End... Exercise#1: Noise Cleaning -- on course website
(+ Matlab tutorial) DUE: Nov. 10 (in 2 weeks)
Course mailing list:
Send email to <hassner@wisdom….>
Vision & Robotics Seminar:
send email <leah@wisdom…> ask to be added to “seminar13” mailing list
![Page 35: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/35.jpg)
![Page 36: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/36.jpg)
Panoramic Mosaic ImageOriginal video clip
Generated Mosaic image
![Page 37: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/37.jpg)
Original
Outliers
Original
Synthesized
Video Removal
![Page 38: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/38.jpg)
Image Segmentation
![Page 39: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/39.jpg)
Image Segmentation
![Page 40: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/40.jpg)
Photometric Stereo
![Page 41: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/41.jpg)
Photometric Stereo
![Page 42: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/42.jpg)
![Page 43: Introduction to Computer and Human Vision](https://reader036.vdocument.in/reader036/viewer/2022062422/56812b96550346895d8fb79f/html5/thumbnails/43.jpg)