![Page 1: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/1.jpg)
Capturing Light… in man and machine
CS194: Image Manipulation, Comp. Vision, and Comp. PhotoAlexei Efros, UC Berkeley, Spring 2020
![Page 3: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/3.jpg)
General CommentsPrerequisites
• Linear algebra!!! (EE16A, Math 54, or Math 110)• Good programming skills (at least CS61B)• Machine Learning experience helpful
Emphasis on programming projects!• Building something from scratch
Graduate Version:• Final project required (not pre-canned), including conference-
style report paper
“No Screens” Policy:• No laptops, no cell phones, no smartphones, etc.
![Page 4: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/4.jpg)
Getting help outside of classCourse Web Page
• http://inst.eecs.berkeley.edu/~cs194-26/
Discussion board: • piazza.comOffice hours• Me: after class plus TBA• see webpage and piazzaHomework Parties• Weekly, evenings in Woz (TBA)Python Image Tutorial• Next Tuesday here at 8pm (hopefully)
![Page 5: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/5.jpg)
Administrative StuffGrading
• Programming Project (60%)• Exam + popup quizzes (20%) Have pen and paper• Final Project (20%) • Class Participation: priceless
Late Policy• Five (5) emergency late days for semester, to be spent
wisely• Max 10% of full credit afterwards
Extra Points• Most projects will have optional “bells & whistles”• These extra points could be used to pad scores on other
projects (but not exams!)
![Page 6: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/6.jpg)
Academic Integrity
• Can discuss projects, but don’t share code
• Don’t look up code or copy from a friend
• If you’re not sure if it’s allowed, ask
• Acknowledge any inspirations
• If you get stuck, come talk to us
![Page 7: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/7.jpg)
Why you should NOT take this class• Project-based class
• No canned problem sets• Not theory-heavy (but will read a few research papers)• No clean rubrics• Open-ended by design
• Need time to think, not just hack• Creativity is a class requirement
• Lots of work…There are easier classes if• you just need some units• you care more about the grade than about learning stuff
• Not worth it if you don’t enjoy it
![Page 8: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/8.jpg)
Now… reasons TO take this class
• It’s your reward after 3 grueling years • You get work with pictures, unleash your
creative potential• Interested in grad school?
![Page 9: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/9.jpg)
Other Questions?
![Page 10: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/10.jpg)
PHOTOGRAPHY
light drawing/ writing
Etymology
![Page 11: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/11.jpg)
Image Formation
Digital Camera
The Eye
Film
![Page 12: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/12.jpg)
Sensor Array
CMOS sensor
![Page 13: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/13.jpg)
Sampling and Quantization
![Page 14: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/14.jpg)
Interlace vs. progressive scan
http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz
![Page 15: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/15.jpg)
Progressive scan
http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz
![Page 16: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/16.jpg)
Interlace
http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz
![Page 17: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/17.jpg)
Rolling Shutter
http://en.wikipedia.org/wiki/Rolling_shutter
![Page 18: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/18.jpg)
Saccadic eye movement
![Page 19: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/19.jpg)
Saccadic eye movement
![Page 20: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/20.jpg)
The Eye
The human eye is a camera!• Iris - colored annulus with radial muscles• Pupil - the hole (aperture) whose size is controlled by the iris• What’s the “film”?
– photoreceptor cells (rods and cones) in the retina
Slide by Steve Seitz
![Page 21: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/21.jpg)
The Retina
Cross-section of eye
Ganglion cell layerBipolar cell layer
Receptor layer
Pigmentedepithelium
Ganglion axons
Cross section of retina
![Page 22: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/22.jpg)
Retina up-close
Light
![Page 23: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/23.jpg)
© Stephen E. Palmer, 2002
Conescone-shaped less sensitiveoperate in high lightcolor vision
Two types of light-sensitive receptors
Rodsrod-shapedhighly sensitiveoperate at nightgray-scale vision
![Page 24: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/24.jpg)
Rod / Cone sensitivity
The famous sock-matching problem…
![Page 25: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/25.jpg)
© Stephen E. Palmer, 2002
Distribution of Rods and Cones
0
150,000100,00050,000
020 40 60 8020406080
Visual Angle (degrees from fovea)
Rods
Cones Cones
Rods
Fovea BlindSpot
# R
ecep
tors
/mm
2
Night Sky: why are there more stars off-center?
![Page 26: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/26.jpg)
Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995
![Page 27: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/27.jpg)
Electromagnetic Spectrum
http://www.yorku.ca/eye/photopik.htm
Human Luminance Sensitivity Function
![Page 28: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/28.jpg)
Why do we see light of these wavelengths?
© Stephen E. Palmer, 2002
…because that’s where theSun radiates EM energy
Visible Light
![Page 29: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/29.jpg)
The Physics of Light
Any patch of light can be completely describedphysically by its spectrum: the number of photons (per time unit) at each wavelength 400 - 700 nm.
400 500 600 700
Wavelength (nm.)
# Photons(per ms.)
© Stephen E. Palmer, 2002
![Page 30: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/30.jpg)
The Physics of Light
# Ph
oton
s
D. Normal Daylight
Wavelength (nm.)
B. Gallium Phosphide Crystal
400 500 600 700
# Ph
oton
s
Wavelength (nm.)
A. Ruby Laser
400 500 600 700
400 500 600 700
# Ph
oton
s
C. Tungsten Lightbulb
400 500 600 700
# Ph
oton
s
Some examples of the spectra of light sources
© Stephen E. Palmer, 2002
![Page 31: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/31.jpg)
The Physics of Light
Some examples of the reflectance spectra of surfaces
Wavelength (nm)
% P
hoto
ns R
efle
cted
Red
400 700
Yellow
400 700
Blue
400 700
Purple
400 700
© Stephen E. Palmer, 2002
![Page 32: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/32.jpg)
The Psychophysical Correspondence
There is no simple functional description for the perceivedcolor of all lights under all viewing conditions, but …...
A helpful constraint:Consider only physical spectra with normal distributions
area
Wavelength (nm.)
# Photons
400 700500 600
mean
variance
© Stephen E. Palmer, 2002
![Page 33: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/33.jpg)
The Psychophysical Correspondence
Mean Hue
yellowgreenblue
# Ph
oton
s
Wavelength© Stephen E. Palmer, 2002
![Page 34: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/34.jpg)
The Psychophysical Correspondence
Variance Saturation
Wavelength
high
medium
low
hi.
med.
low# Ph
oton
s
© Stephen E. Palmer, 2002
![Page 35: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/35.jpg)
The Psychophysical Correspondence
Area Brightness#
Phot
ons
Wavelength
B. Area Lightness
bright
dark
© Stephen E. Palmer, 2002
![Page 36: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/36.jpg)
© Stephen E. Palmer, 2002
400 450 500 550 600 650
REL
ATIV
E AB
SOR
BAN
CE
(%)
W AVELENGTH (nm.)
100
50
440
S
530 560 nm.
M L
Three kinds of cones:
Physiology of Color Vision
• Why are M and L cones so close?• Why are there 3?
![Page 37: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/37.jpg)
Trichromacy
Rods and cones act as filters on the spectrum• To get the output of a filter, multiply its response curve by the
spectrum, integrate over all wavelengths– Each cone yields one number
S
M L
Wavelength
Power
• How can we represent an entire spectrum with 3 numbers?• We can’t! Most of the information is lost
– As a result, two different spectra may appear indistinguishable» such spectra are known as metamers
Slide by Steve Seitz
![Page 38: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/38.jpg)
More Spectra
metamers
![Page 39: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/39.jpg)
© Stephen E. Palmer, 2002
Color Constancy
The “photometer metaphor” of color perception:Color perception is determined by the spectrum of light on each retinal receptor (as measured by a photometer).
![Page 40: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/40.jpg)
© Stephen E. Palmer, 2002
Color Constancy
The “photometer metaphor” of color perception:Color perception is determined by the spectrum of light on each retinal receptor (as measured by a photometer).
![Page 41: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/41.jpg)
© Stephen E. Palmer, 2002
Color Constancy
The “photometer metaphor” of color perception:Color perception is determined by the spectrum of light on each retinal receptor (as measured by a photometer).
![Page 42: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/42.jpg)
© Stephen E. Palmer, 2002
Color Constancy
Do we have constancy over all global color transformations?
60% blue filter Complete inversion
![Page 43: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/43.jpg)
Color Constancy
© Stephen E. Palmer, 2002
Color Constancy: the ability to perceive theinvariant color of a surface despite ecologicalVariations in the conditions of observation.
Another of these hard inverse problems:Physics of light emission and surface reflectionunderdetermine perception of surface color
![Page 44: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/44.jpg)
Camera White Balancing
• Manual• Choose color-neutral object in the photos and normalize
• Automatic (AWB)• Grey World: force average color of scene to grey• White World: force brightest object to white
![Page 45: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/45.jpg)
Color Sensing in Camera (RGB)3-chip vs. 1-chip: quality vs. costWhy more green?
http://www.cooldictionary.com/words/Bayer-filter.wikipedia
Why 3 colors?
Slide by Steve Seitz
![Page 46: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/46.jpg)
Green is in!
R G B
![Page 47: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/47.jpg)
Practical Color Sensing: Bayer Grid
Estimate RGBat ‘G’ cels from neighboring values
http://www.cooldictionary.com/words/Bayer-filter.wikipedia
Slide by Steve Seitz
![Page 48: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/48.jpg)
Color ImageR
G
B
![Page 49: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/49.jpg)
Images in Matlab• Images represented as a matrix• Suppose we have a NxM RGB image called “im”
– im(1,1,1) = top-left pixel value in R-channel– im(y, x, b) = y pixels down, x pixels to right in the bth channel– im(N, M, 3) = bottom-right pixel in B-channel
• imread(filename) returns a uint8 image (values 0 to 255)– Convert to double format (values 0 to 1) with im2double
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.990.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.910.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.920.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.950.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.850.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.330.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.740.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.930.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.990.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.970.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.990.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.910.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.920.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.950.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.850.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.330.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.740.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.930.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.990.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.970.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.990.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.910.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.920.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.950.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.850.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.330.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.740.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.930.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.990.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.970.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
RG
B
rowcolumn
![Page 50: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/50.jpg)
Color spacesHow can we represent color?
http://en.wikipedia.org/wiki/File:RGB_illumination.jpg
![Page 51: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/51.jpg)
Color spaces: RGB
0,1,0
0,0,1
1,0,0
Image from: http://en.wikipedia.org/wiki/File:RGB_color_solid_cube.png
Default color space
R(G=0,B=0)
G(R=0,B=0)
B(R=0,G=0)
RGB cube• Easy for devices• But not perceptual• Where do the grays live?• Where is hue and saturation?
![Page 52: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/52.jpg)
HSV
Hue, Saturation, Value (Intensity)• RGB cube on its vertex
Decouples the three components (a bit)Use rgb2hsv() and hsv2rgb() in Matlab
Slide by Steve Seitz
![Page 53: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/53.jpg)
Color spaces: HSV
Intuitive color space
H(S=1,V=1)
S(H=1,V=1)
V(H=1,S=0)
![Page 54: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/54.jpg)
Color spaces: L*a*b*“Perceptually uniform”* color space
L(a=0,b=0)
a(L=65,b=0)
b(L=65,a=0)
![Page 55: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/55.jpg)
Programming Project #1Prokudin-Gorskii’s Color Photography (1907)
![Page 56: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/56.jpg)
Programming Project #1
![Page 57: Capturing Light… in man and machinecs194-26/sp20/Lectures/CapturingLight.pdfCapturing Light… in man and machine CS194: Image Manipulation, Comp. Vision, and Comp. Photo Alexei](https://reader033.vdocument.in/reader033/viewer/2022042001/5e6dd55e39ef9e4f3a68007b/html5/thumbnails/57.jpg)
Programming Project #1
• How to compare R,G,B channels?• No right answer
• Sum of Squared Differences (SSD):
• Normalized Correlation (NCC):