1 cs 223-b lecture 1 sebastian thrun gary bradski cornea aqueous humor
Post on 21-Dec-2015
215 views
TRANSCRIPT
1
CS 223-BCS 223-BLecture 1Lecture 1
Sebastian Thrun
Gary Bradski
http://robots.stanford.edu/cs223b/index.html
CORNEAAQUEOUSHUMOR
2
Readings
• Computer Vision, Forsyth and Ponce– Chapter 1
• Introductory Techniques for 3D Computer Vision, Trucco and Verri– Chapter 2
3
Lenses and Cameras*
* Slides, where possible, stolen with abandon, many this lecture from Marc Pollefeys comp256, Lect 2
-- Brunelleschi, XVth Century
4
Distant objects appear smaller
A “similar triangle’s” approach to vision. Notes 1.1
Marc Pollefeys
5
Consequences: Parallel lines meet
• There exist vanishing points
Marc Pollefeys
6
Vanishing points
VPL VPRH
VP1VP2
VP3
Different directions correspond to different vanishing points
Marc Pollefeys
7
Implications For Perception*
* A Cartoon Epistemology: http://cns-alumni.bu.edu/~slehar/cartoonepist/cartoonepist.html
Same size things get smaller, we hardly notice…
Parallel lines meet at a point…
8
Implications For Perception 2
Perception must be mapped to a space variant grid
Logrithmic in nature
Steve Lehar
9
The Effect of Perspective
10
Different Projections: Affine projection models: Weak perspective projection
0
'where'
'z
fmmyy
mxx
is the magnification.
When the scene relief is small compared its distance from theCamera, m can be taken constant: weak perspective projection.
Smoosh everything flat onto a parallel plane at distance z0
Marc Pollefeys
11
Affine projection models: Orthographic projection
yy
xx
'
' When the camera is at a(roughly constant) distancefrom the scene, take m=1.
Marc Pollefeys
12
Limits for pinhole cameras
Marc Pollefeys
13
On to Thin Lenses …
Snell’s law
n1 sin 1 = n2 sin 2
Notes 1.2
a b
d e
F
14
Paraxial (or first-order) optics
Snell’s law:
n1 sin 1 = n2 sin 2
Small angles:
n1 1 n22R
nn
d
n
d
n 12
2
2
1
1
R γβα
111
h
d
h
222 R
βγαd
hh
22
11 RR d
hhn
h
d
hn
Sin = y/rTan = y/x Marc Pollefeys
15
Thin Lenses
)1(2 and
11
'
1
n
Rf
fzz
R
n
Z
n
Z
11*
R
n
ZZ
n
1
'
1*
ZR
n
Z
n 11*
'
1111
ZZR
n
R
n
spherical lens surfaces; incoming light parallel to axis; thickness << radii; same refractive index on both sides
'
11* ZR
n
Z
n
R
nn
d
n
d
n 12
2
2
1
1
Notes 1.3 z->Marc Pollefeys
8
16
Thin Lenses summary
)1(2 and
11
'
1 e wher
''
''
n
Rf
fzzz
yzy
z
xzx
http://www.phy.ntnu.edu.tw/java/Lens/lens_e.htmlMarc Pollefeys
17
The depth-of-field
Marc Pollefeys
18
The depth-of-field
fZo
1
Z
1
1
i
iii ZZZ
fZ
Zf
i
i
Zo
yields
d
ZZ
b
Z iii
ii Zbd
bZ
fbdfZ
fZZ ooo /
) ( Z Z Z
0oo
Similar formula for Z Z Z oo o
)( / Z bddZ ii
fZ
ZfZ
o
oi
)(
Z
0o bdfZb
Zdf o
Marc Pollefeys
19
The depth-of-field
fbdfZ
fZZZZZ
/
)(
0
00000
decreases with d, increases with Z0
strike a balance between incoming light and sharp depth range. Notes 1.4
Marc Pollefeys
20
Deviations from the lens model
3 assumptions :
1. all rays from a point are focused onto 1 image point• Remember thin lens small angle assumption
2. all image points in a single plane
3. magnification is constant
Deviations from this ideal are aberrations
0
'
z
fm
Marc Pollefeys
21
Aberrations
chromatic : refractive index function of wavelength
2 types :
1. geometrical
2. chromatic
geometrical : small for paraxial rays
study through 3rd order optics
Marc Pollefeys
22
Geometrical aberrations
spherical aberration
astigmatism
distortion
coma
aberrations are reduced by combining lenses
Marc Pollefeys
23
Spherical aberration
rays parallel to the axis do not converge
outer portions of the lens yield smaller focal lenghts
Marc Pollefeys
24
Astigmatism
Different focal length for inclined rays
Marc Pollefeys
25
Distortion
magnification/focal length different for different angles of inclination
Can be corrected! (if parameters are know)
pincushion(tele-photo)
barrel(wide-angle)
Marc Pollefeys
26
Coma
point off the axis depicted as comet shaped blob
Marc Pollefeys
27
Chromatic aberration
rays of different wavelengths focused in different planes
cannot be removed completely
sometimes achromatization is achieved formore than 2 wavelengths
Marc Pollefeys
28
Vignetting
Marc Pollefeys
29
Calibration Gist: Invert the image formation process
kth collection of points i
Pik
pik
Image plane
x
y
z
0
Cam
era
Rk,Tk
Extrinsic ParamsRotation &Translationto image framecoord. system
f, c, , kIntrinsic Paramsfocuscenter of imageSkew = 0k radial and tangential distortion
(the camera will get several (K) viewsof this grid in rotation)External
coordinatesystem X
Y
Z operator projection theis where),,,,,,( kcfTRPp kkii kk
Note that rotation matrix R has constraints: determinant is 1, inverseis equal to transpose, optimization routine should make use of this.
Then we want the actual projection to be as close as possible toThe point given by the projection operator: over all i pointsand over all k images of grids: kk ii pp
2Argmin],,,,,[
k iiikk kkppkcfTR
• This is typically solved through a gradient decent optimization since the problem is manifestly convex.
• Note that we need a good starting guess for the initial “correct” projection points p’I the optimization then iterates to solution.
• Stereo would then just double the parameters adding left l and right r subscripts and additional summations over r & l.
32
Assumed Perspective Projection
33
Assumed Perspective Projection
34
Cameras
we consider 2 types :
1. CCD
2. CMOS
Marc Pollefeys
35
CCD
separate photo sensor at regular positionsno scanning
charge-coupled devices (CCDs)
area CCDs and linear CCDs2 area architectures : interline transfer and frame transfer
photosensitive
storage
Marc Pollefeys
36
The CCD camera
Marc Pollefeys
37
CMOSSame sensor elements as CCD
Each photo sensor has its own amplifierMore noise (reduced by subtracting ‘black’ image)
Lower sensitivity (lower fill rate)
Uses standard CMOS technologyAllows to put other components on chip
‘Smart’ pixels
Foveon4k x 4k sensor0.18 process70M transistors
Marc Pollefeys
38
CCD vs. CMOS
• Mature technology• Specific technology• High production cost• High power consumption• Higher fill rate• Blooming• Sequential readout
• Recent technology• Standard IC technology• Cheap• Low power• Less sensitive• Per pixel amplification• Random pixel access• Smart pixels• On chip integration
with other components
Marc Pollefeys
39
Colour cameras
We consider 3 concepts:
1. Prism (with 3 sensors)
2. Filter mosaic
3. Filter wheel
… and X3
Marc Pollefeys
40
Prism colour camera
Separate light in 3 beams using dichroic prism
Requires 3 sensors & precise alignment
Good color separation
Marc Pollefeys
41
Prism colour camera
Marc Pollefeys
42
Filter mosaic
Coat filter directly on sensor
Demosaicing (obtain full colour & full resolution image)
Marc Pollefeys
43
Filter wheel
Rotate multiple filters in front of lens
Allows more than 3 colour bands
Only suitable for static scenes
Marc Pollefeys
44
Prism vs. mosaic vs. wheel
Wheel
1
Good
Average
Low
Motion
3 or more
approach
# sensors
Separation
Cost
Frame rate
Artifacts
Bands
Prism
3
High
High
High
Low
3
High-end
cameras
Mosaic
1
Average
Low
High
Aliasing
3
Low-end
cameras
Scientific applications
Marc Pollefeys
45
new color CMOS sensorFoveon’s X3
better image qualitysmarter pixels
Marc Pollefeys
46
The Human Eye
Looking down the optical axis of the eye
Reproduced by permission, the American Society of Photogrammetry andRemote Sensing. A.L. Nowicki, “Stereoscopy.” Manual of Photogrammetry,Thompson, Radlinski, and Speert (eds.), third edition, 1966.
Cross section of the eye
47
Sensors and image processingRGB + B/W happens here
Question: Which way does the light enter?
Light Light
48
Eye cross section
49
The distribution of rods and cones across the retina
Reprinted from Foundations of Vision, by B. Wandell, Sinauer Associates, Inc., (1995). 1995 Sinauer Associates, Inc.
Cones in the fovea
Rods and cones in the periphery
Reprinted from Foundations of Vision, by B. Wandell, Sinauer Associates, Inc., (1995). 1995 Sinauer Associates, Inc.
50
There’s a lot more going on in Vision …i.e. Light and Surfaces
51
Real vision includes invisible inference
52
Real vision includes invisible inference
53
Real vision includes invisible inference