ComputerVision

Cameras, lenses and sensors

Marc PollefeysCOMP 256

ComputerVision

• Camera Models– Pinhole Perspective Projection– Affine Projection

• Camera with Lenses• Sensing• The Human Eye

Reading: Chapter 1.

Cameras, lenses and sensors

ComputerVision

Images are two-dimensional patterns of brightness values.

They are formed by the projection of 3D objects.

Figure from US Navy Manual of Basic Optics and Optical Instruments, prepared by Bureau of Naval Personnel. Reprinted by Dover Publications, Inc., 1969.

ComputerVision

Animal eye:a looonnng

time ago.

Pinhole perspective projection: Brunelleschi, XVth

Century.Camera obscura: XVIth

Century.

Photographic camera:Niepce, 1816.

ComputerVision Distant objects appear smaller

ComputerVision Parallel lines meet

• vanishing point

ComputerVision Vanishing points

VPL VPRH

VP1VP2

VP3

To different directions correspond different vanishing points

ComputerVision Geometric properties of projection

• Points go to points• Lines go to lines• Planes go to whole image

or half-plane• Polygons go to polygons

• Degenerate cases:– line through focal point yields point– plane through focal point yields line

ComputerVision

Pinhole Perspective Equation

⎪⎪⎩

⎪⎪⎨

⎧

=

=

zyfy

zxfx

''

''

ComputerVision

Affine projection models: Weak perspective projection

0

'where''

zfmmyy

mxx −=⎩⎨⎧

−=−= is the magnification.

When the scene relief is small compared its distance from theCamera, m can be taken constant: weak perspective projection.

ComputerVision

Affine projection models: Orthographic projection

⎩⎨⎧

==

yyxx

'' When the camera is at a

(roughly constant) distancefrom the scene, take m=1.

ComputerVision

Planar pinhole perspective

Orthographicprojection

Spherical pinholeperspective

ComputerVision Limits for pinhole cameras

ComputerVision Camera obscura + lens

ComputerVision Lenses

Snell’s law

n1

sin

α1 = n2

sin α2

Descartes’

law

ComputerVision

Paraxial (or first-order) optics

Snell’s law:

n1

sin

α1 = n2

sin α2

Small angles:

n1

α1

≈

n2

α2R

nndn

dn 12

2

2

1

1 −=+

R γβα

111

hdh+=+=

222 R

βγαdhh

−=−=

⎟⎟⎠

⎞⎜⎜⎝

⎛−=⎟⎟

⎠

⎞⎜⎜⎝

⎛+

22

11 RR d

hhnhdhn

ComputerVision

Thin Lenses

)1(2 and11

'1

−==−

nRf

fzz

Rn

Zn

Z11

*

−=+

Rn

ZZn −

=+1

'1

*

ZRn

Zn 11

* −−

=

'1111ZZR

nR

n−=

−−

−

spherical lens surfaces; incoming light ±

parallel to axis; thickness << radii; same refractive index on both sides

'11

* ZRn

Zn

−−

=

Rnn

dn

dn 12

2

2

1

1 −=+

ComputerVision

Thin Lenses

)1(2 and11

'1 e wher

''

''

−==−

⎪⎩

⎪⎨

⎧

=

=

nRf

fzzzyzy

zxzx

http://www.phy.ntnu.edu.tw/java/Lens/lens_e.html

ComputerVision Thick Lens

ComputerVision The depth-of-field

ComputerVision The depth-of-field

fZo

1 Z1 1

i

=+−−

−− Δ+= iii ZZZ

fZ

Zf

i

i

Zo −=

−

−−yields

dZZ

bZ iii

−− Δ+=

Δ ii Z

bdbZ−

=Δ −

fbdfZfZZ oo

o / ) ( Z Z Z

0oo −+

−=−=Δ −−

Similar formula for Z Z Z oo o−=Δ ++

)( / Z bddZ ii −=−

fZZfZ

o

oi

−

=

)( Z

0o bdfZb

Zdf o

−+=−

ComputerVision The depth-of-field

fbdfZfZZZZZ−+

−=−=Δ −−

/ )(

0

00000

decreases with d, increases with Z0strike a balance between incoming light and sharp depth range

ComputerVision Deviations from the lens model

3 assumptions :

1. all rays from a point are focused onto 1 image point

2. all image points in a single plane

3. magnification is constant

deviations from this ideal are aberrations

ComputerVision Aberrations

chromatic : refractive index function of wavelength

2 types :

1. geometrical

2. chromatic

geometrical : small for paraxial rays

study through 3rd

order optics

ComputerVision Geometrical aberrations

spherical aberration

astigmatism

distortion

coma

aberrations are reduced by combining lenses

ComputerVision Spherical aberration

rays parallel to the axis do not converge

outer portions of the lens yield smaller focal lenghts

ComputerVision Astigmatism

Different focal length for inclined rays

ComputerVision Distortion

magnification/focal length different for different angles of inclination

Can be corrected! (if parameters are know)

pincushion(tele-photo)

barrel(wide-angle)

ComputerVision Coma

point off the axis depicted as comet shaped blob

ComputerVision Chromatic aberration

rays of different wavelengths focused in different planes

cannot be removed completely

sometimes achromatization is achieved formore than 2 wavelengths

ComputerVision

Lens materialsreference wavelengths :

λF = 486.13nmλd = 587.56nmλC = 656.28nm

lens characteristics :

1. refractive index nd2. Abbe number Vd = (nd - 1) / (nF - nC )

typically, both should be highallows small components with sufficient refraction

notation : e.g. glass BK7(517642)nd = 1.517 and Vd = 64.2

ComputerVision

Lens materials

additional considerations :humidity and temperature resistance, weight, price,...

Crown Glass

Fused Quartz & Fused Silica

Plastic (PMMA)

Calcium Fluoride

Saphire

Zinc Selenide

6000

18000

Germanium 14000

9000

100 200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400

WAVELENGTH (nm)

ComputerVision

Vignetting

Figure from http://www.vanwalree.com/optics/vignetting.html

The white openings in the top illustrations denote the entrance pupil, which is the image of the aperture stop seen through all lens elements in front of it and from a position on the optical axis. The bottom illustrations show the lens from the semifield

angle. Here, the white openings correspond to the clear aperture for light that is heading for the image corner.

Darkening of images near the corner

ComputerVision Photographs

(Niepce, “La Table Servie,”

1822)

Milestones: Daguerreotypes (1839)Photographic Film (Eastman,1889)Cinema (Lumière

Brothers,1895)Color Photography (Lumière

Brothers, 1908)Television (Baird, Farnsworth, Zworykin, 1920s)

CCD Devices (1970)more recently CMOS

Collection Harlingue-Viollet.

ComputerVision

Cameras

we consider 2 types :

1. CCD

2. CMOS

ComputerVision CCD

separate photo sensor at regular positionsno scanningcharge-coupled devices (CCDs)area CCDs and linear CCDs2 area architectures :

interline transfer and frame transfer

photosensitive

storage

ComputerVision The CCD camera

ComputerVision CMOS

Same sensor elements as CCDEach photo sensor has its own amplifier

More noise (reduced by subtracting ‘black’ image)Lower sensitivity (lower fill rate)

Uses standard CMOS technologyAllows to put other components on chip‘Smart’ pixels

Foveon4k x 4k sensor0.18μ

process70M transistors

ComputerVision CCD vs. CMOS

• Mature technology• Specific technology• High production cost• High power

consumption• Higher fill rate• Blooming• Sequential readout

• Recent technology• Standard IC technology• Cheap• Low power• Less sensitive• Per pixel amplification• Random pixel access• Smart pixels• On chip integration

with other components

ComputerVision Color cameras

We consider 3 concepts:

1. Prism (with 3 sensors)2. Filter mosaic3. Filter wheel

… and X3

ComputerVision Prism color camera

Separate light in 3 beams using dichroic prismRequires 3 sensors & precise alignmentGood color separation

ComputerVision Prism color camera

ComputerVision Filter mosaic

Coat filter directly on sensor

Demosaicing (obtain full colour & full resolution image)

ComputerVision Filter wheel

Rotate multiple filters in front of lensAllows more than 3 colour bands

Only suitable for static scenes

ComputerVision Prism vs. mosaic vs. wheel

Wheel1

GoodAverageLowMotion3 or more

approach# sensorsSeparationCostFramerateArtefactsBands

Prism3

HighHighHighLow3

High-endcameras

Mosaic1

AverageLowHighAliasing3

Low-endcameras

Scientific applications

ComputerVision new color CMOS sensor

Foveon’s X3

better image qualitysmarter pixels

three layers of pixels. The layers of pixels are embedded in silicon to take advantage of the fact that red, green, and blue light penetrate silicon to different depths

ComputerVision The Human Eye

Helmoltz’sSchematicEye

Reproduced by permission, the American Society of Photogrammetry

andRemote Sensing. A.L. Nowicki, “Stereoscopy.”

Manual of Photogrammetry,Thompson, Radlinski, and Speert

(eds.), third edition, 1966.

ComputerVision The distribution of

rods and cones across the retina

Reprinted from Foundations of Vision, by B. Wandell, SinauerAssociates, Inc., (1995). ©

1995 Sinauer

Associates, Inc.

Cones in the fovea

Rods and cones in the periphery

Reprinted from Foundations of Vision, by B. Wandell, SinauerAssociates, Inc., (1995). ©

1995 Sinauer

Associates, Inc.

ComputerVision Next class

Radiometry: lights and surfaces