Download - Eece253 01 Intro
EECE/CS 253 Image Processing
Richard Alan Peters II
Department of Electrical Engineering and
Computer Science
Fall Semester 2011
Lecture Notes: Introduction and Overview
This work is licensed under the Creative Commons Attribution-Noncommercial 2.5 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/2.5/ or
send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA.
. 1. %. {. . {. . . . . .
Introduction and Overview
This presentation is an
overview of some of the
ideas and techniques to be
covered during the course.
25 August 2011 1999-2011 by Richard Alan Peters II 2
1. Image formation
2. Point processing and equalization
3. Color correction
4. The Fourier transform
5. Convolution
6. Image sampling, warping, and stitching
7. Spatial filtering
8. Noise reduction
9. Mathematical morphology
10. High dynamic range imaging
11. Image compression
Topics
25 August 2011 1999-2011 by Richard Alan Peters II 3
Wallace and Gromit
Wallace
Gromit likes cheese
reads Electronics for Dogs
http://www.aardman.com/wallaceandgromit/index.shtml
Wallace and Gromit will be subjects of some of the imagery in this introduction.
25 August 2011 1999-2011 by Richard Alan Peters II 4
Image Formation
25 August 2011 1999-2011 by Richard Alan Peters II 5
Image Formation
25 August 2011 1999-2011 by Richard Alan Peters II 6
Image Formation
projection
through lens image of object
25 August 2011 1999-2011 by Richard Alan Peters II 7
Image Formation
projection onto
discrete sensor
array. digital camera
25 August 2011 1999-2011 by Richard Alan Peters II 8
Image Formation
sensors register
average color. sampled image
25 August 2011 1999-2011 by Richard Alan Peters II 9
Image Formation
continuous colors,
discrete locations. discrete real-
valued image
25 August 2011 1999-2011 by Richard Alan Peters II 10
Digital Image Formation: Quantization
continuous color input
dis
cre
te c
olo
r o
utp
ut
continuous colors
mapped to a finite,
discrete set of colors.
25 August 2011 1999-2011 by Richard Alan Peters II 11
Sampling and Quantization
pixel grid
sampled real image quantized sampled &
quantized
25 August 2011 1999-2011 by Richard Alan Peters II 12
Digital Image
a grid of squares,
each of which
contains a single
color
each square is
called a pixel (for
picture element)
Color images have 3 values per
pixel; monochrome images have
1 value per pixel.
25 August 2011 1999-2011 by Richard Alan Peters II 13
Color Images
Are constructed from three
intensity maps.
Each intensity map is pro-
jected through a color filter
(e.g., red, green, or blue, or
cyan, magenta, or yellow) to
create a monochrome image.
The intensity maps are
overlaid to create a color
image.
Each pixel in a color image is
a three element vector.
25 August 2011 1999-2011 by Richard Alan Peters II 14
Color Images On a CRT
25 August 2011 1999-2011 by Richard Alan Peters II 15
Point Processing
original + gamma - gamma + brightness - brightness
original + contrast - contrast histogram EQ histogram mod
25 August 2011 1999-2011 by Richard Alan Peters II 16
Color Processing
requires some
knowledge of
how we see
colors
25 August 2011 1999-2011 by Richard Alan Peters II 17
Eye’s Light Sensors
#(blue) << #(red) < #(green)
cone density near fovea
25 August 2011 1999-2011 by Richard Alan Peters II 18
Color Sensing / Color Perception These are approximations of the responses to the visible spectrum of the “red”, “green”, and “blue” receptors of a typical human eye.
25 August 2011 1999-2011 by Richard Alan Peters II 19
These are approximations of the responses to the visible spectrum of the “red”, “green”, and “blue” receptors of a typical human eye.
The simultaneous red + blue response causes us to perceive a continuous range of hues on a circle. No hue is greater than or less than any other hue.
Color Sensing / Color Perception
25 August 2011 1999-2011 by Richard Alan Peters II 20
lum
ina
nce
hu
e
sa
tura
tion
photo receptors brain
The eye has 3 types of photoreceptors:
sensitive to red, green, or blue light.
The brain transforms RGB into separate
brightness and color channels (e.g., LHS).
Color Sensing / Color Perception
25 August 2011 1999-2011 by Richard Alan Peters II 21
Color Perception
all bands luminance chrominance
red green blue
16× pixelization of:
luminance and chrominance (hue+saturation) are perceived with different resolutions, as are red, green and blue.
25 August 2011 1999-2011 by Richard Alan Peters II 22
Color Perception
all bands luminance chrominance
red green blue
16× pixelization of:
25 August 2011 1999-2011 by Richard Alan Peters II 23
Color Balance
and Saturation
Uniform changes in color
components result in
change of tint.
E.g., if all G pixel values are multiplied by > 1 then the
image takes a green cast.
25 August 2011 1999-2011 by Richard Alan Peters II 24
Color Transformations
222222218
222222185
17122114 236
227106
240171103
240171160
17121171 240
230166
17 17122 121114 171
222 222222 222185 218
240 240171 171103 160
236 240227 230106 166
Image aging: a transformation, , that mapped:
25 August 2011 1999-2011 by Richard Alan Peters II 25
The 2D Fourier Transform of a Digital Image
21 1
0 0
, , ,
ur vciR C
R C
u v
I r c u v eI
1 1 21
0 0
( , )
ur vcR C i
R CRC
r c
u,v I r c eI
Let I(r,c) be a single-band (intensity) digital image with R
rows and C columns. Then, I(r,c) has Fourier representation
where
are the R x C Fourier coefficients.
these complex exponentials are 2D sinusoids.
25 August 2011 1999-2011 by Richard Alan Peters II 26
2D Sinusoids:
orientation
... are plane waves with
grayscale amplitudes,
periods in terms of lengths, ...
2, cos cos sin 1
2 C R
A c rI r c
A
= phase shift
r
c
25 August 2011 1999-2011 by Richard Alan Peters II 27
2D Sinusoids: ... specific orientations,
and phase shifts.
r
c
r
c
25 August 2011 1999-2011 by Richard Alan Peters II 28
The Value of a Fourier Coefficient …
… is a complex number with a real part and an imaginary part.
If you represent that number as a magnitude, A, and a phase, , …
..these represent the amplitude and offset of the sinusoid with frequency and direction .
25 August 2011 1999-2011 by Richard Alan Peters II 29
The Sinusoid from the Fourier Coeff. at (u,v)
25 August 2011 1999-2011 by Richard Alan Peters II 30
Here is the same coefficient plotted as magnitude, A, and a phase, , and displayed in the space domain as a sinusoid.
I |F{I}| [F{I}]
The Fourier Transform of an Image
magnitude phase
25 August 2011 1999-2011 by Richard Alan Peters II 31
Continuous Fourier Transform
The continuous Fourier transform assumes a continuous image exists in a finite region of an infinite plane.
2 ( )I , , i uc vrr c u v e dudvI
2 ( ), I , i uc vru v r c e dcdrI
The BoingBoing Bloggers
25 August 2011 1999-2011 by Richard Alan Peters II 32
Discrete Fourier Transform
The discrete Fourier transform assumes a digital image exists on a closed surface, a torus.
1 1 2
0 0
I ( )
uc vrR C i
C R
v u
r,c u,v eI
1 1 2
0 0
, I ,cu rv
R C iC R
r c
u v r c eI
The BoingBoing Bloggers
25 August 2011 1999-2011 by Richard Alan Peters II 33
Convolution
16,16 cr
0,0 cr
16,16 cr 16,16 cr
16,16 cr
Sum times 1/5
Sums of shifted and
weighted copies of
images or Fourier
transforms.
25 August 2011 1999-2011 by Richard Alan Peters II 34
Convolution Property of the Fourier Transform
The Fourier Transform of a product equals the convolution of the Fourier Transforms. Similarly, the Fourier Transform of a convolution is the product of the Fourier Transforms
.
bycomputed be can nconvolutio spatiala Then,
tionmultiplicapointwiserepresents
nconvolutiorepresents
.}{
Moreover,
.}{
Then,
).,( and ),( TransformsFourier
have ),( and ),( functionsLet
1GFgf
GFgf
GFgf
vuGvuF
crgcrf
-F
F
F
25 August 2011 1999-2011 by Richard Alan Peters II 35
Sampling, Aliasing, & Frequency Convolution
aliasing (the jaggies) no aliasing (smooth lines)
25 August 2011 1999-2011 by Richard Alan Peters II 36
Sampling,
Aliasing, &
Frequency
Convolution (a) (b)
(c) (d)
(a) aliased
(b) power spectrum
(c) unaliased
(d) power spectrum
25 August 2011 1999-2011 by Richard Alan Peters II 37
8× 16×
Resampling
nearest neighbor nearest neighbor
bicubic interpolation bicubic interpolation
(resizing)
25 August 2011 1999-2011 by Richard Alan Peters II 38
Rotation
and motion blur
25 August 2011 1999-2011 by Richard Alan Peters II 39
Image Warping
25 August 2011 1999-2011 by Richard Alan Peters II 40
Panorama via Overlay
Originals
Merged*
*not so good.
Bru
no P
ostle h
ttp://h
ugin
.sou
rceforg
e.net/tu
torials/tw
o-p
hoto
s/en.sh
tml
25 August 2011 1999-2011 by Richard Alan Peters II 41
Panorama via Stitching
Originals
Merged*
*much better.
Bru
no P
ostle h
ttp://h
ugin
.sou
rceforg
e.net/tu
torials/tw
o-p
hoto
s/en.sh
tml
25 August 2011 1999-2011 by Richard Alan Peters II 42
Gaussian LPF in FD Original Image Power Spectrum
Image size: 512x512 SD filter sigma = 8 Frequency Domain (FD) Filtering
25 August 2011 1999-2011 by Richard Alan Peters II 43
Original Image Filtered Image Filtered Power Spectrum
Image size: 512x512 SD filter sigma = 8 FD Filtering: Lowpass
25 August 2011 1999-2011 by Richard Alan Peters II 44
Original Image Filtered Image Filtered Power Spectrum
Image size: 512x512 FD notch sigma = 8 FD Filtering: Highpass
25 August 2011 1999-2011 by Richard Alan Peters II 45
Original Image Filtered Image Filtered Power Spectrum
Image size: 512x512 FD notch sigma = 8 FD Filtering: Highpass
signed image with 0 at middle gray
25 August 2011 1999-2011 by Richard Alan Peters II 46
original blurred sharpened
Spatial Filtering
25 August 2011 1999-2011 by Richard Alan Peters II 47
Spatial Filtering
bandpass
filter
unsharp
masking
original
25 August 2011 1999-2011 by Richard Alan Peters II 48
Spatial Filtering
bandpass
filter
unsharp
masking
original
signed image with 0 at middle gray
25 August 2011 1999-2011 by Richard Alan Peters II 49
Motion Blur vertical regional
zoom rotational
original
25 August 2011 1999-2011 by Richard Alan Peters II 50
color noise blurred image color-only blur
Noise Reduction
25 August 2011 1999-2011 by Richard Alan Peters II 51
5x5 Wiener filter color noise blurred image
Noise Reduction
25 August 2011 1999-2011 by Richard Alan Peters II 52
Noise Reduction
original periodic
noise
frequency
tuned filter
25 August 2011 1999-2011 by Richard Alan Peters II 53
Shot Noise or Salt & Pepper Noise
+ shot noise - shot noise s&p noise
25 August 2011 1999-2011 by Richard Alan Peters II 54
Nonlinear Filters: the Median
s&p noise original median filter
25 August 2011 1999-2011 by Richard Alan Peters II 55
Nonlinear Filters: Min and Maxmin
+ shot noise min filter maxmin filter
25 August 2011 1999-2011 by Richard Alan Peters II 56
Nonlinear Filters: Max and Minmax
- shot noise max filter minmax
25 August 2011 1999-2011 by Richard Alan Peters II 57
Nonlinear Processing: Binary Morphology
“L” shaped SE
O marks origin
Foreground: white pixels
Background: black pixels
Cross-hatched
pixels are
indeterminate.
25 August 2011 1999-2011 by Richard Alan Peters II 58
Used after opening to grow back pieces of the original
image that are connected to the opening.
Permits the removal of small regions that are disjoint
from larger objects without distorting the small
features of the large objects.
original opened reconstructed
Nonlinear Processing: Binary Reconstruction
25 August 2011 1999-2011 by Richard Alan Peters II 59
“L” shaped SE
O marks origin
Foreground: white pixels
Background: black pixels
Cross-hatched
pixels are
indeterminate.
Nonlinear Processing: Grayscale Morphology
25 August 2011 1999-2011 by Richard Alan Peters II 60
Grayscale Morphology: Opening
opening: erosion then dilation opened & original
25 August 2011 1999-2011 by Richard Alan Peters II 61
Grayscale Morphology: Opening
erosion & opening erosion & opening & original
25 August 2011 1999-2011 by Richard Alan Peters II 62
reconstructed opening original
Nonlinear Processing: Grayscale Reconstruction
25 August 2011 1999-2011 by Richard Alan Peters II 63
Forensic Analysis of Photographs
Photographs by Robert Fenton of a battlefield in the Crimean war taken on 23 April 1855.
From Morris, Errol, “Which Came First, the Chicken or the Egg?”, Parts 1-3, New York
Times, Zoom Editorial Section, 25 Sept. 2007 (pt.1), 7 Oct. 2007 (pt.2), 30 Oct. 2007 (pt.3).
Which came first?
25 August 2011 1999-2011 by Richard Alan Peters II 64
Photographs by Robert Fenton of a battlefield in the Crimean war taken on 23 April 1855. From Morris, Errol, “Which Came First, the
Chicken or the Egg?”, Parts 1-3, New York Times, Zoom Editorial Section, 25 Sept. 2007 (pt.1), 7 Oct. 2007 (pt.2), 30 Oct. 2007 (pt.3).
Which came first?
Forensic Analysis of Photographs
25 August 2011 1999-2011 by Richard Alan Peters II 65
Photographs by Robert Fenton of a battlefield in the Crimean war taken on 23 April 1855. From Morris, Errol, “Which Came First, the
Chicken or the Egg?”, Parts 1-3, New York Times, Zoom Editorial Section, 25 Sept. 2007 (pt.1), 7 Oct. 2007 (pt.2), 30 Oct. 2007 (pt.3).
Forensic Analysis of Photographs
Which came first?
25 August 2011 1999-2011 by Richard Alan Peters II 66
High Dynamic Range
(HDR) Imaging
under exposed
Bartlo
miej O
ko
nek
http
://ww
w.easy
hdr.co
m/ex
amp
les.ph
p
25 August 2011 1999-2011 by Richard Alan Peters II 67
High Dynamic Range
(HDR) Imaging
default exposure
Bartlo
miej O
ko
nek
http
://ww
w.easy
hdr.co
m/ex
amp
les.ph
p
25 August 2011 1999-2011 by Richard Alan Peters II 68
High Dynamic Range
(HDR) Imaging
over exposed
Bartlo
miej O
ko
nek
http
://ww
w.easy
hdr.co
m/ex
amp
les.ph
p
25 August 2011 1999-2011 by Richard Alan Peters II 69
High Dynamic Range
(HDR) Imaging
combined
Bartlo
miej O
ko
nek
http
://ww
w.easy
hdr.co
m/ex
amp
les.ph
p
25 August 2011 1999-2011 by Richard Alan Peters II 70
Image Compression
Yoyogi Park, Tokyo, October 1999. Photo by Alan Peters.
Original image is
5244w x 4716h
@ 1200 ppi:
127MBytes
25 August 2011 1999-2011 by Richard Alan Peters II 71
Image Compression: JPEG JP
EG
qu
alit
y le
ve
l File
siz
e in
byte
s
25 August 2011 1999-2011 by Richard Alan Peters II 72
JP
EG
qu
alit
y le
ve
l File
siz
e in
byte
s
Image Compression: JPEG
25 August 2011 1999-2011 by Richard Alan Peters II 73
Image Compositing
Combine parts from separate images to form a new image.
It’s difficult to do well.
Requires relative positions, orientations, and scales to be correct.
Lighting of objects must be consistent within the separate images.
Brightness, contrast, color balance, and saturation must match.
Noise color, amplitude, and patterns must be seamless.
25 August 2011 1999-2011 by Richard Alan Peters II 74
Prof. Peters in his home office. Needs a better shirt.
Image Compositing Example
25 August 2011 1999-2011 by Richard Alan Peters II 75
This shirt demands a monogram.
Image Compositing Example
25 August 2011 1999-2011 by Richard Alan Peters II 76
He needs some more color.
Image Compositing Example
25 August 2011 1999-2011 by Richard Alan Peters II 77
Nice. Now for the way he’d wear his hair if he had any.
Image Compositing Example
25 August 2011 1999-2011 by Richard Alan Peters II 78
He can’t stay in the office like this.
Image Compositing Example
25 August 2011 1999-2011 by Richard Alan Peters II 79
Where’s a hepcat Daddy-O like this belong?
Image Compositing Example
25 August 2011 1999-2011 by Richard Alan Peters II 80
In the studio!
Collar this jive, Jackson. Like crazy, Man !
Image Compositing Example
25 August 2011 1999-2011 by Richard Alan Peters II 81