digital image processing - university of...
TRANSCRIPT
Digital Image ProcessingCOSC 6380/4393
Lecture – 4
Jan. 23rd, 2020
Slides from Dr. Shishir K Shah and Frank (Qingzhong) Liu
Review: Pre-Introduction
• Example: Measure depth of the water in meters at a certain pier
• Yet another representation
• Image as a mode/format to convey information usually for human consumption
2
Review: WHAT ARE DIGITAL IMAGES?• Images are as variable as the types of radiation that exist and
the ways in which radiation interacts with matter:
3
Image formation
• Let’s design a method to capture reflection
– Idea 1: put a piece of film in front of an object
– Do we get a reasonable image?
4
Light Source
Review: Image formation
• Let’s design a method to capture reflection
– Idea 1: put a piece of film in front of an object
– Do we get a reasonable image?
5
Review: Pinhole camera
• Add a barrier to block off most of the rays
– This reduces blurring
– The opening is known as the aperture
– How does this transform the image?
6
Review: Adding a lens
• A lens focuses light onto the film
– There is a specific distance at which objects are “in focus”• other points project to a “circle of confusion”in the image
– Changing the shape of the lens changes this distance
7
Review: OPTICS OF THE EYE
8
Review: PHOTORECEPTORS
• Rods are 1-2 microns in diameter; the cones are 2-3 microns in diameter in the fovea, but increase in diameter away from the fovea (No rods in the fovea)
• Cones are densely packed in the fovea and quickly decrease in density as a function of eccentricity
• Rods increase in density out to approximately 20 degree eccentricity, beyond which their density begins to decline
9
Review: OPTICAL IMAGING GEOMETRY
• We will quantify how the geometry of a 3-D scene projects to the geometry of the image intensities:
object
lens
image
sensingplate,
emulsion, etc
light source(point source)
emitted rays
reflectedrays
focallength
10
UPRIGHT PROJECTION GEOMETRY
X
Y
Z
lens center
f = focal lengthimage plane
Upright Projection Model
x
y
(X, Y, Z) = (0, 0, 0)
11
SOLVING PERSPECTIVE PROJECTION• Using similar triangles we can solve for the relationship
between 3-D coordinates in space and 2-D image coordinates• Redraw the imaging geometry once more, this time making
apparent two pairs of similar triangles:
b
f
B
C
a
f
A
C
a
b
A
B
C
f
12
IMAGE ACQUISITION
object
lens
image
sensingplate,
emulsion, etc
light source(point source)
emitted rays
reflectedrays
focallength
13
14
How Do We Generate A Digital Image?
• Start with a picture of something
Slide by K. R. Castleman
15
How Do We Generate A Digital Image?
• Start with a picture of something
• Lay a grid over the picture
Slide by K. R. Castleman
16
How Do We Generate A Digital Image?
• Start with a picture of something
• Lay a grid over the picture
• Measure the brightness/intensity in each of the squares
Slide by K. R. Castleman
A Simple Image Formation Model
object
lens
image
sensingplate,
emulsion, etc
light source(point source)
emitted rays
reflectedrays
focallength
17
i(x, y)
f(x, y)
r(x, y)
Weeks 1 & 2 18
A Simple Image Formation Model
( , ) ( , ) ( , )
( , ) : intensity at the point ( , )
( , ) : illumination at the point ( , )
(the amount of source illumination incident on the scene)
( , ) : reflectance/transmissivity
f x y i x y r x y
f x y x y
i x y x y
r x y
at the point ( , )
(the amount of illumination reflected/transmitted by the object)
where 0 < ( , ) < and 0 < ( , ) < 1
x y
i x y r x y
𝑓 𝑥, 𝑦 = 𝑖 𝑥, 𝑦 𝑟(𝑥, 𝑦)
Weeks 1 & 2 19
Some Typical Ranges of Reflectance
• Illumination - i(x, y)Lumen — A unit of light flow or luminous flux Lumen per square meter (lm/m2) — The metric unit of measure for
illuminance of a surface– 90,000 lm/m^2 clear day– 10,000 lm/m^2 cloudy day– 1,000 lm/m^2 Indoor Office– 0.1 lm/m^2 clear evening
• Reflectance - r(x, y)
– 0.01 for black velvet– 0.65 for stainless steel– 0.80 for flat-white wall paint – 0.90 for silver-plated metal– 0.93 for snow
Representation of intensity
• If 𝑙 = 𝑓 𝑥, 𝑦
• Let 𝐿𝑚𝑖𝑛 ≤ 𝑙 ≤ 𝐿𝑚𝑎𝑥
• Using previous intensities,
– We may expect, 𝐿𝑚𝑖𝑛 ≅ 10 & 𝐿𝑚𝑎𝑥 ≅ 1000, for Indoor
• 𝐿𝑚𝑖𝑛, 𝐿𝑚𝑎𝑥 → 𝑔𝑟𝑒𝑦 𝑠𝑐𝑎𝑙𝑒
21
How Do We Generate A Digital Image?
• Start with a picture of something
• Lay a grid over the picture
• Measure the brightness in each of the squares
Slide by K. R. Castleman
1 2 3 4 5
22
How Do We Generate A Digital Image?
• Start with a picture of something
• Lay a grid over the picture
• Measure the brightness in each of the squares
• The resulting array of numbers(digits) is the digital image
Slide by K. R. Castleman
23
How Do We Generate A Digital Image?
• Each number represents the brightness (0 – Max) at the corresponding position in the image
• Each number is the “gray level” or “pixel value” of the corresponding pixel.
Slide by K. R. Castleman
25
What are the Pixels?
• Every pixel has a location in the image.
• A pixel’s location is specified by it’s row number and column number (x,y address).
• Every pixel has a gray level value.
Slide by K. R. Castleman
26
What Happened To Our Tiger?
• We only used 169 pixels (not enough).
• Increase to 26 X 26 pixels
• Here he is with 676 pixels.
Slide by K. R. Castleman
27
What Happened To Our Tiger?
• We only used 169 pixels (not enough).
• 52 X 52
• Here he is with 2704 pixels.
Slide by K. R. Castleman
28
What Happened To Our Tiger?
• We only used 169 pixels (not enough).
• 130 X 130
• Here he is with 16,900.
Slide by K. R. Castleman
29
What Happened To Our Tiger?
• We only used 169 pixels (not enough).
• 260 X 260
• Here he is with 67,600 pixels.
Slide by K. R. Castleman
Weeks 1 & 2 30
Image Acquisition
Transform illumination energy into
digital images
Weeks 1 & 2 31
Image Acquisition Using a Single Sensor
Weeks 1 & 2 32
Image Acquisition Using Sensor Strips
Weeks 1 & 2 33
Image Acquisition Process
Weeks 1 & 2 34
Sensor Response Waveform
Transform illumination energy into
digital images
Weeks 1 & 2 36
Response from a raster scan
A / D CONVERSION
• For computer processing, the analog image must
undergo ANALOG / DIGITAL (A/D) CONVERSION -
Consists of sampling and quantization
A / D CONVERSION
• For computer processing, the analog image must
undergo ANALOG / DIGITAL (A/D) CONVERSION -
Consists of sampling and quantization
Sampling• Each video raster is converted from a continuous
voltage waveform into a sequence of voltage samples:
A / D CONVERSION (contd.)
• Video digitizer board interfaces with the video camera
• Some new “all-digital cameras” include A/D inside the camera
Sampled Image• A sampled image is an array of numbers representing the sampled
(row, column) image intensities
• Each of these picture elements is called a pixel
A / D CONVERSION (contd.)
• Typically the image array is square (N x N) with dimensions
that are a power of 2: N = 2 M (for simple computer
addressing)
M = 7 128 x 128 (2 14 ~ 16,000 pixels)
M = 8 256 x 256 (2 16 ~ 65,500 pixels)
M = 9 512 x 512 (2 18 ~ 262,000 pixels)
M = 10 1024 x 1024 (2 20 ~ 1,000,000 pixels)
• Important that the image be sampled sufficiently densely
• Otherwise the image quality will be severely degraded
• This can be expressed mathematically (The Sampling
Theorem) but the effects are very visually obvious
Sampling: Example
VS
169 Samples 67,600 Samples
Review: Representation of intensity
• If 𝑙 = 𝑓 𝑥, 𝑦
• Let 𝐿𝑚𝑖𝑛 ≤ 𝑙 ≤ 𝐿𝑚𝑎𝑥
• Using previous intensities,
– We may expect, 𝐿𝑚𝑖𝑛 ≅ 10 & 𝐿𝑚𝑎𝑥 ≅ 1000
• 𝐿𝑚𝑖𝑛, 𝐿𝑚𝑎𝑥 → 𝑔𝑟𝑒𝑦 𝑠𝑐𝑎𝑙𝑒
QUANTIZATION
• Each pixel gray level is quantized: assigned one of a finite set of
numbers (generally integers indexed from 0 to K-1
• Typically there K = 2 B possible gray levels:
• Each pixel is represented by B bits, where usually 1 B 8
• The pixel intensities or gray levels must be quantized sufficiently
densely so that excessive information is not lost
• This is hard to express mathematically, but again, quantization
effects are visually obvious
DIGITAL IMAGE
REPRESENTATION
• Once an image is digitized (A/D) and stored it is an array of voltage or magnetic potentials
• Not easy to work with from an algorithmic point of view
• The representation that is easiest to work with from an algorithmic perspective is that of a matrix of integers
Matrix Image Representation• Denote a (square) image matrix I = [I(i, j); 0 < i, j < N-1]
where
• (i, j) = (row, column)
• I(i, j) = image value at coordinate or pixel (i, j)
DIGITAL IMAGE
REPRESENTATION (contd.)
• Example - Matrix notation
• Example - Pixel notation - an N x N image
What’s the minimum number
of bits/pixel allocated?
DIGITAL IMAGE
REPRESENTATION (contd.)• Example - Binary Image
(2-valued, usually
BLACK and WHITE)
• Another way of depicting the
image:
Weeks 1 & 2 48
Representing Digital Images
• Discrete intensity interval [0, L-1], L=2k
• Aka. Dynamic Range
• The number b of bits required to store a M × N digitized image
Weeks 1 & 2 49
Representing Digital Images
• Discrete intensity interval [0, L-1], L=2k
• The number b of bits required to store a M × N digitized image
total bits = M × N × k
Weeks 1 & 2 50
Representing Digital Images
Weeks 1 & 2 51
Spatial and Intensity Resolution
• Spatial resolution— A measure of the smallest discernible detail in an image
— stated with line pairs per unit distance, dots (pixels) per unit distance, dots per inch (dpi)
• Intensity resolution— The smallest discernible change in intensity level
— stated with 8 bits, 12 bits, 16 bits, etc.
Weeks 1 & 2 52
Spatial Resolution
Weeks 1 & 2 53
Intensity Resolution
Weeks 1 & 2 54
Spatial and Intensity Resolution
Resampling
• Once the image is acquired.
• How to
– Enlarge an image
– Shrink an image
– Zoom in
• Zooming Example:
– Initial image size = 500 X 500
– Required image size (= X 1.5) = 750 X 750
Zooming
8 X 5 Image
3 X 3 Image
Create a Grid5 X 8 Image
3 X 3 Image
Fill in values by preserving some sense spatial relationship between intensity values
Shrink5 X 8 Image
3 X 3 Image
Overlap5 X 8 Image
3 X 3 Image
Fill in values preserving spatial relationship
How to fill values5 X 8 Image
3 X 3 Image
Fill in values by preserving some sense spatial relationship between intensity values
Weeks 1 & 2 62
Image Interpolation
• Interpolation — Process of using known data to estimate unknown values
e.g., zooming, shrinking, rotating, and geometric correction
• Interpolation (sometimes called resampling) — an imaging method to increase (or decrease) the number of pixels in a digital image.
Some digital cameras use interpolation to produce a larger image than the
sensor captured or to create digital zoom
http://www.dpreview.com/learn/?/key=interpolation
Interpolation: Nearest Neighbor5 X 8 Image
3 X 3 Image
Fill in values preserving spatial relationship
Interpolation: Nearest Neighbor5 X 8 Image
3 X 3 Image
Fill in values preserving spatial relationship
66
Interpolation: Nearest neighbor
Original
Zoom
67
Interpolation: Nearest neighbor
Zoom
Original
Interpolation (1D)
• Known points 𝑥1𝑎𝑛𝑑 𝑥2 with values
• 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 𝑓 → ℝ
• 𝑓 𝑥1 = 𝐼1and 𝑓 𝑥2 = 𝐼2• How to find the value 𝐼 at point 𝑥
𝑥1 𝑥2
𝐼1 𝐼2
𝑥
𝐼
Linear Interpolation
• Underlying assumption: 𝑓is linear
𝑥1 𝑥2
𝐼1 𝐼2
𝑥
𝐼
Linear Interpolation
• Underlying assumption: 𝑓is linear𝑓 𝑧 = 𝑎𝑧 + 𝑏
𝑥1 𝑥2
𝐼1 𝐼2
𝑥
𝐼
Linear Interpolation
• Underlying assumption: 𝑓is linear𝑓 𝑧 = 𝑎𝑧 + 𝑏
𝑓 𝑥1 = 𝑎𝑥1 + 𝑏
𝑓 𝑥2 = 𝑎𝑥2 + 𝑏
𝑥1 𝑥2
𝐼1 𝐼2
𝑥
𝐼
Linear Interpolation
• Underlying assumption: 𝑓is linear𝑓 𝑧 = 𝑎𝑧 + 𝑏
𝑓 𝑥1 = 𝑎𝑥1 + 𝑏
𝑓 𝑥2 = 𝑎𝑥2 + 𝑏𝑓 𝑥2 − 𝑓 𝑥1 = 𝑎𝑥2 + 𝑏 − (𝑎𝑥1 + 𝑏)
𝐼2 − 𝐼1 = 𝑎 𝑥2 − 𝑥1⇒ 𝐼2 − 𝐼1 ∝ (𝑥2 − 𝑥1)
𝑥1 𝑥2
𝐼1 𝐼2
𝑥
𝐼
Linear Interpolation
𝐼2 − 𝐼1 ∝ 𝑥2 − 𝑥1𝐼 − 𝐼1 ∝ ?
𝑥1 𝑥2
𝐼1 𝐼2
𝑥
𝐼
Linear Interpolation
𝐼2 − 𝐼1 ∝ 𝑥2 − 𝑥1𝐼 − 𝐼1 ∝ 𝑥 − 𝑥1
𝑥1 𝑥2
𝐼1 𝐼2
𝑥
𝐼
Linear Interpolation
𝐼2 − 𝐼1 ∝ 𝑥2 − 𝑥1𝐼 − 𝐼1 ∝ 𝑥 − 𝑥1
Dividing them,𝐼2 − 𝐼1𝐼 − 𝐼1
=𝑥2 − 𝑥1𝑥 − 𝑥1
𝑥1 𝑥2
𝐼1 𝐼2
𝑥
𝐼
Linear Interpolation
Solve for 𝐼𝐼2 − 𝐼1𝐼 − 𝐼1
=𝑥2 − 𝑥1𝑥 − 𝑥1
(𝐼2−𝐼1)𝑥 − 𝑥1𝑥2 − 𝑥1
= 𝐼 − 𝐼1
𝐼 = 𝐼1 + (𝐼2−𝐼1)𝑥 − 𝑥1𝑥2 − 𝑥1
𝑥1 𝑥2
𝐼1 𝐼2
𝑥
𝐼
Linear Interpolation
Solve for 𝐼𝐼2 − 𝐼1𝐼 − 𝐼1
=𝑥2 − 𝑥1𝑥 − 𝑥1
(𝐼2−𝐼1)𝑥 − 𝑥1𝑥2 − 𝑥1
= 𝐼 − 𝐼1
𝐼 = 𝐼1 + (𝐼2−𝐼1)𝑥 − 𝑥1𝑥2 − 𝑥1
𝑥1 𝑥2
𝐼1 𝐼2
𝑥
𝐼
Linear Interpolation
Solve for 𝐼
𝐼 =𝐼1 𝑥2 − 𝑥1 + (𝐼2−𝐼1) 𝑥 − 𝑥1
𝑥2 − 𝑥1
𝐼 =𝐼1 𝑥2 − 𝑥 + 𝐼2 𝑥 − 𝑥1
𝑥2 − 𝑥1
𝐼 =𝐼1 𝑥2 − 𝑥
𝑥2 − 𝑥1+𝐼2 𝑥 − 𝑥1𝑥2 − 𝑥1
𝑥1 𝑥2
𝐼1 𝐼2
𝑥
𝐼
Example: Linear Interpolation
Solve for 𝐼
𝑥1 = 0 𝑥2 = 1
𝐼1 = 10 𝐼2 = 15
𝑥 = 0.3
𝐼 =?
Example: Linear Interpolation
Solve for 𝐼
𝐼 =𝐼1 𝑥2 − 𝑥
𝑥2 − 𝑥1+𝐼2 𝑥 − 𝑥1𝑥2 − 𝑥1
𝑥1 = 0 𝑥2 = 1
𝐼1 = 10 𝐼2 = 15
𝑥 = 0.3
𝐼 =?
Example: Linear Interpolation
Solve for 𝐼
𝐼 =𝐼1 𝑥2 − 𝑥
𝑥2 − 𝑥1+𝐼2 𝑥 − 𝑥1𝑥2 − 𝑥1
𝐼 =10(1 − 0.3)
1 − 0+15 0.3 − 0
1 − 0𝐼 = 7 + 4.5 = 11.5
𝑥1 = 0 𝑥2 = 1
𝐼1 = 10 𝐼2 = 15
𝑥 = 0.3
𝐼 =11.5
Bi-Linear Interpolation(2D)Q11 = (x1, y1),Q12 = (x1, y2),Q21 = (x2, y1),and Q22 = (x2, y2)𝑓 𝑄𝑖 → 𝑖𝑛𝑡𝑒𝑛𝑠𝑖𝑡𝑦 𝑎𝑡 𝑄𝑖Find the value at 𝑃
https://en.wikipedia.org/wiki/Bilinear_interpolation
Bi-Linear Interpolation(2D)Q11 = (x1, y1),Q12 = (x1, y2),Q21 = (x2, y1),and Q22 = (x2, y2)
Find the value at 𝑃
https://en.wikipedia.org/wiki/Bilinear_interpolation
Bi-Linear Interpolation(2D)Q11 = (x1, y1),Q12 = (x1, y2),Q21 = (x2, y1),and Q22 = (x2, y2)
Find the value at 𝑃
https://en.wikipedia.org/wiki/Bilinear_interpolation
Bi-Linear Interpolation(2D)Q11 = (x1, y1),Q12 = (x1, y2),Q21 = (x2, y1),and Q22 = (x2, y2)
Find the value at 𝑃
https://en.wikipedia.org/wiki/Bilinear_interpolation
Example
𝐼 21,14 = 162,
𝐼 21,15 = 95,𝐼 20,14 = 91,𝐼 20,15 = 210𝐼 20.2, 14.5 = ?
https://en.wikipedia.org/wiki/Bilinear_interpolation
Example
𝐼 21,14 = 162,
𝐼 21,15 = 95,𝐼 20,14 = 91,𝐼 20,15 = 210𝐼 20.2, 14.5 = ?
https://en.wikipedia.org/wiki/Bilinear_interpolation
Example
𝐼 21,14 = 162,
𝐼 21,15 = 95,𝐼 20,14 = 91,𝐼 20,15 = 210𝐼 20.2, 14.5 = ?
https://en.wikipedia.org/wiki/Bilinear_interpolation
Example
𝐼 21,14 = 162,
𝐼 21,15 = 95,𝐼 20,14 = 91,𝐼 20,15 = 210𝐼 20.2, 14.5 = ?
https://en.wikipedia.org/wiki/Bilinear_interpolation
Bilinear Interpolation5 X 8 Image
3 X 3 Image
Nearest neighbor Interpolation
Bilinear Interpolation
Nearest neighbor Interpolation
Bilinear Interpolation
Bilinear: Alternative algorithm
• An alternative way to write the solution to the interpolation problem is
• Not linear but quadratic
Weeks 1 & 2 97
Image Interpolation:
Bicubic Interpolation
3 3
3
0 0
( , ) i j
ij
i j
f x y a x y
• The intensity value assigned to point (x,y) is obtained by the
following equation
• The sixteen coefficients are determined by using the sixteen nearest neighbors.
Bilinear Interpolation
Bicubic Interpolation