Download - Microsoft PowerPoint - DIP Lecture
Digital Image Processing
SHADAB KHANARRO
What is Image Processing??
� Essentially it’s a tool for analyzing image data.
� It’s concerned with extracting meaningful data from real world images.
� Digital image processing has evolved enormously in recent times, and is still evolving, one of the hottest topic of research all across the globe.
Typical applications of IP
� Automated visual inspection system:
Checking for objects with defects visually.
� Satellite image processing.
� Classification (OCR), identification
(Handwriting, Fingerprints) etc.
� Can satisfy tight tolerances and it also has no
subjectivity so we can change parameters for
inspection.
Typical application of IP
� Biomedical Field: An extensive use of IP
techniques for improving the dark images
which are typical in biomedical field.
e.g. MEMICA arm system
� Robotics: UGV, UAV, AUV, ROV.
� Miscellaneous: Image forensics, Movies,
Industries, Defense etc.
Traffic Monitoring
Face Detection
Medical IP
Stanley: Beginning of a new era
Morphing
Image representation:
� Computers can not handle continuous images but only arrays of digital numbers.
� Images are represented as a 2-D arrays of points (2-D matrix).
� A point on this 2-D grid (corresponding to the image matrix elements) is called Pixel (Picture element).
� It represents the average irradiance over the area of the pixel
Image Basics:� Image: x, y, f (x, y) distribution in 2-D space. One with finite value &
discrete values of above all is a digital image.� The tristimulus theory of color perception implies that any color can
be obtained from a mix of the three primaries, red, green and blue.� Radiance is the total amount of energy that flows from the light
source, and it is usually measured in watts (W).� Luminance, measured in lumens (lm), gives a measure of the
amount of energy an observer perceives from a light source. For example, light emitted from a source operating in the far infrared region of the spectrum could have significant energy (radiance), but an observer would hardly perceive it; its luminance would be almost zero
� Brightness is a subjective descriptor of light perception that ispractically impossible to measure. It embodies the achromatic notionof intensity and is one of the key factors in describing color sensation.
Overlapping of primaries
Image sensing:
Image acquisition:
f(x,y) = reflectance(x,y) * illumination(x,y)
Reflectance in [0,1], illumination in [0,inf]
Sampling and Quantization
A digital image a[m,n] described in a
2D discrete space is derived from an analog image a(x,y) in a 2D contin
uous space through a sampling process that is frequently referred to a
s digitization. If our samples are d
apart, we can write this as:f [i ,j] = Quantize { f(i d, j d) }
The image can now be represented
as a matrix of integer values
62 79 23 119 120 105 4 0
10 10 9 62 12 78 34 0
10 58 197 46 46 0 0 48
176 135 5 188 191 68 0 49
2 1 1 29 26 37 0 77
0 89 144 147 187 102 62 208
255 252 0 166 123 62 0 31
166 63 127 17 1 0 99 30
S & Q continued…
Gray level resolution
Spatial Resolution
x
y
f(x,y)
Image as function:
Types of images:
� Color Images: A color image is just three functions pasted together. We can write this as a “vector-
valued” function:
� BGR� [0-255]
( , )
( , ) ( , )
( , )
r x y
f x y g x y
b x y
=
Gray scale images
� Each pixel value has only one component,
grayscale value. This represents the
brightness on a scale of 0-255 scale. It is also
stored in memory with R=G=B.
� Back conversion into color images is possible
with some techniques involving mapping of
2^8 to 2^24, uses ANN.
Binary images
� Each pixel can take only two values that is 0
and 1.
� Easiest to work with.
� Fast processing time.
� For rendering it on gray scale,
1 � 255
0 � 0
Image zooming and shrinking
� Zooming can be viewed as oversampling the
image and shrinking can be viewed as
undersampling an image. (NNI)
� Various other techniques such as Bilinear
interpolation, cubic interpolation etc. are
being used for producing better results and
reducing contours in the image.
Zooming
Neighborhood
� or 4 connectivity neighborhood (VN) of a pixel at position (x,y) is given by following locations:
(x+1,y), (x,y+1), (x-1,y), (x,y-1)
� or 8 connectivity neighbor-
-hood of a pixel at (x,y) is given
by following locations:
(x+1,y),(x,y+1),(x-1,y),
(x,y-1),(x+1,y+1),(x+1,y-1)
(x-1,y+1),(x-1,y-1)
N 4
N 8
N 4
N 8
N 4
and N8
= { b: (b,a) =1 } Lcity
= { b: (b,a) =1} Lchess
Distance measurement
� Types of distances between the pixels:
1. or cityblock distance:
= |x-s| + |y-t|
2. or chessboard distance:
= max {|x-s|, |y-t|}
3. Euclidean Distance
D4
D4
D8
D8
D e p,q` a
= x@ s` a2
+ y@ tb c2F G
1
2
ffff
Connected set, component, region, boundary� Let S represent a subset of pixels in an image. Two pixels p and q
are said to be connected in S if there exists a path between themconsisting entirely of pixels in S. For any pixel p in S, the set of pixelsthat are connected to it in S is called a connected component of S. Ifit only has one connected component, then set S is called aconnected set. Let R be a subset of pixels in an image. We call R aregion of the image if R is a connected set. The boundary (alsocalled border or contour) of a region R is the set of pixels in theregion that have one or more neighbors that are not in R. Edgesare formed from pixels with derivative values that exceed a presetthreshold. Depending on the type of connectivity and edge operatorsused, the edge extracted from a binary region will be the same as theregion boundary. Boundary defined.
Image operations:
� Image Processing: image in -> image out
� Image Analysis: image in -> measurements out
� Image Understanding: image in -> high-level description out
As we shall see in further lectures that we shall apply some kind of transformation function on the image such that:
g(x,y) = T{f(x,y)}
Application of this transformation function is done in these ways:
Operation over an image
N2- the output value at a specific
coordinate is dependent on all the values
in the input image.
* Global
P2- the output value at a specific
coordinate is dependent on the input
values in the neighborhood of that same
coordinate.
* Local
Image size = N x N;
neighborhood size = P x
P. The complexity is
specified in operations
per pixel
constant- the output value at a specific
coordinate is dependent only on the
input value at that same coordinate.
* Point
Generic
Complexit
y/Pixel
CharacterizationOperation
Image Enhancement in Spatial Domain
SHADAB KHANARRO
Image enhancement in spatial domain� The principal objective of enhancement is to process
an image so that the result image is more suitable
than the original image for a specific application. There are two broad categories:
� Spatial domain: These approaches are based on direct manipulation of pixels in an image.
� Frequency domain: These techniques are based on modifying the Fourier transform of an image.
Spatial domain basics
� Spatial domain refers to the aggregate of pixels composing an image. Spatial domain methods are
procedures that operate directly on these pixels
g(x,y) = T [f(x,y)]
where f(x, y) is the input image, g(x, y) is the processed image, and T is an operator on f, defined over some neighborhood of (x, y). In addition, T can
operate on a set of input images, such as performing the pixel-by-pixel sum of K images for
noise reduction
Spatial domain basics
� The simplest form of T is when the neighborhood is of size 1*1 (that is, a single pixel). In this case, g depends only on the value of f at (x, y), and T becomes a gray-level (also called an intensity or mapping) transformation function of the form
s = T (r)
where, for simplicity in notation, r and s are variables denoting, respectively, the gray level of f(x, y) and g(x, y) at any point (x, y)
Thresholding function
Point processing and Masks
� Usually enhancement at any point in an image depends on the gray level at that point, techniques in this category often are referred to as point processing.
� The general approach is to use a function of the values of f in a predefined neighborhood of (x, y) to determine the value of g at (x, y). One of the principal approaches in this formulation is based on the use of so-called masks. Basically, a mask is a small 2-D array, such as the one shown in which the values of the mask coefficients determine the nature of the process, such as image sharpening. Enhancement techniques based on this type of approach often are referred to as mask processing or filtering
w(2,1) w(3,1)
w(3,3)
w(2,2)
w(3,2)
w(3,2)
w(1,1)
w(1,2)
w(3,1)
Spatial domain techniques
What is Contrast stretching?
How to enhance the contrast?
Low contrast � image values concentrated near a
narrow range (mostly dark, or mostly bright, or mostly
medium values)
• Contrast enhancement � change the image value
distribution to cover a wide range
• Contrast of an image can be revealed by its histogram
Some basic transformation functions
Image Negatives
s = L-1-r
Log Transformation
s = c log(1+r)
Power law transformation
s = crγ
Image negative
s = L-1-r
Log transformation
s = c log(1+r)
Power law transformation
� The exponent in the
power law transform
is referred to as
“gamma”.
s = crγ
Gamma Correction
Since all image rendering
devices can vary in
gamma error, the images
which are being
telecasted or uploaded
on internet are usually
pre processed to an
averaged gamma value
so that image looks fine
on most of them.
Power law transformation
Magnetic resonance (MR) image of a fractured human spine. (b)–(d) Results of applying the transformation in with c=1 and g=0.6, 0.4, and 0.3, respectively
Power law transformation
(a) Aerial image.
(b)–(d) Results of
applying the
transformation with
c=1 and
g=3.0, 4.0, and
5.0, respectively.
(Original image
for this example
courtesy of
NASA.)
Log/power law transformation
f
Piecewise linear stretching
Contrast stretching
Contraststretching(a) Form of
transformationfunction.
(b) A low-contrastimage.
(c) Resultof contraststretching.
(d) Result ofthresholding.
Contrast stretching
� If and the transformation is a linear function
that produces no changes in gray levels. If ,
and the transformation becomes a
thresholding function that creates a binary image.
Intermediate values of ( ) and ( ) produce
various degrees of spread in the gray levels of the
output image, thus affecting its contrast. In general,
is assumed so that the function is single valued and monotonically increasing. This condition
preserves the order of gray levels.
r1 = s1r2 = s2
r1 = r2
s1 = 0 s2 = L@ 1
r1 , s1r2 , s2
r1 , r2 and s1 , s2
Gray level slicing
(a) Thistransformationhighlights range[A, B] of graylevels and reducesall others to aconstant level.(b) Thistransformationhighlights range[A, B] butpreserves allother levels.(c) An image.(d) Result ofusing thetransformationin (a).
Gray level slicing
Bit plane slicing
� Usually the visually significant data is stored
in higher four bits, rest of the data accounts
towards subtle detail of an image.
� Used when we require to modify the
contribution made by any particular bit plane.
An 8 bit fractal image
A fractal is an image generated by mathematical expressions
Contribution of each bit plane
� The eight bit planes of the image in earlier slide. The number at the bottom, right of each image identifies the bit plane.
Histogram
� The histogram of a digital image with gray
levels in the range [0, L-1] is a discrete
function , where is the gray level
and is the number of pixels in the image
having gray level .
� Loosely speaking, gives an estimate of
the probability of occurrence of gray level .
h rk
` a= nk
nk
rk k th
rk
p r k
` a
r k
Histogram
The graph on the right shows the histogram of the image on left, notice the distribution of discrete lines over the x coordinate( )nk
Histogram
High Contrast imagewhose pixels have a large variety of gray tones. The histogram is not far from a uniform.
Low Contrast image
The histogram will be
narrow and will be
concentrated on toward
the middle of the gray
scale.
� Can two images have same
histogram??
Different images But have the same histogram
More examples of histogram
Histogram Normalization
� It’s a very common to normalize the histogram.
� It’s done by dividing the values by the number of pixels in an image, hence a normalized histogram is given by:
� Note that sum of all components of a normalized histogram is equal to 1
p rk
` a=
nk
n
fffff
Histogram equalization
� Histogram equalization is done in order to
have a uniform spread of histogram over the
plane.
� Let us consider a transformation function of
the form:
s=T(r)
that produces a level s for every pixel having
a pixel value r in the original image.
0 ≤ r ≤ 1
Histogram equalization
� Assumptions for T(r):
(a) T(r) is a single valued and monotonically
increasing function in the interval
and
(b)
� Validity of assumptions
For inverse transformation
Non inversion of gray levels
0 ≤ r ≤ 1
0 ≤ T r` a
≤ 1 for 0 ≤ r ≤ 1
� Consider the pdf of r and the pdf of
s. Then we have,
Consider a transformation function of the
form:
pr r` a
ps s` a
ps s` a
= pr r` a dr
ds
ffffffLLLLL
MMMMM
s = T r` a
= Z0
r
pr w` a
dw
� According to Leibniz’s rule the derivative of a
definite integral with respect to its upper limit is simply the integrand
value at that limit.
� Applying this rule to
equation stated previously we get,
ds
dr
ffffff= d
T r` a
dr
fffffffffff
=d
dr
ffffffZo
r
pr w` a
dw
HLJ
IMK
= pr r` a
� Therefore we have,
= 1
Which shows that applying the transformation function of the form as described, will produce a uniform histogram.
The discrete formulation is:
Thus a processed image is obtained by mapping each pixel with value in the input image into a corresponding pixel with level
in the output image via the transformation function mentioned above
ps s` a
sk = T rk
` a= X
j = 0
k
pr r j
b c
rksk
Histogram
Equalization
Notice how
the vertical
bars get
Separated
and cover
the entire
range in the
Transformed
result
Spatial filtering and Masks
� Sometime we need to manipulate values obtained from
neighboring pixels for forming the output image.
Example: How can we compute an average value of
pixels in a 3x3 region center at a pixel z ?
4
4
67
6
1
9
2
2
2
7
5
2
26
4
4
5
212
1
3
3
4
2
9
5
7
7
35 8222
Pixel z
Image
� How masking is done.
Masking continued….
4
4
67
6
1
9
2
2
2
7
5
2
26
4
4
5
212
1
3
3
4
2
9
5
7
7
35 8222
Pixel z
Step 1. Selected only needed pixels
4
67
6
9
1
3
3
4
……
……
Masking continued…
4
67
6
9
1
3
3
4
……
……
Step 2. Multiply every pixel by 1/9 and then sum up the values
19
16
9
13
9
1
69
17
9
19
9
1
49
14
9
13
9
1
⋅+⋅+⋅+
⋅+⋅+⋅+
⋅+⋅+⋅=y
1 1
1
1
1
1
1
1
19
1
X
Mask or
Window or
Template
Averaging an image
Question: How to compute the 3x3 average values at every pixels?
4
4
67
6
1
9
2
2
2
7
5
2
26
4
4
5
212
1
3
3
4
2
9
5
7
7
Solution: Imagine that we have
a 3x3 window that can be placed
everywhere on the image
Masking Window
4.3
Step 1: Move the window to the first location where we want to
compute the average value and then select only pixels
inside the window.
4
4
67
6
1
9
2
2
2
7
5
2
26
4
4
5
212
1
3
3
4
2
9
5
7
7
Step 2: Compute
the average value
∑∑= =
⋅=3
1
3
1
),(9
1
i j
jipy
Sub image p
Original image
4 1
9
2
2
3
2
9
7
Output image
Step 3: Place the
result at the pixel
in the output image
Step 4: Move the
window to the next
location and go to Step 2
The 3x3 averaging method is one example of the mask
operation or Spatial filtering.
� The mask operation has the corresponding mask (sometimes
called window or template).
� The mask contains coefficients to be multiplied with pixel
values.
w(2,1) w(3,1)
w(3,3)
w(2,2)
w(3,2)
w(3,2)
w(1,1)
w(1,2)
w(3,1)
Mask coefficients
1 1
1
1
1
1
1
1
19
1
Example : moving averaging
The mask of the 3x3 moving average
filter has all coefficients = 1/9
The mask operation at each point is performed by:
1. Move the reference point (center) of mask to the
location to be computed
2. Compute sum of products between mask coefficients
and pixels in subimage under the mask.
p(2,1)
p(3,2)p(2,2)
p(2,3)
p(2,1)
p(3,3)
p(1,1)
p(1,3)
p(3,1)
……
……
Subimage
w(2,1) w(3,1)
w(3,3)
w(2,2)
w(3,2)
w(3,2)
w(1,1)
w(1,2)
w(3,1)
Mask coefficients
∑∑= =
⋅=N
i
M
j
jipjiwy1 1
),(),(
Mask frame
The reference point
of the mask
The spatial filtering on the whole image is given by:
1. Move the mask over the image at each location.
2. Compute sum of products between the mask coefficients
and pixels inside subimage under the mask.
3. Store the results at the corresponding pixels of the
output image.
4. Move the mask to the next location and go to step 2
until all pixel locations have been used.
(a) Original image, of size500*500 pixels. (b)–(f) Resultsof smoothing with squareaveraging filter masks of sizesn=3, 5, 9, 15, and 35,respectively. The blacksquares at the top are of sizes3, 5, 9, 15, 25, 35, 45, and 55pixels, respectively; theirBorders are 25 pixels apart. The letters at the bottom range insize from 10 to 24 points, inincrements of 2 points; the largeletter at the top is 60 points. Thevertical bars are 5 pixelswide and 100 pixels high; theirseparation is 20 pixels. Thediameter of the circles is 25pixels, and their borders are 15pixels apart; their gray levelsrange from 0% to 100%black in increments of 20%.Thebackground of the image is 10%black. The noisy rectanglesare of size 50*120 pixels.
Image smoothening and thresholding
(a) Image from the Hubble Space Telescope. (b) Image processed by a 15*15
averaging mask. (c) Result of thresholding (b)
Order statistics filter
Subimage
Original image
Moving
window
Statistic parameters
Mean, Median, Mode,
Min, Max, Etc.
Output image
Example of filtering
(a) X-ray image of circuit board corrupted by salt-and-pepper noise. (b) Noise reduction with a 3*3 averaging mask. (c) Noise reduction with a 3*3 median filter.
Derivatives of an image
� ANY DEFINITION WE USE FOR A FIRST DERIVATIVE
(1) MUST BE ZERO IN FLAT SEGMENTS (AREAS OF CONSTANT GRAY-LEVEL
VALUES);
(2) MUST BE NONZERO AT THE ONSET OF A GRAY LEVEL
STEP OR RAMP; AND
(3) MUST BE NONZERO ALONG RAMPS.
SIMILARLY, ANY DEFINITION OF A SECOND DERIVATIVE
(1) MUST BE ZERO IN FLAT AREAS;
(2) MUST BE NONZERO AT THE ONSET AND END OF A GRAY-LEVEL STEP
OR RAMP; AND
(3) MUST BE ZERO ALONG RAMPS OF CONSTANT SLOPE.
SINCE WE ARE DEALING WITH DIGITAL QUANTITIES WHOSE VALUES ARE
FINITE,THE MAXIMUM POSSIBLE GRAY-LEVEL CHANGE ALSO IS FINITE, AND THE
SHORTEST DISTANCE OVER WHICH THAT CHANGE CAN OCCUR IS BETWEEN
ADJACENT PIXELS. A FIRST ORDER DIFFERENTIAL EQUATION IS GIVEN AS:
∂ f
∂x
fffffff= f x + 1` a
@ f x` a
� A SECOND ORDER DIFFERENTIAL EQUATION IS
� WE CONSIDER THE IMAGE SHOWN NEXT AND CALCULATE TE FIRST AND SECOND ORDER DERVIVATIVE ALONG A LINE WHICH PASSES THROUGH THE ISOLATED POINT
∂2
f
∂ x2
fffffffffff= f x + 1` a
+ f x@ 1` a
@ 2f x` a
FIRST, WE NOTE THAT THE FIRST-ORDER DERIVATIVE ISNONZERO ALONG THE ENTIRE RAMP, WHILE THESECONDORDER DERIVATIVE IS NONZERO ONLY AT THE ONSETAND END OF THE RAMP. BECAUSE EDGES IN AN IMAGERESEMBLE THIS TYPE OF TRANSITION, WE CONCLUDETHAT FIRST-ORDER DERIVATIVES PRODUCE “THICK”EDGES AND SECOND-ORDER DERIVATIVES, MUCH FINERONES. NEXT WE ENCOUNTER THE ISOLATED NOISE POINT.HERE, THE RESPONSE AT AND AROUND THE POINT ISMUCH STRONGER FOR THE SECOND- THAN FOR THEFIRST-ORDER DERIVATIVE.
FINALLY, IN THIS CASE, THE RESPONSE OF THE TWODERIVATIVES IS THE SAME AT THE GRAY-LEVEL STEP (INMOST CASES WHEN THE TRANSITION INTO A STEP IS NOTFROM ZERO, THE SECOND DERIVATIVE WILL BE WEAKER).
(1) FIRST-ORDER DERIVATIVES GENERALLY PRODUCE
THICKER EDGES IN AN IMAGE.
(2) SECOND-ORDER DERIVATIVES HAVE A STRONGER
RESPONSE TO FINE DETAIL, SUCH AS THIN LINES AND
ISOLATED POINTS.
(3)FIRST ORDER DERIVATIVES GENERALLY HAVE A
STRONGER RESPONSE TO A GRAY-LEVEL STEP.
(4)SECOND ORDER DERIVATIVES PRODUCE A DOUBLE
RESPONSE AT STEP CHANGES IN GRAY LEVEL. WE ALSO
NOTE OF SECOND-ORDER DERIVATIVES THAT, FOR
SIMILAR CHANGES IN GRAY-LEVEL VALUES IN AN IMAGE,
THEIR RESPONSE IS STRONGER TO A LINE THAN TO A
STEP, AND TO A POINT THAN TO A LINE.
USE OF SECOND DERIVATIVES FOR ENHANCEMENT–THE LAPLACIAN
IT CAN BE SHOWN (ROSENFELD AND KAK [1982]) THAT
THE SIMPLEST ISOTROPIC DERIVATIVE OPERATOR IS THE
LAPLACIAN, WHICH, FOR A FUNCTION (IMAGE) F(X, Y) OF
TWO VARIABLES, IS DEFINED AS
SINCE DERIVATIVE OF ANY ORDEWR IS A LINEAR
OPERATOR THE LAPLACIAN OPERATOR IS ALSO A LINEAR
OPERATOR
HENCE THE PARTIAL SECOND ORDER DERIVATIVE ALONG
X- AXIS IS
AND THE PARTIAL SECOND ORDER DERIVATIVE
ALONG Y- AXIS IS
HENCE OUR LAPLACIAN LINEAR OPERATOR’S
DISCRETE FORMULATION IS
� THE MASK WHICH IMPLIES THE ABOVE
FORMULATION IS
(a) Filter mask used to
implement the digital
Laplacian, as defined above
(b) Mask used to implement
an extension of this equation
that includes the diagonal
neighbors.
(c) and (d) are two other
implementations
of the Laplacian.
BECAUSE THE LAPLACIAN IS A DERIVATIVE OPERATOR, ITS USE HIGHLIGHTS
GRAY-LEVEL DISCONTINUITIES IN AN IMAGE AND DEEMPHASIZES REGIONS
WITH SLOWLY VARYING GRAY LEVELS. THIS WILL TEND TO PRODUCE IMAGES
THAT HAVE GRAYISH EDGE LINES AND OTHER DISCONTINUITIES, ALL
SUPERIMPOSED ON A DARK, FEATURELESS BACKGROUND. BACKGROUND
FEATURES CAN BE “RECOVERED” WHILE STILL PRESERVING THE SHARPENING
EFFECT OF THE LAPLACIAN OPERATION SIMPLY BY ADDING THE ORIGINAL
AND LAPLACIAN IMAGES. AS NOTED IN THE PREVIOUS PARAGRAPH, IT IS
IMPORTANT TO KEEP IN MIND WHICH DEFINITION OF THE LAPLACIAN IS USED.
IF THE DEFINITION USED HAS A NEGATIVE CENTER COEFFICIENT, THEN WE
SUBTRACT, RATHER THAN ADD, THE LAPLACIAN IMAGE TO OBTAIN A
SHARPENED RESULT. THUS, THE BASIC WAY IN WHICH WE USE THE
LAPLACIAN FOR IMAGE ENHANCEMENT IS AS FOLLOWS:
(a) Image of the
North Pole of the
moon.
(b) Laplacian
Filtered image.
(c) Laplacian
image scaled for
Display
purposes.
(d) Image
enhanced by
using equation
stated earlier
Unsharp Masking and High Boost Filtering
-1 -1
-1
k+8
-1
-1
-1
-1
-1
-1 0
0
k+4
-1
-1
0
-1
0
Equation:
∇+
∇−=
),(),(
),(),(),(
2
2
yxfyxkf
yxfyxkfyxfhb
The center of the mask is negative
The center of the mask is positive
Result of HB Filtering
(b) Laplacian of (a)
computed with the mask in
Using k=0.
(c) Laplacian
enhanced image
using the mask (b) with
k=1. (d) Same
as (c), but using
k=1.7.
References
� Digital Image Processing [2e], Rafael C
Gonzalez and R E Woods, Prentice Hall
India.
End of lecture 1
I look forward to receive any kind of constructive criticism or suggestions to improve myself in my further presentations.
SHADAB KHAN
+91 99004 04678
Post your doubts at:
E-mail.
Image Processing community at Orkut.