digital image processing by: eng. mohanned dawoud 1

Digital Image Processing

By: Eng. Mohanned Dawoud

1

Digital image representation

• Digital image is a finite collection of discrete samples (pixels).

• Each pixel having its own discrete value in a finite range.

• The images may be obtained by a digital camera, scanner, electron microscope, ultrasound stethoscope, or any other optical or non-optical sensor.

2

Digital image representation (Cont.)

• Examples of digital image are:• Digital photographs.• Satellite images.• Radiological images (x-rays, mammograms).• Binary images, fax images, engineering

drawings.

3

Digital image representation (Cont.)

• Computer graphics, CAD drawings, and vector graphics reproductions is a possible source of an image.

• One goal of intermediate level image processing may be to reconstruct a model (e.g. vector representation) for a given digital image.

4

Digitization

5

Digitization (Cont.)

The main parameters of the digitization are:

• Image resolution: the number of samples in the grid.

• pixel accuracy: how many bits are used per sample.

6


• The quality of the images increase as the resolution and the bits per pixel increase.

• There are a few exceptions when reducing the number of bits increases the image quality because of increasing the contrast.

• If we have a certain amount of bits to allocate for an image, it makes difference how to choose the digitization parameters.

7


8


Bits per pixel:

Resolution12468

32 321282565127681,024

64 645121,0242,0483,0724,096

128 1282,0484,0968,19212,28816,384

256 2568,19216,38432,76849,15265,536

512 51232,76865,536131,072196,608262,144

1024 1024131,072262,144524,288786,4321,048,576

9


• The properties of human eye implies some upper limits.

• It is known that the human eye can observe at most one thousand different gray levels in ideal conditions.

• In any practical situations 8 bits per pixel (256 gray level) is usually enough.

10


• In a laser quality printing, as in this lecture notes, even 6 bits (64 levels) used.

• If the pixels represent some physical measure and/or the image will be analyzed by a computer, the additional accuracy may be useful.

11


• If the image has very fine structure exceeding the sampling resolution, it may cause so called aliasing effect.‑

• The digitized image has patterns that does not exists in the original

Original signal Digitized signal

12

Color image models

• Visible light is composed of relatively narrow band of frequencies in the electromagnetic energy spectrum approximately between 400 and 700 nm.

• A green object, for example, reflects light with wavelength primarily in the 500 to 570 nm range, while absorbing most of the energy at other wavelengths.

• A white object reflects light that is relatively balanced in all visible wavelengths.

13

Color image models (Cont.)

• According to the theory of the human eye, all colors are seen as variable combinations of the three so-called primary colors red (R), green (G), and blue (B).

• (International Commission on Illumination) designated in 1931 the following specific wavelength values to the primary colors:– blue (B) = 435.8 nm– green (G) = 546.1 nm– red (R) = 700.0 nm

14


• RGB color model

Cyan

YellowRed

WhiteMagenta

Blue

BlackGreen

(0,1,0)

(0,0,1)

(1,0,0)

G ray S

ca le

R

G

B

15


• CMY color model– The CMY color model is closely related to the RGB

model.– Its primary colors are C (cyan), M (magenta), and Y

(yellow).

The RGB to CMY conversion can be performed by:

C R

M G

Y B

1

1

116


• YUV color model– The basic idea in the YUV color model is to

separate the color information apart from the brightness information.

The components of YUV are:Y R G B

U B Y

V R Y

0 3 0 6 0 1. . .

17


• YUV color model– Y represents the luminance of the image, while

U,V consists of the color information i.e. chrominance.

– The luminance component can be considered as a gray-scale version of the RGB image.

18


• YUV color modelThe advantages of YUV compared to RGB are:– The brightness information is separated from the color

information.– The correlations between the color components are

reduced.– Most of the information is collected to the Y component,

while the information content in the U and V is less.

The YUV color system is adopted in the JPEG image compression standard.

19


• YIQ color model– YIQ is a slightly different version of YUV.– It is mainly used in North American television

systems.– Here Y is the luminance component, just as in

YUV.– I and Q correspond to U and V of YUV color

systems.

20


• The RGB to YIQ conversion can be calculated by:

• The YIQ color model can also be described corresponding to YUV:

Y R G B

I R G B

Q R G B

0 299 0 587 0 114

0 596 0 275 0 321

0 212 0 523 0 311

. . .

. . .

. . .

Y R G B

I V U

Q V U

0 3 0 6 0 1

0 74 0 27

0 48 0 41

. . .

. .

. .21

Color image models (Cont.)• HSI color model

– The HSI model consists of hue (H), saturation (S), and intensity (I).

– Intensity corresponds to the luminance component (Y) of the YUV and YIQ models.

– Hue is an attribute associated with the dominant wavelength in a mixture of light waves, i.e. the dominant color as perceived by an observer.

– Saturation refers to relative purity of the amount of white light mixed with hue.

– The advantages of HSI are: • The intensity is separated from the color information (the same holds

for the YUV and YIQ models though). • The hue and saturation components are intimately related to the way

in which human beings perceive color.

22


• All colors lie inside the triangle whose vertices are defined by the three initial colors.

• Let us draw a vector from the central point of the triangle to the color point P.

• The hue (H) is the angle of the vector with respect to the red axis, For example 0 indicates red color, 60 yellow, 120 green, and so on.

• Saturation (S) is the degree to which the color is undiluted by white and is proportional to the distance to the center of the triangle.

Red Green

Blue

CyanMagenta

YellowPH

Red Green

Blue

White

Black

Intensity

23


• The RGB to HSI conversion can be summarized as follows:

HR G R B

R G R B G BB G

HR G R B

R G R B G B

1

360

12

11

360

12

1

2

1

2

cos

cos ,

, if

otherwise

S = 1-3

R G BR G B

I R G B

min , ,

1

3

24


25

Summary

• Good quality photographs needs 24 bits per pixel.

• 8 bits (256 colors) is often sufficient to represent the icons in Windows desktop if the colors are properly chosen.

• A color palette of 256 specially chosen colors may generated to approximate the image.

26

Summary

Image typeTypical bppNo. ofcolors

Commonfile formats

Binary image12JBIG, PCX, GIF, TIFF

Gray-scale8256JPEG, GIF, PNG, TIFF

Color image2416.6 106JPEG, PNG, TIFF

Color palette image8256GIF, PNG

Video image2416.6 106MPEG

27

Basic image processing operations

Changing resolution:• A reduced resolution version of a given image is

sometimes needed for a preview purpose.• A preview image (or thumbnail) must be small

enough to allow fast access but also with sufficient quality so that the original image is still recognizable.

• Sometimes the image resolution may be reduced just for saving memory.

28

Changing resolution

Resolution reduction is formally defined as follows:

• given an image of N M pixels.• generate an image of size N/c M/c pixels

(where c is a zooming factor) so that the visual content of the image is preserved as well as possible.

29

Changing resolution

There are two alternative strategies: • Sub sampling.• Averaging.

In both cases, the input image is divided into blocks of c×c pixels. For each block, one representative pixel is generated to the output image.

30

Changing resolution

Sub sampling• Any of the input pixels is chosen, e.g. the

upper leftmost pixel in the block.Averaging • The pixel depends on the values of all input

pixels in the block.• It could be chosen as the average, weighted

average, or the median of the pixels.

31

Changing resolution

Averaging results in smoother image whereas sub sampling preserves more details.

32

Changing resolution

• The resolution of an image must sometimes be increased.

• For each input pixels there are c×c output pixels to be generated.

• straightforward method simply takes copies of the input pixel but this results in a jagged (blocky) image where the pixels are clearly visible.

33

Changing resolution

• A more sophisticated method known as bilinear interpolation generates the unknown pixel values by taking the linear combination of the four nearest known pixel values.

a

x

j

i

c

b

d

f x a i b a j c a i j a b c d 34

Changing resolution

Resolution enhancementResolution reduction

?

? ? ?

? ? ?? ? ? ? ? ?

? ? ? ? ? ? ? ?? ? ? ?

? ? ? ? ? ? ? ?? ? ? ?

? ? ? ? ? ? ? ?

????

35

Changing resolution

36

Gray-level transforms

• A general gray-level transform can be described as:

y = f(x)where x is the original pixel value and y is the result after transform

• The function f depends only on the pixel value, and some global information in the image given by the frequency distribution of the pixels (i.e. histogram).

37


• The transform can also use prior knowledge of the image given by the user of the image processing application.

• The transform, however, is independent from the neighboring pixel values.

38


Constant addition and negation• The simplest form of global transform are constant

addition (also known as DC-shift) and negation. • Constant addition is used to enhance dark images.• Constant negation can be used for displaying medical

images and photographs on screen with monochrome positive film with the idea of using the resulting negatives as normal slides.

39


• Constant addition: f(x) = x + c• Constant Negation: f(x) = c - x

40


Contrast Stretching• The visible quality of a low contrast image can be

improved by contrast stretching.• This is based on an assumption that the dynamic scale,

or the relevant information of the image is concentrated between two pixel values x1 and x2.

• These values are already prior knowledge.• The scale of the histogram in the range [x1, x2] is

enlarged while the scales below x1 and above x2 are compressed.

41


42


Range compression• A counter example to the previous situation

appear when the dynamic range of an image far exceeds the capability of the display device.

• In this case only the brightest parts of the image are visible.

43


• An effective way to compress the dynamic range of pixel values is to perform the following intensity transform:

Range compression: f(x) = c log( 1 + |x| )

44


Gray-level slicing• Suppose that the gray-level values that are of

particular interest are known.• These pixel values can then be separated from the

background by the gray-level slicing technique.

45


• The method assigns a bright pixel value to the pixels of interest, and a dark value to the rest of the pixels belonging to the "background".

• In the former case, the gray-level slicing technique thus performs a kind of thresholding (explained later).

46


47


Quantization and global thresholding• Quantization is a technique where the number

of gray-levels are reduced.• For 8 bit encoding image, the image can be

quantized for example to 16 levels simply by taking the 4 most significant bits of the pixel values.

48


• This operation performs a uniform quantization, where the range of each gray-level value is equal.

• The applications of quantization can be found in image compression.

• It can also be used as a basic tool in image segmentation.

49


• It can help a human observer to detect possible objects in an image that is otherwise not seen because of the smoothness of the image.

• Quantization generates artificial edges into the images which may be of help in the analysis.

50


• Quantization is also necessary when displaying 256-level gray-scale images on a VGA-display that can only show a maximum of 64 gray-levels.

• Thresholding performs a two-level quantization of the image.

51


• The purpose of thresholding is to classify the image pixels according to some threshold criterion.

• The operation splits the pixels into two (or more) groups in the same manner as the gray-level slicing.

52


53


Histogram equalization• Sometimes the histogram of an image

contains mostly dark pixels; this is the case of an insufficiently exposed photograph.

54


• The image can be enhanced by constant addition.

• histogram equalization is generally more efficient technique for enhancement.

• It is also applicable whenever the contrast of the image is too small for whatever reason.

55


• The idea of the method is to spread the histogram as evenly as possible over the full intensity scale.

56


57


58

Filtering

• Filtering is an image processing operation where the value of a pixel depends on the values of its neighboring pixels.

• Each of the pixels are processed separately with a predefined window (or template, or mask).

59

Filtering

• Weighted sum of the pixels inside the window is calculated using the weights given by a mask as shown below

w1

w4

w7

w2

w5

w8

w3

w6

w9

General mask for filtering with a 33 window.

60

Filtering

• The result of the sum replaces the original value in the processed image:

f x w xi ii

1

9

w1

w4

w7

w2

w5

w8

w3

w6

w9

61

Filtering

• In the case of border pixels, the part of the mask lying outside of the image is assumed to have the same pixel values as that of the border pixels.

62

Filtering

• Note that the filtering is a parallel operation, i.e. the neighboring values used in the calculations are always taken from the original image, not from the processed image.

63

Filtering

Low-pass filtering• Low-pass filtering (or averaging filtering, or

smoothing) reduces the high frequency components (or noise) in the image by averaging the pixel values over a small region (block).

64

Filtering

• Low-pass filtering reduces noise and makes the image generally smoother, especially near the edges.

• The level of smoothing can be changed by increasing the size of the window.

65

Filtering

High-pass filtering• High-pass filtering is the opposite operation to

low-pass filtering.• The low frequency components are eliminated

and only the high frequency components in the image are retained.

66

Filtering

• Sharpening is done by adding the result of the filtering to the original image.

• Sharpening enhances the pixels near edges and makes it easier to observe details in the image.

• The use of negative weights in the mask may result in negative values, thus the pixel values must be scaled back to [0, 255].

67

Filtering

68

Filtering

Median filtering• Low-pass and high-pass filters are in the class

of linear filters; they can always be described by a weighting mask.

• Median filtering, on the other hand, belong to a class of rank filters.

69

Filtering

• In rank filters, the pixels within the window are ranked (or sorted) and the result of the filtering is chosen according to the ordering of the pixel values.

• In median filtering the new value of a pixel is the median of the pixel values in the window.

• The parameter of the filtering is the size and the shape of the filtering window (mask).

70

Filtering

• The median filter is used for removing noise.• It can remove isolated impulsive noise and at

the same time it preserves the edges and other structures in the image.

• Contrary to average filtering it does not smooth the edges.

71

Filtering

72

Filtering

73

Filtering

74

Filtering

75

Filtering

76

Filtering

77

Filtering

78

Segmentation

• Detection of discontinuities• Thresholding.• Region-based segmentation• Watershed Segmentation

79

Introduction to segmentation• The main purpose is to find meaningful regions with

respect to a particular application− To detect homogeneous regions

− To detect edges (boundaries, contours)• Segmentation of non trivial images is one of the difficult

task in image processing. Still under research• Applications of image segmentation include

− Objects in a scene (for object-based retrieval)

− Objects in a moving scene (MPEG4)

− Spatial layout of objects (Path planning for a mobile robots)

80

Principal approaches• Edge based methods

− Based on discontinuity: ex. to partition an image based on abrupt changes in intensity

• Region based methods− Based on similarity: to partition an image into regions that are

similar according to a set of predefined criteria

Solution can be based on intensity, texture, color, motion, etc.

81

Segmentation


82

Detection of discontinuities• 3 basic types of gray-level discontinuities:

−points , lines , edges

• The common way is to run a mask through the image

83

Point detection• A point has been detected if |R| T,

– T is a nonnegative threshold

84

Line detection

• If |Ri| > |Rj| for all ji – the point is within line i.

• Use one mask to detect lines of a given direction

85

Edge Detection

• First derivative – detect if a point is on the edge

• Second derivative – detect the midpoint of the edge (zero-crossing property)

86

Edge detection in noisy images• Examples of a ramp edge corrupted by

random Gaussian noise of mean 0 and = 0.0, 0.1, 1.0 and 10.0.

87

Edge detection: calculating derivatives

• First derivative: magnitude of the gradient

,f

y

fx

f

G

G

y

x

yxyx GGy

f

x

fGG

2

122

2122 ][|f|

• To calculate: apply gradient masks

Roberts: Prewitt: Sobel:

88

Example

89

Edge detection: calculating derivatives

• Second derivative: Laplacian

• To calculate: apply laplacian masks

2

2

2

22 ),(),(

y

yxf

x

yxff

90

Edge detection: Laplacian of Gaussian

• Laplacian combined with smoothing as a precursor to find edges via zero-crossing.

2

2

2)( r

erh

where r2 = x2+y2, and

is the standard deviation

2

2

24

222 )(

r

er

rh

91

Mexican hat

the coefficient must be sum to zero

92

Examplea) Original image

b) Sobel Gradient

c) Spatial Gaussian smoothing function

d) Laplacian mask

e) LoG

f) Threshold LoG

g) Zero crossing

93

Segmentation


94

Thresholdingimage with dark background and a light object

image with dark background and two light objects

• Global – when T is the same for all points of the image• Local or Dynamic – when T depends on (x,y)• Adaptive – when T depends on I(x,y)

95

Global thresholding• Based on visual inspection of histogram• Automatically

– Select an initial estimate T0.– Segment the image using T0: regions G1 and G2

consisting of pixels with gray level values >T0 and T0

– Compute the average gray level values 1 and 2 for the pixels in regions G1 and G2

– T1 = 0.5 (1 + 2)– Repeat until | Ti - Ti+1|< Tth

96

Global thresholding: example

Tth = 03 iterations with result T = 125

97

Adaptive thresholding

98

Multispectral thresholding

99

Segmentation


100

Overview

• Introduction to Regions• Region Segmentation – Approaches• Region Representation• Data Structures• Split and Merge• Color Image Segmentation

101

What is a Region?

• Basic definition :- A group of connected pixels with similar properties.

• Important in interpreting an image because they may correspond to objects in a scene.

• For correct interpretation, image must be partitioned into regions that correspond to objects or parts of an object.

102

Partitioning – How?

• Partitioning into regions done often by using gray values of the image pixels.

• Two general approaches :– Region-based segmentation– Boundary estimation using edge detection

103

Region-based Approach

• Pixels corresponding to an object grouped together and marked.

• Assumption: Points on same object will project to spatially close pixels on the image with similar gray values.

104

Region-based Approach (continued)

Important Questions:1. How do we define similarity measure

S?2. What threshold T do we use? Does it

change or stay constant?3. If we wish to add q’s neighbor r, do we

use S(p, r), S(q, r), or something else?

105

106

Limitation

• The assumption does not hold true in all cases.

• To overcome this, group pixels using given principles and use domain-dependent knowledge.

• Match regions to object models.

Edge Detection Approach

• Segmentation by finding pixels on a region boundary.

• Edges found by looking at neighboring pixels.

• Region boundary formed by measuring gray value differences between neighboring pixels

107

108

Segmentation versus Edge Detection

• Closed boundaries

• Computation based on similarity

• Boundaries formed not necessarily closed

• Computation based on difference

109

Region Segmentation

• Problem Statement:- Given a set of image pixels and a homogeneity predicate

P(.), find a partition of the image into a set of n regions Ri such that

n

i

TrueRi1

Region Segmentation – Problem Statement (Continued)

• A segmentation is the partition of an image R into sub-regions {Ri} such that

• A region can be defined by a predicate P such that P(Ri) = TRUE if all pixels within the region satisfy a specific property.

• P(Ri Rj) = FALSE for i j.

• Region-Growing– Start with seed points– Grow around current seed

regions by attaching pixels of the same property to the growing sub-regions.

• Region splitting and Merging– Apply the predicate to a sub-

region. If it is true, stop splitting. Else, split the region.

– Quadtree is one way of splitting.

– Neighboring sub-regions with the same predicates can be merged.

1

;

s. t. is a connected region

n

i i ji

i

R R R R

R

110

Original Image Region Segmented Image

111

112

Histograms• They are applied globally to the

entire image. This can be a drawback in complex scenes.

• Very useful in initial segmentation on the basis of gray levels.

• Very different scenes may give a strikingly similar histogram

representation.

Histogram for both images

Region Representation

• Different representations suitable to different applications.

• Two general classes:- 1. Array representation - same size arrays, membership arrays 2. Hierarchical representation - pyramid, quad tree

113

Array representation

• Uses array of same size as original image with entries indicating the region to which a pixel belongs.

• Simplest example – binary image

Image Data

Array Region Representation114

Membership Arrays

• Membership arrays (images) commonly called masks. Each region has a mask that indicates which pixels belong to which region.

• Advantage – a pixel can be allowed to be a member of more than one region.

115

116

Hierarchical Representation

• Images represented at many resolutions• As resolution decrease, array size decrease and

some data is lost. More difficult to recover information.

• But, memory and computation requirements are also decreased.

• Used to accommodate various segmentation algorithms which operate at different resolutions.

117

Pyramids

• An ‘n x n’ image represented by the image and ‘k’ reduced versions of the image.

• Pixel at level ‘l’ has combined information from several pixels at level ‘l+1’

• Top level – level 0 – single pixel• Bottom level – original image• Simplest method for resolution reduction is

averaging .

118

Pyramid Structure

119

Quad Trees

• Extension of pyramids for binary images.

• Three types of nodes – white, black, gray. • White or black node – no splitting.

• Gray node – split into 4 sub-regions.

• Each node is either a leaf or has 4 children.

Quad Tree ExampleBinary Image

120

121

Quad Tree ExampleNon-binary image

1122 3344

1122 3344

1122 3344

Not uniformNot uniform

uniformuniform

Quad Tree Example

122

123

Data Structures for Segmentation

• Data structures used for storing region information during operations like splitting and merging.

• Commonly used data structures:- 1. RAG – Region Adjacency Graph 2. Picture trees

124

RAG

• Nodes represent regions; arcs between nodes represent a common boundary between regions.

• Different properties of regions stored in node data structure.

• After initial segmentation, results stored in RAG and regions combined to obtain a better segmentation.

125

RAG Example

126

Picture Trees

• Emphasize inclusion of a region within another region.

• Recursive splitting of an image into component parts. Splitting stops with constant characteristic conditions.

• Quad tree is a special case.

127

Region Merging• Combine regions considered similar based on a few region

characteristics.• Determining similarity between two regions is most

important step.• Approaches for judging similarity based on: Gray values Color Texture Size Shape

128

Segmentation by Morphological Watersheds

• Stable segmentation; continuous segmentation boundaries.

• Topographic - Visualizing in 3 dimensions: two spatial coordinates versus gray levels

129

Watershed 101

• Three types of points: 1. regional minimum 2. drop of water fall to single minimum 3. drop of water fall to more than one

minimum• Catchment basin or watershed• Divide lines or watershed lines

130

Watersheds: Concept and Method

• Watershed lines hold the key.• Basic idea: Hole in regional minimums topography uniformly flooded

from below build dams to prevent merging dam boundaries correspond to divide lines.

• Process continued till max. level of flooding (corresponding to highest gray-level value) is reached.

• Watershed lines form connected path – continuous boundaries.

131

Example of Watershed Segmentation

Initial Image Topographic Surface

Final watersheds

Topics to search

132

• Discrete Wavelet transforms.• Image features.• Video similarity.• Digital watermarking.• Sound echo.

digital image processing by: eng. mohanned dawoud 1

Documents

image resolution

image quality

digitized image

examples of digital

given digital image

digital image processingby

color image models cont

digitization cont