m. wu: enee631 digital image processing (spring'09) point operations spring ’09 instructor:...

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Point OperationsPoint Operations

Spring ’09 Instructor: Min Wu

Electrical and Computer Engineering Department,

University of Maryland, College Park

bb.eng.umd.edu (Select ENEE631 S’09) [email protected]

ENEE631 Spring’09ENEE631 Spring’09Lecture 3 (2/2/2009)Lecture 3 (2/2/2009)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec3 – Point Operations [2]

OverviewOverview

Last Time

– Color representation and perception not a precise spectrum analyzer => perceive color via 3

types of cones

– Human visual properties for monochrome vision not a precise intensity/energy meter

=> depend on surrounding contrast and spatial frequency

Today

– Wrap up monochrome vision: spatial frequency and quality metrics– Point processing (zero-memory operations)

See reading list on the last slide of each lecture notes

Assignment-1 will be posted on course webpage

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

; 2

00

9)


dot

dot

Visual Angle and Spatial FrequencyVisual Angle and Spatial Frequency

Visual angle matters more than absolute size and distance– Smaller but closer object vs. larger but farther object– Eyes can distinguish about 25-30 lines per degree in bright illumination

25 lines per degree translate to 500 lines for distance=4 x screenheight

Spatial Frequency– Measures the extent of spatial transition

in unit of “cycles per visual degree”

Visibility thresholds– Eyes are most sensitive to medium spatial freq.

and least sensitive to high frequencies ~ similar to a band-pass filter

– More sensitive to horizontal and vertical changes than other orientations

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Overall Monochrome Vision ModelOverall Monochrome Vision Model

From Jain’s Fig.3.9 (pp57)


Image Fidelity CriteriaImage Fidelity Criteria

Subjective measures– Examination by human viewers– Goodness scale: excellent, good, fair, poor, unsatisfactory– Impairment scale: unnoticeable, just noticeable, … – Comparative measures

with another image or among a group of images

Objective (Quantitative) measures– Mean square error and variations– Pro:

Simple, less dependent on human subjects, and easy to handle mathematically

– Con: Not always reflect human perception

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Mean-square CriterionMean-square Criterion

Average (or sum) of squared difference of pixel luminance between two images

Signal-to-noise ratio (SNR)

– SNR = 10 log10 ( s2 / e

2 ) in unit of decibel (dB) s

2 image variance e

2 variance of noise or error

– PSNR = 10 log10 ( A2 / e2 )

A is peak-to-peak value PSNR is about 12-15 dB higher than SNR

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Image Enhancement via Point OperationsImage Enhancement via Point Operations

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Point Operations / Intensity TransformPoint Operations / Intensity Transform Basic idea

– “Zero memory” operation each output of a pixel only depend on the

input intensity at the pixel

– Map a given gray or color level u to a new level v, i.e. v = f ( u )

– Doesn’t bring in new info.– But can improve visual appearance or make features easier to detect

Example-1: Color coordinate transformations

– RGB of each pixel luminance + chrominance components etc.

Example-2: Scalar quantization

– quantize pixel luminance/color with fewer bits

input gray level u

outp

ut g

ray

leve

l

v

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

Typical Types of Gray-level TransformationTypical Types of Gray-level Transformation

Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 3)

L


Example: Negative TransformationExample: Negative Transformation

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)



Example: Log TransformationExample: Log Transformation

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 3)L

U ~ logL


Gamma Characteristics & Gamma CorrectionGamma Characteristics & Gamma Correction

Non-linearity in CRT display

– Voltage U vs. Displayed luminance L’ L’ ~ U where = 2.0 ~ 2.5

Use preprocessing to compensate -distortion

– U ~ L 1/

– log(L) gives similar compensation curve to -correction => used for many practical applications

– Camera may have L1/c capturing distortion with c = 1.0-1.7

Power-law transformations are also useful for general purpose contrast manipulation

U

L’

L’ = a U

L

U ~ logL

~ L1/

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


Example of Gamma CorrectionExample of Gamma Correction

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 3)


Luminance HistogramLuminance Histogram

Represents the relative frequency of occurrence of the various gray levels in the image

– For each gray level, count the # of pixels having that level– Can group nearby levels to form a big bin & count #pixels in it

( From Matlab Image Toolbox Guide Fig.10-4 )

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


Luminance Histogram (cont’d)Luminance Histogram (cont’d)

Interpretation

– Treat pixel values as instantiations of a random variable – Histogram is an estimate of the probability distribution of the r.v.

“Unbalanced” histogram doesn’t fully utilize the dynamic range

– Low contrast image ~ histogram concentrating in a narrow luminance range

– Under-exposed image ~ histogram concentrating on the dark side– Over-exposed image ~ histogram concentrating on the bright side

Balanced histogram gives more pleasant look and reveals rich content

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

Example: Balanced and Unbalanced Histograms Example: Balanced and Unbalanced Histograms



Contrast Stretching for Low-Contrast ImagesContrast Stretching for Low-Contrast Images

Stretch the over-concentrated graylevels in histogram via a nonlinear mapping

– Piece-wise linear stretching function– Assign slopes of the stretching region to be greater than 1

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

07

)

expandcompress compress

dynamic range( Images from Matlab Image Toolbox Guide Fig.10-10 & 10-11; Sketch from B. Liu ’06 )


Contrast Stretching: ExampleContrast Stretching: Example

original

stretched

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Clipping and ThresholdingClipping and Thresholding

Clipping

– Special case of contrast stretching with = = 0

– Useful for noise reduction when the signal of interest mostly lie in range [a,b]

Thresholding

– Special case of clipping with a = b = T– Useful for binarization of scanned binary

images documents, signatures, fingerprints

input gray level u

outp

ut g

ray

leve

l

v

a bo

input gray level u

outp

ut g

ray

leve

l

v

To

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Examples of Histogram EqualizationExamples of Histogram Equalization( From Matlab Image

Toolbox Guide Fig.10-10 & 10-11 )

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Equalization Example (cont’d)Equalization Example (cont’d)

original

equalized

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


How to Do Histogram Equalization?How to Do Histogram Equalization?

Approach: map input luminance u to the corresponding v– Use cumulative distribution as an intermediate step

– v will be approximately uniform for u with discrete prob. distribution b/c all pixels in one bin are mapped to a new bin (no

splitting)

gray level u

c.d.

f

P(U<=u)

u0

o

1

255

Fu0

gray level vc.

d.f

P(V<=v)

v0

o

1

255

Fu0

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

© 2

00

2)


The Math Behind Histogram EqualizationThe Math Behind Histogram Equalization Goal: Map the luminance of each pixel to a new value such that the

output image has approximately uniform distribution of gray levels

To find what mapping to use: first model pixel as r.v.– How to generate r.v. with desired distribution? Match c.d.f

Want to transform one r.v. with certain p.d.f. to a new r.v. with uniform p.d.f.– For r.v. U with continuous p.d.f. over [0,1], find c.d.f.– Construct a new r.v. V by a monotonically increasing mapping v(u) s.t.

– Can show V is uniformly distributed over [0,1] FV(v0) = v0 for any v0 FV(v0) = P(V v0) = P( FU(u) v0) = P( U F-1

U(v0) ) = FU( F-1U(v0)

) = v0

Or: pV(v) dv = pU(u) du ; dv/du = pU(u) => pV(v) = 1

– For u in discrete prob. distribution, rounding may be needed => the output v will be approximately uniform

u

UU dxxpuUPuFv0

)(][)(

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Histogram Equalization AlgorithmHistogram Equalization Algorithm

1-L ..., 0, ifor )(

)()( 1

0

L

ii

iiU

xh

xhxp

ux

iU

i

xpv )(

)1(1

'min

min Lv

vvRoundv

v [0,1]

• Map discrete v [0,1] to

v’ { 0, …, L-1 }• vmin is the smallest positive value of v• vmin 0 1 L-1

ux

iU

i

xpv )( Uniform quantization

u v v’

pU(xi)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Histogram Equalization: A Mini-ExampleHistogram Equalization: A Mini-Example

– xi 0 1 2 3 4 5 6 7 (L=8)

– p(xi) 0.1 0.2 0.4 0.15 0.1 0.05 0 0

– v 0.1 0.3 0.7 0.85 0.95 1.0 1.0 1.0

– v’ 0 [1.5] [4.7] [5.8] [6.6] 7 7 7 (L-1)/(1-vmin) = 7.78

0 2 5 6 7 7 7 7

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

)1(1

'min

min Lv

vvRoundv


Summary: Contrast Stretching vs. Histogram Eq.Summary: Contrast Stretching vs. Histogram Eq.

What are in common?

What are different?

input gray level u

outp

ut g

ray

leve

l

v

a bo

gray level u

c.d.

f

P(U<=u)

u0

o

1

255

Fu0

gray level v

c.d.

f

P(V<=v)

v0

o

1

255

Fu0

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Generalization of Histogram EqualizationGeneralization of Histogram Equalization Histogram specification

– Want output v with specified p.d.f. pV(v)

– Basic idea: match c.d.f. Can think as using uniformly

distributed r.v. W as an intermediate step

W = FU(u) = FV(v) V = F-1V (FU(u)

)

– Approximation in the intermediate step needed for discrete r.v.

W1 = FU(u) , W2 = FV(v) take v s.t. its w2 is equal to or just above w1U

MC

P E

NE

E6

31

Slid

es

(cre

ate

d b

y M

.Wu

© 2

00

1/2

00

4)



Visual QuantizationVisual Quantization

Contouring effect– Visible contours on smoothly changing regions for uniform quantized

luminance values with less than 5-6 bits/pixel– Human eyes are sensitive to contours (high contrast between two areas)

How to reduce contour effect at lower bits/pixel?– Use “dithering” to break contours by adding noise before quantization

8 bits / pixel 4 bits / pixel 2 bits / pixel

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Use Dithering to Remove Contour ArtifactsUse Dithering to Remove Contour Artifacts

30 31

28 29

31

30

27 28 29

30 31 32

28

27

27

29

31 32 3331

33

31

30

32

33

original

24 24

24 24

24

24

24 24 24

24 24 40

24

24

24

24

24 40 4024

40

24

24

40

40

quantized (step=16)

+3 -1

+2 -1

+1

+2

-3 +2 -1

0 +2 -1

-1

0

-1

+2

+2 +3 -2-1

-1

0

-3

+2

-1

noise pattern

33 30

30 28

32

32

24 30 28

30 33 31

27

27

26

31

33 35 3130

32

31

27

34

32

original + noise

40 24

24 24

40

40

24 24 24

24 40 24

24

24

24

24

40 40 2424

40

24

24

40

40

quantized version of (original + noise)

artificial contour

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

© 2

00

3)


Summary of Today’s LectureSummary of Today’s Lecture

Image fidelity criteria

Point operations (mostly with zero-memory)– Gamma correction– Contrast stretching– Histogram equalization

Next time– Dithering / Halftoning– 2-D Systems and Transforms

Readings– Gonzalez’s book 3.1—3.3

Question for Today– Finish the proof in histogram equalization

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


4 bits / pixel

ExampleExample

3 bits / pixel


Tool-1: Contrast QuantizationTool-1: Contrast Quantization

[recall] Visual Sensitivity– Non-uniform for luminance– Just noticeable difference is almost uniform

in relative changes (“contrast”): L / L ~ 0.02 About 50 levels of contrast

~ 6 bits with uniform quantizer; ~ 4-5 bits with MMSE quantizer (take account of prob. distributions)

Uniformly quantize contrast instead of luminance– Can help reduce some number of bits required per pixel while

maintaining good visual quality– Overall provide non-uniform quantization on luminance

f(.) lum.-to-contrast

g(.) contrast-to luminance

Quantizeru c c’ u’

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

/20

07

)

0 (black)

255 (white)


Tool-2: “Pseudorandom Noise” QuantizationTool-2: “Pseudorandom Noise” Quantization

Break contours (boundaries of const. gray area)

– Add uniform zero-mean pseudo random noise before quantization keep average values unchanged

– Can achieve reasonable quality with 3-bit quantizer

random noiseU[-A, A]

Quantizeru(m,n) u’(m,n)

(display)

v(m,n) v’(m,n)

_r(m,n)

“dither”

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

/20

07

)


HalftoneHalftone

How about 1-bit quantizer?

– Extreme case of low bits/pixel– Can we obtain a grayscale look

with only black and white? Photo printing in newspaper and books Image display in the old black/white monitor

Change density of black dots

– Low/band-pass filtering effect in HVS can’t resolve individual tiny dots gives a feel of grayscale based on

the average # of black dots per unit area

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

/20

07

)


Dithering for Halftone ImagesDithering for Halftone Images Halftone images

– Extreme case of low bits/pixel: use two colors (b/w) to represent gray shades

Extend the idea of pseudorandom noise quantizer– Upsample each pixel to form a resolution cell of many dots– Form dither signal by tiling halftone screen to be same size as the image– Apply quantizer on the sum of dither signal and image signal

Perceived gray level– Equal to the percentage of black dots perceived in one resolution cell

r(m,n) [0,A]Pseudorandom halftone screen

Luminanceu(m,n) [0,A]

Bi-leveldisplay

v(m,n) v’(m,n)

Threshold

A0

1

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Examples of Halftone ImagesExamples of Halftone Images Trade spatial resolution with pixel depth

– Let eyes serve as a low-pass-filter – The average intensity of a cell matches the

intensity of original pixel (or neighborhood)

– May also apply error diffusion

Applications– Photo printing in newspaper and books– Image display in the old black/white monitor– Often use a periodic “dither screen” as noise

pattern

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)

m. wu: enee631 digital image processing (spring'09) point operations spring ’09 instructor:...

Documents

snrumcp enee631 slides

fewer bitsumcp enee631

spatial frequencytodaywrap

spatial frequencyvisual

pixel luminancecolor

visual appearance

color level u

monochrome visionnot