eece253 01 intro

Post on 21-Apr-2015

17 Views

Category:

Documents

6 Downloads

Preview:

Click to see full reader

TRANSCRIPT

EECE/CS 253 Image Processing

Richard Alan Peters II

Department of Electrical Engineering and

Computer Science

Fall Semester 2011

Lecture Notes: Introduction and Overview

This work is licensed under the Creative Commons Attribution-Noncommercial 2.5 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/2.5/ or

send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA.

. 1. %. {. . {. . . . . .

Introduction and Overview

This presentation is an

overview of some of the

ideas and techniques to be

covered during the course.

25 August 2011 1999-2011 by Richard Alan Peters II 2

1. Image formation

2. Point processing and equalization

3. Color correction

4. The Fourier transform

5. Convolution

6. Image sampling, warping, and stitching

7. Spatial filtering

8. Noise reduction

9. Mathematical morphology

10. High dynamic range imaging

11. Image compression

Topics

25 August 2011 1999-2011 by Richard Alan Peters II 3

Wallace and Gromit

Wallace

Gromit likes cheese

reads Electronics for Dogs

http://www.aardman.com/wallaceandgromit/index.shtml

Wallace and Gromit will be subjects of some of the imagery in this introduction.

25 August 2011 1999-2011 by Richard Alan Peters II 4

Image Formation

25 August 2011 1999-2011 by Richard Alan Peters II 5

Image Formation

25 August 2011 1999-2011 by Richard Alan Peters II 6

Image Formation

projection

through lens image of object

25 August 2011 1999-2011 by Richard Alan Peters II 7

Image Formation

projection onto

discrete sensor

array. digital camera

25 August 2011 1999-2011 by Richard Alan Peters II 8

Image Formation

sensors register

average color. sampled image

25 August 2011 1999-2011 by Richard Alan Peters II 9

Image Formation

continuous colors,

discrete locations. discrete real-

valued image

25 August 2011 1999-2011 by Richard Alan Peters II 10

Digital Image Formation: Quantization

continuous color input

dis

cre

te c

olo

r o

utp

ut

continuous colors

mapped to a finite,

discrete set of colors.

25 August 2011 1999-2011 by Richard Alan Peters II 11

Sampling and Quantization

pixel grid

sampled real image quantized sampled &

quantized

25 August 2011 1999-2011 by Richard Alan Peters II 12

Digital Image

a grid of squares,

each of which

contains a single

color

each square is

called a pixel (for

picture element)

Color images have 3 values per

pixel; monochrome images have

1 value per pixel.

25 August 2011 1999-2011 by Richard Alan Peters II 13

Color Images

Are constructed from three

intensity maps.

Each intensity map is pro-

jected through a color filter

(e.g., red, green, or blue, or

cyan, magenta, or yellow) to

create a monochrome image.

The intensity maps are

overlaid to create a color

image.

Each pixel in a color image is

a three element vector.

25 August 2011 1999-2011 by Richard Alan Peters II 14

Color Images On a CRT

25 August 2011 1999-2011 by Richard Alan Peters II 15

Point Processing

original + gamma - gamma + brightness - brightness

original + contrast - contrast histogram EQ histogram mod

25 August 2011 1999-2011 by Richard Alan Peters II 16

Color Processing

requires some

knowledge of

how we see

colors

25 August 2011 1999-2011 by Richard Alan Peters II 17

Eye’s Light Sensors

#(blue) << #(red) < #(green)

cone density near fovea

25 August 2011 1999-2011 by Richard Alan Peters II 18

Color Sensing / Color Perception These are approximations of the responses to the visible spectrum of the “red”, “green”, and “blue” receptors of a typical human eye.

25 August 2011 1999-2011 by Richard Alan Peters II 19

These are approximations of the responses to the visible spectrum of the “red”, “green”, and “blue” receptors of a typical human eye.

The simultaneous red + blue response causes us to perceive a continuous range of hues on a circle. No hue is greater than or less than any other hue.

Color Sensing / Color Perception

25 August 2011 1999-2011 by Richard Alan Peters II 20

lum

ina

nce

hu

e

sa

tura

tion

photo receptors brain

The eye has 3 types of photoreceptors:

sensitive to red, green, or blue light.

The brain transforms RGB into separate

brightness and color channels (e.g., LHS).

Color Sensing / Color Perception

25 August 2011 1999-2011 by Richard Alan Peters II 21

Color Perception

all bands luminance chrominance

red green blue

16× pixelization of:

luminance and chrominance (hue+saturation) are perceived with different resolutions, as are red, green and blue.

25 August 2011 1999-2011 by Richard Alan Peters II 22

Color Perception

all bands luminance chrominance

red green blue

16× pixelization of:

25 August 2011 1999-2011 by Richard Alan Peters II 23

Color Balance

and Saturation

Uniform changes in color

components result in

change of tint.

E.g., if all G pixel values are multiplied by > 1 then the

image takes a green cast.

25 August 2011 1999-2011 by Richard Alan Peters II 24

Color Transformations

222222218

222222185

17122114 236

227106

240171103

240171160

17121171 240

230166

17 17122 121114 171

222 222222 222185 218

240 240171 171103 160

236 240227 230106 166

Image aging: a transformation, , that mapped:

25 August 2011 1999-2011 by Richard Alan Peters II 25

The 2D Fourier Transform of a Digital Image

21 1

0 0

, , ,

ur vciR C

R C

u v

I r c u v eI

1 1 21

0 0

( , )

ur vcR C i

R CRC

r c

u,v I r c eI

Let I(r,c) be a single-band (intensity) digital image with R

rows and C columns. Then, I(r,c) has Fourier representation

where

are the R x C Fourier coefficients.

these complex exponentials are 2D sinusoids.

25 August 2011 1999-2011 by Richard Alan Peters II 26

2D Sinusoids:

orientation

... are plane waves with

grayscale amplitudes,

periods in terms of lengths, ...

2, cos cos sin 1

2 C R

A c rI r c

A

= phase shift

r

c

25 August 2011 1999-2011 by Richard Alan Peters II 27

2D Sinusoids: ... specific orientations,

and phase shifts.

r

c

r

c

25 August 2011 1999-2011 by Richard Alan Peters II 28

The Value of a Fourier Coefficient …

… is a complex number with a real part and an imaginary part.

If you represent that number as a magnitude, A, and a phase, , …

..these represent the amplitude and offset of the sinusoid with frequency and direction .

25 August 2011 1999-2011 by Richard Alan Peters II 29

The Sinusoid from the Fourier Coeff. at (u,v)

25 August 2011 1999-2011 by Richard Alan Peters II 30

Here is the same coefficient plotted as magnitude, A, and a phase, , and displayed in the space domain as a sinusoid.

I |F{I}| [F{I}]

The Fourier Transform of an Image

magnitude phase

25 August 2011 1999-2011 by Richard Alan Peters II 31

Continuous Fourier Transform

The continuous Fourier transform assumes a continuous image exists in a finite region of an infinite plane.

2 ( )I , , i uc vrr c u v e dudvI

2 ( ), I , i uc vru v r c e dcdrI

The BoingBoing Bloggers

25 August 2011 1999-2011 by Richard Alan Peters II 32

Discrete Fourier Transform

The discrete Fourier transform assumes a digital image exists on a closed surface, a torus.

1 1 2

0 0

I ( )

uc vrR C i

C R

v u

r,c u,v eI

1 1 2

0 0

, I ,cu rv

R C iC R

r c

u v r c eI

The BoingBoing Bloggers

25 August 2011 1999-2011 by Richard Alan Peters II 33

Convolution

16,16 cr

0,0 cr

16,16 cr 16,16 cr

16,16 cr

Sum times 1/5

Sums of shifted and

weighted copies of

images or Fourier

transforms.

25 August 2011 1999-2011 by Richard Alan Peters II 34

Convolution Property of the Fourier Transform

The Fourier Transform of a product equals the convolution of the Fourier Transforms. Similarly, the Fourier Transform of a convolution is the product of the Fourier Transforms

.

bycomputed be can nconvolutio spatiala Then,

tionmultiplicapointwiserepresents

nconvolutiorepresents

.}{

Moreover,

.}{

Then,

).,( and ),( TransformsFourier

have ),( and ),( functionsLet

1GFgf

GFgf

GFgf

vuGvuF

crgcrf

-F

F

F

25 August 2011 1999-2011 by Richard Alan Peters II 35

Sampling, Aliasing, & Frequency Convolution

aliasing (the jaggies) no aliasing (smooth lines)

25 August 2011 1999-2011 by Richard Alan Peters II 36

Sampling,

Aliasing, &

Frequency

Convolution (a) (b)

(c) (d)

(a) aliased

(b) power spectrum

(c) unaliased

(d) power spectrum

25 August 2011 1999-2011 by Richard Alan Peters II 37

8× 16×

Resampling

nearest neighbor nearest neighbor

bicubic interpolation bicubic interpolation

(resizing)

25 August 2011 1999-2011 by Richard Alan Peters II 38

Rotation

and motion blur

25 August 2011 1999-2011 by Richard Alan Peters II 39

Image Warping

25 August 2011 1999-2011 by Richard Alan Peters II 40

Panorama via Overlay

Originals

Merged*

*not so good.

Bru

no P

ostle h

ttp://h

ugin

.sou

rceforg

e.net/tu

torials/tw

o-p

hoto

s/en.sh

tml

25 August 2011 1999-2011 by Richard Alan Peters II 41

Panorama via Stitching

Originals

Merged*

*much better.

Bru

no P

ostle h

ttp://h

ugin

.sou

rceforg

e.net/tu

torials/tw

o-p

hoto

s/en.sh

tml

25 August 2011 1999-2011 by Richard Alan Peters II 42

Gaussian LPF in FD Original Image Power Spectrum

Image size: 512x512 SD filter sigma = 8 Frequency Domain (FD) Filtering

25 August 2011 1999-2011 by Richard Alan Peters II 43

Original Image Filtered Image Filtered Power Spectrum

Image size: 512x512 SD filter sigma = 8 FD Filtering: Lowpass

25 August 2011 1999-2011 by Richard Alan Peters II 44

Original Image Filtered Image Filtered Power Spectrum

Image size: 512x512 FD notch sigma = 8 FD Filtering: Highpass

25 August 2011 1999-2011 by Richard Alan Peters II 45

Original Image Filtered Image Filtered Power Spectrum

Image size: 512x512 FD notch sigma = 8 FD Filtering: Highpass

signed image with 0 at middle gray

25 August 2011 1999-2011 by Richard Alan Peters II 46

original blurred sharpened

Spatial Filtering

25 August 2011 1999-2011 by Richard Alan Peters II 47

Spatial Filtering

bandpass

filter

unsharp

masking

original

25 August 2011 1999-2011 by Richard Alan Peters II 48

Spatial Filtering

bandpass

filter

unsharp

masking

original

signed image with 0 at middle gray

25 August 2011 1999-2011 by Richard Alan Peters II 49

Motion Blur vertical regional

zoom rotational

original

25 August 2011 1999-2011 by Richard Alan Peters II 50

color noise blurred image color-only blur

Noise Reduction

25 August 2011 1999-2011 by Richard Alan Peters II 51

5x5 Wiener filter color noise blurred image

Noise Reduction

25 August 2011 1999-2011 by Richard Alan Peters II 52

Noise Reduction

original periodic

noise

frequency

tuned filter

25 August 2011 1999-2011 by Richard Alan Peters II 53

Shot Noise or Salt & Pepper Noise

+ shot noise - shot noise s&p noise

25 August 2011 1999-2011 by Richard Alan Peters II 54

Nonlinear Filters: the Median

s&p noise original median filter

25 August 2011 1999-2011 by Richard Alan Peters II 55

Nonlinear Filters: Min and Maxmin

+ shot noise min filter maxmin filter

25 August 2011 1999-2011 by Richard Alan Peters II 56

Nonlinear Filters: Max and Minmax

- shot noise max filter minmax

25 August 2011 1999-2011 by Richard Alan Peters II 57

Nonlinear Processing: Binary Morphology

“L” shaped SE

O marks origin

Foreground: white pixels

Background: black pixels

Cross-hatched

pixels are

indeterminate.

25 August 2011 1999-2011 by Richard Alan Peters II 58

Used after opening to grow back pieces of the original

image that are connected to the opening.

Permits the removal of small regions that are disjoint

from larger objects without distorting the small

features of the large objects.

original opened reconstructed

Nonlinear Processing: Binary Reconstruction

25 August 2011 1999-2011 by Richard Alan Peters II 59

“L” shaped SE

O marks origin

Foreground: white pixels

Background: black pixels

Cross-hatched

pixels are

indeterminate.

Nonlinear Processing: Grayscale Morphology

25 August 2011 1999-2011 by Richard Alan Peters II 60

Grayscale Morphology: Opening

opening: erosion then dilation opened & original

25 August 2011 1999-2011 by Richard Alan Peters II 61

Grayscale Morphology: Opening

erosion & opening erosion & opening & original

25 August 2011 1999-2011 by Richard Alan Peters II 62

reconstructed opening original

Nonlinear Processing: Grayscale Reconstruction

25 August 2011 1999-2011 by Richard Alan Peters II 63

Forensic Analysis of Photographs

Photographs by Robert Fenton of a battlefield in the Crimean war taken on 23 April 1855.

From Morris, Errol, “Which Came First, the Chicken or the Egg?”, Parts 1-3, New York

Times, Zoom Editorial Section, 25 Sept. 2007 (pt.1), 7 Oct. 2007 (pt.2), 30 Oct. 2007 (pt.3).

Which came first?

25 August 2011 1999-2011 by Richard Alan Peters II 64

Photographs by Robert Fenton of a battlefield in the Crimean war taken on 23 April 1855. From Morris, Errol, “Which Came First, the

Chicken or the Egg?”, Parts 1-3, New York Times, Zoom Editorial Section, 25 Sept. 2007 (pt.1), 7 Oct. 2007 (pt.2), 30 Oct. 2007 (pt.3).

Which came first?

Forensic Analysis of Photographs

25 August 2011 1999-2011 by Richard Alan Peters II 65

Photographs by Robert Fenton of a battlefield in the Crimean war taken on 23 April 1855. From Morris, Errol, “Which Came First, the

Chicken or the Egg?”, Parts 1-3, New York Times, Zoom Editorial Section, 25 Sept. 2007 (pt.1), 7 Oct. 2007 (pt.2), 30 Oct. 2007 (pt.3).

Forensic Analysis of Photographs

Which came first?

25 August 2011 1999-2011 by Richard Alan Peters II 66

High Dynamic Range

(HDR) Imaging

under exposed

Bartlo

miej O

ko

nek

http

://ww

w.easy

hdr.co

m/ex

amp

les.ph

p

25 August 2011 1999-2011 by Richard Alan Peters II 67

High Dynamic Range

(HDR) Imaging

default exposure

Bartlo

miej O

ko

nek

http

://ww

w.easy

hdr.co

m/ex

amp

les.ph

p

25 August 2011 1999-2011 by Richard Alan Peters II 68

High Dynamic Range

(HDR) Imaging

over exposed

Bartlo

miej O

ko

nek

http

://ww

w.easy

hdr.co

m/ex

amp

les.ph

p

25 August 2011 1999-2011 by Richard Alan Peters II 69

High Dynamic Range

(HDR) Imaging

combined

Bartlo

miej O

ko

nek

http

://ww

w.easy

hdr.co

m/ex

amp

les.ph

p

25 August 2011 1999-2011 by Richard Alan Peters II 70

Image Compression

Yoyogi Park, Tokyo, October 1999. Photo by Alan Peters.

Original image is

5244w x 4716h

@ 1200 ppi:

127MBytes

25 August 2011 1999-2011 by Richard Alan Peters II 71

Image Compression: JPEG JP

EG

qu

alit

y le

ve

l File

siz

e in

byte

s

25 August 2011 1999-2011 by Richard Alan Peters II 72

JP

EG

qu

alit

y le

ve

l File

siz

e in

byte

s

Image Compression: JPEG

25 August 2011 1999-2011 by Richard Alan Peters II 73

Image Compositing

Combine parts from separate images to form a new image.

It’s difficult to do well.

Requires relative positions, orientations, and scales to be correct.

Lighting of objects must be consistent within the separate images.

Brightness, contrast, color balance, and saturation must match.

Noise color, amplitude, and patterns must be seamless.

25 August 2011 1999-2011 by Richard Alan Peters II 74

Prof. Peters in his home office. Needs a better shirt.

Image Compositing Example

25 August 2011 1999-2011 by Richard Alan Peters II 75

This shirt demands a monogram.

Image Compositing Example

25 August 2011 1999-2011 by Richard Alan Peters II 76

He needs some more color.

Image Compositing Example

25 August 2011 1999-2011 by Richard Alan Peters II 77

Nice. Now for the way he’d wear his hair if he had any.

Image Compositing Example

25 August 2011 1999-2011 by Richard Alan Peters II 78

He can’t stay in the office like this.

Image Compositing Example

25 August 2011 1999-2011 by Richard Alan Peters II 79

Where’s a hepcat Daddy-O like this belong?

Image Compositing Example

25 August 2011 1999-2011 by Richard Alan Peters II 80

In the studio!

Collar this jive, Jackson. Like crazy, Man !

Image Compositing Example

25 August 2011 1999-2011 by Richard Alan Peters II 81

top related