lecture8 sampling f11 - nyu tandon school of...

53
Image Sampling and Image Sampling and Resizing Yao Wang Polytechnic Institute of NYU, Brooklyn, NY 11201 With contribution from Zhu Liu Partly based on A. K. Jain, Fundamentals of Digital Image Processing

Upload: dinhhanh

Post on 08-Mar-2018

228 views

Category:

Documents


3 download

TRANSCRIPT

Image Sampling andImage Sampling and Resizing

Yao WangPolytechnic Institute of NYU, Brooklyn, NY 11201

With contribution from Zhu LiuPartly based on A. K. Jain, Fundamentals of Digital Image Processing

Lecture Outline

• IntroductionN q ist sampling and interpolation• Nyquist sampling and interpolation theorem

• Common sampling and interpolation filters• Sampling rate conversion of discrete

images (image resizing)

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 2

Illustration of Image Sampling and InterpolationInterpolation

dx=dy=2mmy

16 mm

dx=dy=1mm

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 3

How to choose dx, dy to reach a good trade off between accuracy and cost of storage, transmission, processing?

Uniform Sampling• f(x,y) represents the original continuous

image, fs(m,n) the sampled image, andimage, fs(m,n) the sampled image, and the reconstructed image.

• Uniform sampling),(ˆ yxf

Uniform sampling),,(),( ynxmfnmfs

– ∆x and ∆y are vertical and horizontal

.1,...,0;1,...,0 NnMmy

sampling intervals. fs,x=1/∆x, fs,y=1/ ∆y are vertical and horizontal sampling frequencies.

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 4

Image Sampling as Product with Impulse TrainProduct with Impulse Train

• Periodic impulse sequence

1 1M N

1

0

1

0),(),(

M

m

N

nyx nymxyxp

1 1

1

0

1

0

~

)()(

),(),(),(),(),(

M N

M

m

N

nyxyxs

f

nymxnmfyxpyxfyxf

0 0

),(),(m n

yxs nymxnmf

1m = 0

∆x1

…p(x,y)

M - 1

x y∆y

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 5

n= 0 1 N-12x y

Fourier Transform of Impulse Train• 1D

nfut

uPtnttp s

)(1)()()(

tfwhere

t

s

nnm

1

,

• 2D

nfvmfuyx

vuPynyxmxyxpnm

ysxsnm

),(1),(),(),(,

,,,

yf

xfwhere ysxs

1,1

,,

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 6

Frequency Domain Interpretation of SamplingSampling

• Sampling is equivalent to multiplication of the original signal with a sampling pulsethe original signal with a sampling pulse sequence.

s yxpyxfyxf ),(),(),(

I f d i

nm

ynyxmxyxpwhere,

),(),(

• In frequency domains vuPvuFvuF

11),(),(),(

nm

ysxss

ysxs

nmysxs nfvmfuF

yxvuF

ffwhere

nfvmfuyx

vuP,

,,,

,, ),(1),(

1,1

),(1),(

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 7

ysxs yf

xf ,, ,

Frequency Domain Interpretation of Sampling in 1D

Original signal

p p g

Sampling impulse train

The spectrum of the sampled signal includes the original

S l d i l

includes the original spectrum and its aliases (copies) shifted to k fs , k=+/- 1,2,3,…

Sampled signalfs > 2fm

s

When fs< 2fm , aliases overlap with the original spectrum >Sampled signal

fs < 2fm(Aliasing effect)

original spectrum -> aliasing artifact

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 8

Sampling of 1D Sinusoid Signals

Sampling aboveNyquist rateyqs=3m>s0

Reconstructedi i l=original

Sampling underNyquist rates=1.5m<s0

ReconstructedReconstructed!= original

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 9

Aliasing: The reconstructed sinusoid has a lower frequency than the original!

Frequency Domain Interpretation of Sampling in 2DSampling in 2D

• The sampled signal contains replicas of the original spectrum shifted by multiplesthe original spectrum shifted by multiples of sampling frequencies.

u uu u

fm x

fs,x

fs,x>2fm,x

vv

fm,x

fm,y fs,y

fs,y>2fm,y

Original spectrum F(u v) Sampled spectrum F (u v)

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 10

Original spectrum F(u,v) Sampled spectrum Fs(u,v)

Illustration of Aliasing Phenomenon

u u

fm,x fs,xfs x<2fm x

v vfm,yfs,y

s,x fm,xfs,y<2fm,y

Original spectrum F(u,v) Sampled spectrum Fs(u,v)

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 11

p p s( )

Nyquist Sampling Rate

• Nyquist sampling rateTo be able to preserve the original signal in– To be able to preserve the original signal in the sampled signals, these aliasing components should not overlap with thecomponents should not overlap with the original one. This requires that the sampling frequency fs,x, fs,y must be at least twice of the , ,yhighest frequency of the signal, known as Nyquist sampling rate.

• Interpolation– Remove all the aliasing components

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 12

Nyquist Sampling and Reconstruction TheoremTheorem

• A band-limited image with highest frequencies at fm,x, fm,ycan be reconstructed perfectly from its samples, provided that the sampling frequencies satisfy: fs,x >2fm,x, fs,y>2fm,y

• The reconstruction can be accomplished by the ideal• The reconstruction can be accomplished by the ideal low-pass filter with cutoff frequency at fc,x = fs,x/2, fc,y = fs,y/2, with magnitude ∆x∆y.

yfyf

xfxf

yxhotherwise

fv

fuyxvuH

ys

ys

xs

xsysxs

,

,

,

,,, sinsin

),(0

2||,

2||),(

• The interpolated image

)(sin)(sin),(),(ˆ ,, ymyfxmxf

nmfyxf ysxss

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 13

)()(),(),(

,, ymyfxmxfnmfyxf

ysxsm ns

Applying Nyquist Theorem• Two issues

– The signals are not bandlimited.The signals are not bandlimited.• A bandlimiting filter with cutoff frequency fc=fs/2 needs to be

applied before sampling. This is called prefilter or sampling filterfilter.

– The sinc filter is not realizable.• Shorter, finite length filters are usually used in practice for

b th filt d i t l ti filtboth prefilter and interpolation filter.

• A general paradigm

Prefilter Interpolation(postfilter)A D

B C

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 14

Sampling pulse fs

Non-ideal Sampling and InterpolationInterpolation

AliasedComponent

AliasedComponent

Pre-filteredSignal

-fs fs0

Non-idealInterpolationfilter

Aliasing Imaging

filter

Non ideal prefiltering causes AliasingNon ideal interpolation filter causes Imaging

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 15

p g g

Sampling a Sinusoidal Signal )1,2()1,2(

21),()24cos(),( vuvuvuFyxyxf

S l d t ∆ ∆ 1/3 f f 3Sampled at ∆x=∆y=1/3 fs,x=fs,y=3

v vOriginal Spectrum Sampled Spectrum

3(-2,1)

3(-2,1)

u

3

-3 3

(2,-1)

u

3

-3 3(2,-1) Ideal-3 (2, 1) -3

Original pulse Replicated pulse

interpolation filter

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 16

Replication center)22cos(),(ˆ yxyxf

Sampling in 2D: Sampling a 2D Sinusoidal PatternSampling a 2D Sinusoidal Pattern

f(x,y)=sin(2*π*(3x+y))Sampling: dx=0 01 dy=0 01 f(x,y)=sin(2*π*(3x+y))Sampling: dx=0.01,dy=0.01Satisfying Nyquist ratefx,max=3, fy,max=1fs,x=100>6, fs,y=100>2

Sampling: dx=0.2,dy=0.2(Displayed with pixel replication)Sampling at a rate lower than Nyquist rate

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 17

A Simple Prefilter – Averaging Filter• Each sampling value is the mean value of the

original continuous function in a rectangularoriginal continuous function in a rectangular region of dimension ∆x and ∆y, i.e:

dxdyyxfnmf )(1)(

n

nmDyx

s dxdyyxfyx

nmf,),(

),(),(

where

m

The equivalent prefilter is

ynyynxmxxmD nm )2/1()2/1(,)2/1()2/1(,

yvyv

xuxuyxvuH

otherwise

yyxxyxyxh

sinsin),(

0

2/||,2/||1),(

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 18

otherwise 0

How good is the averaging Filter?

• Look at its frequency response, how far is it from the ideal low pass filterit from the ideal low pass filter

Ideal low pass filter with cutoff at fs/2p fs

Averaging filter

f 2f 3f-f2f3f

Averaging filter

fs 2fs 3fsfs-2fs-3fs

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 19

Reconstruction of continuous image from Samplesfrom Samples

• The interpolation problem• Interpretation as weighted average of sample• Interpretation as weighted average of sample

values• Interpretation as filteringInterpretation as filtering• What filter should we use?

– Ideal interpolation filter: remove the replicatedIdeal interpolation filter: remove the replicated spectrum using ideal low-pass filter with cutoff frequency at fs/2 (the sinc function)

)()(sin

)()(sin

),(),(ˆ,

,

,

,

ynyfynyf

xmxfxmxf

nmfyxfys

ys

xs

xs

m ns

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 20

Interpretation as a weighted average of sample valuesof sample values

• The value of a function at arbitrary point (x, y) is estimated from a weighted sum of its sample values in the neighborhood of ([x/∆x],weighted sum of its sample values in the neighborhood of ([x/∆x], [y/∆y]):– Let h(x,y; m,n) specifies the weight assigned to sample m,n, when

determining the image value at x,y

nm

s nmfnmyxhyxf,

),(),;,(),(ˆ

H(x;8)

H(x;9)H(x;7)

x

H(x;10)

H(x;9)

H(x;6)

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 21

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 n

H(x;5)

Desirable Properties of the Weight FunctionFunction

• The weighting function h(x,y;m,n) should depend only on distance between (x,y) and the spatial location of (m,n), i ei.e.

Should be a decreasing function of the distance

).,(),;,( ynyxmxhnmyxh

• Should be a decreasing function of the distance– Higher weight for nearby samples

• Should be an even function of the distanceL ft i hb d i ht i hb f di t h th– Left neighbor and right neighbor of same distance have the same weight

– h1(x)=h1(-x)• Generally Separable:• Generally Separable:

– h2(x,y)=h1(x) h1(y) • To retain the original sample values, should have

h(0 0)=1 h(mx ny)=0

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 22

– h(0,0)=1, h(mx,ny)=0

Interpretation as Filtering• The weighted

average operation is ),(),(),(ˆ

~

,nmfynyxmxhyxf s

nm

equivalent to filtering fs(x,y) with h(x,y)

• Usually h(x y) is ),(),(

),(),(),(

~

,

yxfyxh

ynyxmxnmfyxf

s

nmss

• Usually, h(x,y) is separable h(x,y) = hx(x)hy(y) ),(),(),(

),(),(~

ddynxmnmfyxh

ddfyxh

s

s

• To retain the original sample values, should have )()(

),(),(),(,

,

hf

ddynxmyxhnmfnm

s

nm

should have– h(0,0)=1,

h(mx,ny)=0Nyquist filter

),(),(),(ˆ

),(),(

~

,

yxfyxhyxf

ynyxmxhnmf

s

nms

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 23

– Nyquist filter

A Simple Interpolation Filter: Sample-And-Hold (pixel replication)Sample And Hold (pixel replication)

• The interpolated value at a point is obtained from that of its nearest sampleobtained from that of its nearest sample

ynyynxmxxmnmfyxf s )2/1()2/1(,)2/1()2/1(),(),(ˆ

• Corresponding interpolation filter is

yyyxxx

yxh2/2/,2/2/1

),( n otherwise

y0

),(

m(m-1/2)∆x

m(m+1/2)∆x

(n 1/2)∆y

0th order interpolation filter

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 24

(n-1/2)∆y(n+1/2)∆y

Bilinear Filter

f0fa

f1(m,n) (m,n+1)

b

0 1aa

b

q1 q 2)1,1()1,()1()(

),1(),()1()(:1

2

1

Snmafnmfaqf

nmafnmfaqfStep

10)1( affafa

(m+1,n) (m+1,n+1)

pq1 q2

)()()1()(:2

21 qbfqfbpfStep

1D Linear interpolation

2D bilinear interpolation

Corresponding interpolation filter

yyyxxx

yy

xx

yxh ,||1||1),(

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 25

otherwise0

Which filter is better?

• Recall what happened in the frequency domain when we sample an imagedomain when we sample an image

• Ideal filter: ½ band ideal low pass filter• Quantitatively we can evaluate how far is

the filter from the ideal filter• But we should also look at visual artifacts

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 26

Frequency Domain Interpretation of Sampling in 2DSampling in 2D

• The sampled signal contains replicas of the original spectrum shifted by multiplesthe original spectrum shifted by multiples of sampling frequencies.

u uu u

fm x

fs,x

fs,x>2fm,x

vv

fm,x

fm,y fs,y

fs,y>2fm,y

Original spectrum F(u v) Sampled spectrum F (u v)

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 27

Original spectrum F(u,v) Sampled spectrum Fs(u,v)

Ideal Interpolation Filter• The ideal interpolation filter should be a low-pass filter

with cutoff frequency at fc,x = fs,x/2, fc,y = fs,y/2, with y ymagnitude ∆x∆y

yfxfyxh

fv

fuyxvuH ysxs

ysxs,,

,, sinsin)(2

||,2

||)(

yfxf

yxhotherwise

yvuHysxs ,,

),(0

2||

2||),(

The sinc filter

• The interpolated image

The sinc filter

Weight function h(x,y;m,n)

)()(sin

)()(sin

),(),(ˆ,

,

,

,

ynyfynyf

xmxfxmxf

nmfyxfys

ys

xs

xs

m ns

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 28

Comparison of Different Interpolation FiltersFilters

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 29

Image Resizing• Image resizing:

– Enlarge or reduce the image size (number of pixels)– Equivalent to

• First reconstruct the continuous image from samples• Then Resample the image at a different sampling ratep g p g

– Can be done w/o reconstructing the continuous image explicitly

• Image down-sampling (resample at a lower rate)Image down-sampling (resample at a lower rate)– Spatial domain view– Frequency domain view: need for prefilter

• Image up-sampling (resample at a higher rate)– Spatial domain view– Different interpolation filters

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 30

Different interpolation filters• Nearest neighbor, Bilinear, Bicubic

Image Down-Sampling

• Example: reduce a 512x512 image to 256x256 = factor– reduce a 512x512 image to 256x256 = factor of 2 downsampling in both horizontal and vertical directionsvertical directions

– In general, we can down-sample by an arbitrary factor in the horizontal and vertical ydirections

• How should we obtain the smaller image ?How should we obtain the smaller image ?

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 31

Down Sampling by a Factor of Two

8x8 Image 4x4 Image

• Without Pre-filtering (simple approach)Without Pre filtering (simple approach)

Averaging Filter

)2,2(),( nmfnmfd

• Averaging Filter4/)]12,12()2,12()12,2()2,2([),( nmfnmfnmfnmfnmfd

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 32

Problem of Simple Approach• Aliasing if the effective sampling rate is below

the Nyquist sample rate = 2 * highest frequencythe Nyquist sample rate 2 highest frequency in the original continuous signal

• We need to prefilter the signal before down-p gsampling

• Ideally the prefilter should be a low-pass filter with a cut-off frequency half of the new sampling rate.– In digital frequency of the original sampled image, the

cutoff frequency is ¼.• In practice we may use simple averaging filter

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 33

• In practice, we may use simple averaging filter

Down Sampling by a Factor of K

)(f̂Hs ↓K

f(m,n) fd(m,n)),( nmf

Pre-filtering Down-sampling

),(ˆ),( KnKmfnmfd

For factor of K down sampling, the prefilter should be low pass filter with cutoff at fs/(2K), if fs is the original sampling frequency

I t f di it l f th t ff h ld b 1/(2K)

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 34

In terms of digital frequency, the cutoff should be 1/(2K)

Example: Image Down-Sample

Without prefiltering

With prefiltering (no aliasing, but blurring!)

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 35

Down-Sampling Using Matlab• Without prefiltering

– If f(,) is an MxN image, down-sampling by a factor of K can be done simply by

>> g=f(1:K:M,1:K:N)

• With prefilteringp g– First convolve the image with a desired filter

• Low pass filter with digital cutoff frequency 1/(2K)– In matlab, 1/2 is normalized to 1In matlab, 1/2 is normalized to 1

– Then subsample>> h=fir1(N, 1/K)

%design a lowpass filter with cutoff at 1/K%design a lowpass filter with cutoff at 1/K.>> fp=conv2(f,h)>> g=fp(1:K:M,1:K:N)

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 36

Image Up-Sampling• Produce a larger image from a smaller one

– Eg. 512x512 -> 1024x1024M ll l b bit f t L– More generally we may up-sample by an arbitrary factor L

• Questions: – How should we generate a larger image?– Does the enlarged image carry more information?

• Connection with interpolation of a continuous image from discrete image– First interpolate to continuous image, then sampling at a higher sampling

rate, L*fs– Can be realized with the same interpolation filter, but only evaluate at

x=mx’ y=ny’ x’=x/L y’=y/Lx=mx , y=ny , x =x/L, y =y/L– Ideally using the sinc filter!

)(sin)(sin)()(ˆ ,, ymyfxmxf

ff ysxs

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 37

)()(),(),(

,

,

,

,

ymyfxmxfnmfyxf

ys

ys

xs

xs

m ns

Example: Factor of 2 Up-Sampling

(m,n+1)(m,n) (2m,2n) (2m,2n+1)

(2m+1 2n) (2m+1 2n+1)

(m+1,n+1)(m+1,n)

(2m+1,2n) (2m+1,2n+1)

Green samples are retained in the interpolated image;Orange samples are estimated from surrounding green samples.

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 38

Nearest Neighbor Interpolation(pixel replication)(pixel replication)

(m n+1)(m n)

a

b

(m,n+1)

(m’/M,n’/M)

(m,n)

a

(m+1,n+1)(m+1,n)

O[m’,n’] (the resized image) takes the value of the sample nearest to (m’/M,n’/M) in I[m,n] (the original image):

/Mn'=n/Mm'=m0 5)]+(n(int)0 5)+(mI[(int)=]n'O[m'

Also known as pixel replication: each original pixel is replaced by MxM pixels of

/M.nn/M,mm,0.5)]+(n(int)0.5),+(mI[(int) ]n,O[m

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 39

Also known as pixel replication: each original pixel is replaced by MxM pixels of the sample valueEquivalent to using the sample-and-hold interpolation filter.

Special Case: M=2

(m,n+1)(m,n) (2m,2n) (2m,2n+1)

(2m+1 2n) (2m+1 2n+1)

(m+1,n+1)(m+1,n)

(2m+1,2n) (2m+1,2n+1)

Nearest Neighbor:O[2m,2n]=I[m,n]O[2m,2n+1]= I[m,n]O[2m+1,2n]= I[m,n]

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 40

O[2m+1,2n+1]= I[m,n]

Bilinear Interpolation

(m,n) (m,n+1)

a

b

(m’/M,n’/M)

• O(m’,n’) takes a weighted average of 4 samples nearest to (m’/M,n’/M) in I(m,n).

(m+1,n) (m+1,n+1)

O(m ,n ) takes a weighted average of 4 samples nearest to (m /M,n /M) in I(m,n).

• Direct interpolation: each new sample takes 4 multiplications:O[m’,n’]=(1-a)*(1-b)*I[m,n]+a*(1-b)*I[m,n+1]+(1-a)*b*I[m+1,n]+a*b*I[m+1,n+1]

• Separable interpolation:i) interpolate along each row y: F[m,n’]=(1-a)*I[m,n]+a*I[m,n+1]ii) interpolate along each column x’: O[m’,n’]=(1-b)*F[m’,n]+b*F[m’+1,n]

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 41

Special Case: M=2

(m,n+1)(m,n) (2m,2n) (2m,2n+1)

(2m+1 2n) (2m+1 2n+1)

(m+1,n+1)(m+1,n)

(2m+1,2n) (2m+1,2n+1)

Bilinear Interpolation:O[2m,2n]=I[m,n]O[2m,2n+1]=(I[m,n]+I[m,n+1])/2O[2m+1,2n]=(I[m,n]+I[m+1,n])/2

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 42

O[2m+1,2n+1]=(I[m,n]+I[m,n+1]+I[m+1,n]+I[m+1,n+1])/4

Bicubic Interpolation

(m-1,n+1)(m-1,n)

b

(m,n) (m,n+1)(m,n-1) (m,n+2)

a

(m+1,n+1)(m+1,n)

(m’/M,n’/M)

(m+1,n-1) (m+1,n+2)

(m+2,n) (m+2,n+1)

O( ’ ’) i i t l t d f 16 l t t ( ’/M ’/M) i I( )• O(m’,n’) is interpolated from 16 samples nearest to (m’/M,n’/M) in I(m,n).• Direct interpolation: each new sample takes 16 multiplications• Separable interpolation:

i) interpolate along each row y: I[m,n]->F[m,n’] (from 4 samples)ii) i t l t l h l ’ F[ ’] O[ ’ ’] (f 4 l )

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 43

ii) interpolate along each column x’: F[m,n’]-> O[m’,n’] (from 4 samples)

Interpolation Formula

(m n) (m n+1)(m n 1) (m n+2)

(m-1,n+1)(m-1,n)(m-1,n-1) (m-1,n+2)

b

(m,n) (m,n+1)(m,n-1) (m,n+2)

F(m’,n-1) F(m’ n) F(m’ n+1) F(m’ n+2)a

(m+1,n+1)(m+1,n)

(m’/M,n’/M)

(m+1,n-1) (m+1,n+2)

(m+2,n) (m+2,n+1)

( , ) F(m ,n) F(m ,n+1) F(m ,n+2)

( , ) ( , )

(m+2,n-1) (m+2,n+2)

IbbIbbbIbbIbbF ]2[)1(]1[)1(][)21(]1[)1(]'[ 22322

mMmb

Mmmwhere

nmIbbnmIbbbnmIbbnmIbbnmF

','(int)

],,2[)1(],1[)1(],[)21(],1[)1(],'[ 22322

nnnmFaanmFaaanmFaanmFaanmO

''],2,'[)1(]1,'[)1(],'[)21(]1,'[)1(]','[ 22322

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 44

nMna

Mnnwhere ,(int)

Special Case: M=2

(2m,2n) (2m,2n+1)(m-1,n+1)(m-1,n)

(2m+1 2n) (2m+1 2n+1)

(m,n) (m,n+1)(m,n-1) (m,n+2)

(2m,2n+1) (2m+1,2n) (2m+1,2n+1)

(m+1,n+1)(m+1,n)(m+1,n-1) (m+1,n+2)

Bicubic interpolation in Horizontal direction

(m+2,n) (m+2,n+1)

Bicubic interpolation in Horizontal direction

F[2m,2n]=I[m,n]F[2m,2n+1]= -(1/8)I[m,n-1]+(5/8)I[m,n]+(5/8)I[m,n+1]-(1/8)I(m,n+2)

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 45

Same operation then repeats in vertical direction

Comparison of Interpolation MethodsInterpolation Methods

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 46

Resize_peak.m

Up-Sampled from w/o Prefiltering

NearestneighborOriginal

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 47

Bilinear Bicubic

Up-Sampled from with Prefiltering

NearestneighborOriginal

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 48

Bilinear Bicubic

Matlab for Image Resizing[img]=imread('fruit.jpg','jpg');

%downsampling without prefiltering

i 1 i i (i 0 5 ' ')img1=imresize(img,0.5,'nearest');

%upsampling with different filters:

img2rep=imresize(img1,2,'nearest');

i 2li i i (i 1 2 'bili ')img2lin=imresize(img1,2,'bilinear');

img2cubic=imresize(img1,2,'bicubic');

%down sampling with filtering%down sampling with filtering

img1=imresize(img,0.5,'bilinear',11);

%upsampling with different filters

img2rep=imresize(img1 2 'nearest');img2rep=imresize(img1,2, nearest );

img2lin=imresize(img1,2,'bilinear');

img2cubic=imresize(img1,2,'bicubic');

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 49

Filtering View: Up Sampling by a Factor of Kof K

)(~fHi↑K

f(m,n) fu(m,n)),( nmf

Zero-padding Post-filtering

Kfl lfKKf )//(

otherwise

KofmultiplearenmifKnKmfnmf

0,)/,/(

),(~

lk

u lnkmflkhnmf,

),(~),(),(

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 50

Ideally H should be a low pass filter with cutoff at 1/2K in digital frequency, or fs/2K in continuous frequency

Homework (1)1. Consider a 1D signal f(t) = sin(4πt). Illustrate the original and the sampled

signal f(n) obtained with a sampling interval ∆t = 1/3. Draw on the same figure the interpolated signal from the sampled one using the sample-and-hold and the linear interpolation filter respectively Explain theand-hold and the linear interpolation filter, respectively. Explain the observed phenomenon based on both the Nyquist sampling theorem as well as physical interpretation. What is the largest sampling interval that can be used to avoid aliasing?

2. Consider a function f(x, y) = cos2π(4x + 2y) sampled with a sampling2. Consider a function f(x, y) cos2π(4x 2y) sampled with a sampling period of ∆x = ∆y = ∆ = 1/6 or sampling frequency fs = 1/∆ = 6.

a) Assume that it is reconstructed with an ideal low-pass filter with cut-off frequency fcx = fcy = 1/2fs. Illustrate the spectra of the original, sampled, and reconstructed signals. Give the spatial domain function representation of the reconstructed signal Is the result as expected?reconstructed signal. Is the result as expected?

b) If the reconstruction filter has the following impulse response:

otherwise

yxyxh

02/,2/1

),(

Illustrate the spectra of the reconstructed signal in the range -fs ≤ u,v ≤ fs. Give a spatial domain function representation of the reconstructed signal if the reconstruction filter is band-limited to (-fs ≤ u,v ≤ fs). (i.e., this filter remains the

otherwise0

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 51

( s , s) ( ,same for the frequency range -fs ≤ u,v ≤ fs, and is set to 0 outsize this range.)

Homework(2)3. (Computer Assignment) Write your own program or programs

which can: a) Down sample an image by a factor of 2, with and without using the averaging filter; b) Up-sample the previously d l d i b f t f 2 i th i l li tidown-sampled images by a factor of 2, using the pixel replication and bilinear interpolation methods, respectively. You should have a total of 4 interpolated images, with different combination of down-sampling and interpolation methods. Your program could ith di tl di l th d i d ieither directly display on screen the processed images during

program execution, or save the processed images as computer files for display after program execution. Run your program with the image Barbara. Comment on the quality of the down/up

l d i bt i d ith diff t th dsampled images obtained with different methods.Note: you should not use the ”imresize” function in Matlab to do this assignment. But you are encouraged to compare results of your program with ”resize”.y p g

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 52

Reading

• R Gonzalez “Digital Image Processing ”• R. Gonzalez, Digital Image Processing, Section 2.4

• A K Jain “Fundamentals of Digital ImageA.K. Jain, Fundamentals of Digital Image Processing,” Section 4.1-4.4

Yao Wang, NYU-Poly EL5123: Sampling and Resizing 53