![Page 1: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/1.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell
Sampling and Reconstruction
![Page 2: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/2.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 2
Reading
Required:Watt, Section 14.1
Recommended:Ron Bracewell, The Fourier Transform and ItsApplications, McGraw-Hill.Don P. Mitchell and Arun N. Netravali,“Reconstruction Filters in Computer ComputerGraphics ,” Computer Graphics, (Proceedingsof SIGGRAPH 88). 22 (4), pp. 221-228, 1988.
![Page 3: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/3.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3
What is an image?
We can think of an image as a function, f, from R2 to R:f( x, y ) gives the intensity of a channel at position ( x, y )Realistically, we expect the image only to be defined over arectangle, with a finite range:
f: [a,b]x[c,d] [0,1]
A color image is just three functions pasted together. Wecan write this as a “vector-valued” function:
We’ll focus in grayscale (scalar-valued) images for now.
( , )
( , ) ( , )
( , )
r x y
f x y g x y
b x y
! "# $
= # $# $% &
![Page 4: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/4.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 4
Images as functions
![Page 5: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/5.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 5
Digital images
In computer graphics, we usually create or operate ondigital (discrete) images:
Sample the space on a regular gridQuantize each sample (round to nearest integer)
If our samples are Δ apart, we can write this as:f[i ,j] = Quantize{ f(i Δ, j Δ) }
![Page 6: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/6.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 6
Motivation: filtering and resizing
What if we now want to:smooth an image?sharpen an image?enlarge an image?shrink an image?
Before we try these operations, it’s helpfulto think about images in a moremathematical way…
![Page 7: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/7.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 7
Fourier transforms
We can represent a function as a linear combination(weighted sum) of sines and cosines.We can think of a function in two complementary ways:
Spatially in the spatial domainSpectrally in the frequency domain
The Fourier transform and its inverse convert betweenthese two domains:
Frequencydomain
Spatialdomain
!
F(s) = f (x)e" i 2# sx
"$
$
% dx
!
f (x) = F(s)ei 2" sx
#$
$
% ds
![Page 8: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/8.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 8
Fourier transforms (cont’d)
Where do the sines and cosines come in?
f(x) is usually a real signal, but F(s) is generallycomplex:
If f(x) is symmetric, i.e., f(x) = f(-x)), then F(s) =Re(s). Why?
Frequencydomain
Spatialdomain
!
F(s) = f (x)e" i 2# sx
"$
$
% dx
!
f (x) = F(s)ei 2" sx
#$
$
% ds
!
F(s) = Re(s) " iIm(s) = F(s)e"i 2# s
![Page 9: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/9.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 9
1D Fourier examples
![Page 10: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/10.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 10
2D Fourier transform
Frequencydomain
Spatialdomain
Spatial domain Frequency domain
!
F(sx,sy ) = f (x,y)e" i 2# sxx
"$
$
%"$
$
% e"i 2# syydxdy
!
f (x,y) = F(sx,sy )ei 2" sxx
#$
$
%#$
$
% ei 2" syydsxdsy
!
f (x,y)
!
F(sx,sy )
![Page 11: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/11.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 11
2D Fourier examples
Spatial domain
Frequency domain
!
f (x,y)
!
F(sx,sy )
![Page 12: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/12.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 12
Convolution
One of the most common methods for filtering afunction is called convolution.In 1D, convolution is defined as:
where
!
) h (x) = h("x)
!
g(x) = f (x) * h(x)
= f ( " x )h(x # " x )d " x #$
$
%
= f ( " x )) h ( " x # x)d " x
#$
$
%
![Page 13: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/13.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 13
Convolution properties
Convolution exhibits a number of basic, butimportant properties.Commutativity:
Associativity:
Linearity:
!
a(x)"b(x) = b(x)" a(x)
!
[a(x)"b(x)]"c(x) = a(x)"[b(x)"c(x)]
!
a(x)"[k # b(x)] = k # [a(x)"b(x)]
a(x)" (b(x) + c(x)) = a(x)"b(x) + a(x)"c(x)
![Page 14: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/14.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 14
Convolution in 2D
In two dimensions, convolution becomes:
where
* =
f(x,y) h(x,y) g(x,y)
!
) h (x,y) = h("x,"y)
!
g(x,y) = f (x,y)" h(x,y)
= f ( # x , # y )h(x $ # x )(y $ # y )d # x d # y $%
%
&$%
%
&
= f ( # x , # y )) h ( # x $ x)( # y $ y)d # x d # y
$%
%
&$%
%
&
![Page 15: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/15.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 15
Convolution theorems
Convolution theorem: Convolution in thespatial domain is equivalent tomultiplication in the frequency domain.
Symmetric theorem: Convolution in thefrequency domain is equivalent tomultiplication in the spatial domain.!
f " h# F $H
!
f " h# F $H
![Page 16: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/16.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 16
Convolution theorems
Theorem
Proof (1))(*)()(
)()()*(
)(*)()(
)()()*(
GFFG
GFGF
gffg
gfgf
1-1-1-
1-1-1-
FFF
FFF
FFF
FFF
=
=
=
=
!
F( f * g) = f ( " t )g(t # " t )d " t e#i$ t
dt#%
%
&#%
%
&
= f ( " t )g(t # " t )e# i$ " t
e# i$ ( t# " t )
dtd " t #%
%
&#%
%
&
= f ( " t )g( " " t )e# i$ " t
e# i$ " " t
d " " t d " t #%
%
&#%
%
&
= f ( " t )e# i$ " t
d " t g( " " t )e#i$ " " t
d " " t #%
%
&#%
%
&
= F( f )F(g)
![Page 17: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/17.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 17
1D convolution theorem example
![Page 18: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/18.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 18
2D convolution theorem example
*
f(x,y)
h(x,y)
g(x,y)
|F(sx,sy)|
|H(sx,sy)|
|G(sx,sy)|
![Page 19: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/19.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 19
The delta function
The Dirac delta function, δ(x), is a handytool for sampling theory.It has zero width, infinite height, and unitarea. It is usually drawn as:
![Page 20: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/20.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 20
Sifting and shifting
For sampling, the delta function has two importantproperties.Sifting:
Shifting:
!
f (x)"(x # a) = f (a)"(x # a)
!
f (x)"#(x $ a) = f (x $ a)
![Page 21: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/21.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 21
The shah/comb functionA string of delta functions is the key to sampling.The resulting function is called the shah or combfunction:
which looks like:
Amazingly, the Fourier transform of the shahfunction takes the same form:
where so = 1/T.
!
III(x) = "(x # nT)n= 0
$
%
!
III(s) = "(s# ns0)
n= 0
$
%
![Page 22: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/22.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 22
Now, we can talk about sampling.
The Fourier spectrum gets replicated by spatial sampling!How do we recover the signal?
Sampling
![Page 23: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/23.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 23
Sampling and reconstruction
![Page 24: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/24.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 24
Sampling and reconstruction in 2D
![Page 25: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/25.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 25
Sampling theorem
This result is known as the Sampling Theoremand is generally attributed to Claude Shannon(who discovered it in 1949) but was discoveredearlier, independently by at least 4 others:A signal can be reconstructed from its samples without
loss of information, if the original signal has no energy infrequencies at or above ½ the sampling frequency.For a given bandlimited function, the minimumrate at which it must be sampled is the Nyquistfrequency.
![Page 26: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/26.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 26
Reconstruction filters
The sinc filter, while “ideal”, has two drawbacks: It has large support (slow to compute) It introduces ringing in practice
We can choose from many other filters…
![Page 27: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/27.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 27
Cubic filtersMitchell and Netravali (1988) experimented with cubic filters,reducing them all to the following form:
The choice of B or C trades off between being too blurry or having toomuch ringing. B=C=1/3 was their “visually best” choice.The resulting reconstruction filter is often called the “Mitchell filter.”
!
r(x) =1
6
(12 " 9B " 6C) x3
+ ("18 +12B + 6C) x2
+ (6 " 2B)
(("B " 6C) x3
+ (6B + 30C) x2
+ ("12B " 48C) x + (8B + 24C)
0
#
$ %
& %
x <1
1' x < 2
otherwise
![Page 28: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/28.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 28
Reconstruction filters in 2D
We can also perform reconstruction in 2D…
![Page 29: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/29.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 29
Aliasing
Sampling rate is too low
![Page 30: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/30.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 30
Aliasing
What if we go below the Nyquist frequency?
![Page 31: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/31.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 31
Anti-aliasing
Anti-aliasing is the process of removing the frequenciesbefore they alias.
![Page 32: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/32.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 32
We can fill the “magic” box with analytic pre-filtering of the signal:
Why may this not generally be possible?
Anti-aliasing by analytic prefiltering
![Page 33: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/33.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 33
Filtered downsamplingAlternatively, we can sample the image at a higher rate, and then filter thatsignal:
We can now sample the signal at a lower rate. The whole process is calledfiltered downsampling or supersampling and averaging down.
![Page 34: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/34.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 34
Practical upsamplingWhen resampling a function (e.g., when resizing an image), you do notneed to reconstruct the complete continuous function.For zooming in on a function, you need only use a reconstruction filterand evaluate as needed for each new sample.Here’s an example using a cubic filter:
![Page 35: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/35.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 35
Practical upsampling
This can also be viewed as:1.putting the reconstruction filter at the desired location2.evaluating at the original sample positions3.taking products with the sample values themselves4. summing it up
Important: filter should always be normalized!
![Page 36: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/36.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 36
Practical downsamplingDownsampling is similar, but filter has larger support and smalleramplitude.Operationally:
1. Choose filter in downsampled space.2. Compute the downsampling rate, d, ratio of new sampling rate to old
sampling rate3. Stretch the filter by 1/d and scale it down by d4. Follow upsampling procedure (previous slides) to compute new values
![Page 37: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/37.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 37
2D resamplingWe’ve been looking at separable filters:
How might you use this fact for efficient resampling in 2D?!
r2D (x,y) = r
1D (x)r1D (y)
![Page 38: Sampling and Reconstructionfussell/courses/cs384g...University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 3 What is an image? We can think of an image as](https://reader034.vdocument.in/reader034/viewer/2022052613/604be2f8128b780490705ff9/html5/thumbnails/38.jpg)
University of Texas at Austin CS384G - Computer Graphics Spring 2010 Don Fussell 38
Next class: Image Processing
Reading:Jain, Kasturi, Schunck, Machine Vision.McGraw-Hill, 1995.Sections 4.2-4.4, 4.5(intro), 4.5.5, 4.5.6, 5.1-5.4. (from course reader)
Topics:- Implementing discrete convolution- Blurring and noise reduction- Sharpening- Edge detection