![Page 1: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/1.jpg)
Topic 4:Topic 4:
Local analysis of image Local analysis of image
patchespatchespatchespatches
•• What do we mean by an image “patch”?What do we mean by an image “patch”?
•• Applications of local image analysisApplications of local image analysis
•• Visualizing 1D and 2D intensity functionsVisualizing 1D and 2D intensity functions
![Page 2: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/2.jpg)
Local Image PatchesLocal Image Patches
So far, we have considered pixels completely independently
of each other (as RGB values or, as vectors [R, G, B])
In reality, photos have a great deal of structure
This structure can be analyzed at a local level (eg., small
groups of nearby pixels) or a global one (eg. entire image)
pixelpixel
![Page 3: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/3.jpg)
Local Image PatchesLocal Image Patches
Qualitatively, we can think of many different types of patches in an
image
Patches corresponding to a “corner” in the image
![Page 4: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/4.jpg)
Local Image PatchesLocal Image Patches
Qualitatively, we can think of many different types of patches in an
image
Patches corresponding to an “edge” in the image
![Page 5: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/5.jpg)
Local Image PatchesLocal Image Patches
Qualitatively, we can think of many different types of patches in an
image
Patches of uniform texture
![Page 6: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/6.jpg)
Local Image PatchesLocal Image Patches
Qualitatively, we can think of many different types of patches in an
image
Patches that originate from a single surface
![Page 7: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/7.jpg)
Local Image PatchesLocal Image Patches
Qualitatively, we can think of many different types of patches in an
image
Or patches with perceptually-significant “features”
![Page 8: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/8.jpg)
Local Image PatchesLocal Image Patches
When is a group of pixels considered a local patch?
The notion of a patch is relative. It can be a single pixel
![Page 9: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/9.jpg)
Local Image PatchesLocal Image Patches
When is a group of pixels considered a local patch?
There is no answer to this question!
The notion of a patch is relative. It can be a single pixel
![Page 10: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/10.jpg)
Local Image PatchesLocal Image Patches
When is a group of pixels considered a local patch?
There is no answer to this question!
The notion of a patch is relative. It can be the entire image
![Page 11: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/11.jpg)
Local Image PatchesLocal Image Patches
We will begin with mathematical properties and methods that
apply mostly to very small patches (e.g., 3x3)
… and eventually consider descriptions that apply to entire
images
![Page 12: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/12.jpg)
Topic 4:Topic 4:
Local analysis of image Local analysis of image
patchespatchespatchespatches
•• What do we mean by an image “patch”?What do we mean by an image “patch”?
•• Applications of local image analysisApplications of local image analysis
•• Visualizing 1D and 2D intensity functionsVisualizing 1D and 2D intensity functions
![Page 13: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/13.jpg)
Patches: Why Do We Care?Patches: Why Do We Care?
Many applications…
• Recognition
• Inspection
• Video-based tracking
• Special effects
![Page 14: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/14.jpg)
Face Recognition and AnalysisFace Recognition and Analysis
http://petapixel.com/2012/03/30/facial-recognition-software-guesses-age-based-on-a-photo/
![Page 15: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/15.jpg)
TrackingTracking
M. Zervos, H. BenShitrit and P. Fua, Real time multi-object tracking using multiple cameras
![Page 16: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/16.jpg)
Editing & Manipulating PhotosEditing & Manipulating Photos
Object removal from a photo
OriginalOriginal NewNew
(Criminisi et al, CVPR 2003)
![Page 17: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/17.jpg)
Editing & Manipulating PhotosEditing & Manipulating Photos
Colorization of black and white photos
Original (B&W)Original (B&W) New (Color)New (Color)
(Levin & Weiss, SIGGRAPH 2004)
![Page 18: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/18.jpg)
Editing & Manipulating PhotosEditing & Manipulating Photos
Scissoring objects from a photo so
urce
imag
esso
urce
imag
es
composite imagecomposite image
sour
ce im
ages
sour
ce im
ages
![Page 19: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/19.jpg)
Giving Photos a “Painted” LookGiving Photos a “Painted” Look
From P. Litwinowicz’s SIGGRAPH’97 paper
“Processing Images and Videos for an
Impressionist Effect”
![Page 20: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/20.jpg)
Topic 4:Topic 4:
Local analysis of image Local analysis of image
patchespatchespatchespatches
•• What do we mean by an image “patch”?What do we mean by an image “patch”?
•• Applications of local image analysisApplications of local image analysis
•• Visualizing 1D and 2D Visualizing 1D and 2D image patches as intensity image patches as intensity
functionsfunctions
![Page 21: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/21.jpg)
Visualizing An Image Visualizing An Image as aas a Surface in 3DSurface in 3D
Gray-scale image
A gray-scale image is
like a function I(x,y)
![Page 22: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/22.jpg)
Image Image ⇔⇔ Surface in 3DSurface in 3D
Gray-scale image SurfaceSurface
And we can visualize this function in 3D
![Page 23: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/23.jpg)
Image Image ⇔⇔ Surface in 3DSurface in 3D
Gray-scale image SurfaceSurfaceSurfaceSurface
![Page 24: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/24.jpg)
Gray-scale image
Image Image ⇔⇔ Surface in 3DSurface in 3D
Image patchImage patch
The same
applies to image
patches
![Page 25: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/25.jpg)
Gray-scale image
Image Image ⇔⇔ Surface in 3DSurface in 3D
Image patchImage patch
Surface patchSurface patch
Patches have their
own coordinate
system.
![Page 26: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/26.jpg)
BTW, notice image noiseBTW, notice image noise
![Page 27: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/27.jpg)
Visualizing a Row or Column Visualizing a Row or Column as a as a Graph in 2DGraph in 2D
Gray-scale image Graph in 2D
Another way of visualizing image data is as a graph in 2D
![Page 28: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/28.jpg)
Image row or column Image row or column ⇔⇔ Graph in 2DGraph in 2D
Gray-scale image Graph in 2D
And of course, we can do this for a 1D patch.
![Page 29: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/29.jpg)
Today we’ll learn aboutToday we’ll learn about
4.1. Today’s lecture is about modeling image data taking into
account more than one (potentially noisy) single pixel.
We will focus on 1D patches.
Methods include:
Computing derivatives of 1D patches using polynomial
fitting via Least-squares, weighted least squares and fitting via Least-squares, weighted least squares and
RANSAC
![Page 30: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/30.jpg)
where are we, and what will where are we, and what will
come after?come after?come after?come after?
••Subtopics:Subtopics:
1. Local analysis of 1D image patches (today)
2. Local analysis of 2D curve patches
3. Local analysis of 2D image patches
![Page 31: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/31.jpg)
Local Analysis of Image Patches: OutlineLocal Analysis of Image Patches: Outline
As graph in 2DAs graph in 2D As curve in 2DAs curve in 2D As surface in 3DAs surface in 3D
![Page 32: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/32.jpg)
Topic 4:Topic 4:
Local analysis of image Local analysis of image
patchespatchespatchespatches
••Subtopics:Subtopics:
1. Local analysis of 1D image patches
2. Local analysis of 2D curve patches
3. Local analysis of 2D image patches
![Page 33: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/33.jpg)
Topic 4.1:Topic 4.1:
Local analysis of 1D image Local analysis of 1D image
patchespatchespatchespatches
•• Taylor series approximation of 1D intensity patchesTaylor series approximation of 1D intensity patches
• Estimating derivatives of 1D intensity patches
Least-squares fitting
Weighted least-squares fitting
Robust polynomial fitting: RANSAC
![Page 34: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/34.jpg)
Topic 4.1:Topic 4.1:
Local analysis of 1D image Local analysis of 1D image
patchespatchespatchespatches
•• Taylor series approximation of 1D intensity patchesTaylor series approximation of 1D intensity patches
• Estimating derivatives of 1D intensity patches:
Least-squares fitting
Weighted least-squares fitting
Robust polynomial fitting: RANSAC
![Page 35: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/35.jpg)
LeastLeast--Squares Polynomial FittingSquares Polynomial Fitting
Taylor approximation: Fit a polynomial Taylor approximation: Fit a polynomial to to the pixel the pixel intensities in a patchintensities in a patch
•• All pixels contribute equally to estimate of All pixels contribute equally to estimate of derivative(s) at patch center (i.e., at x=0)derivative(s) at patch center (i.e., at x=0)
IntensityIntensity
Pixel (x)Pixel (x)00ww--ww
![Page 36: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/36.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2DIf we knew the derivatives of
I(x) at x=0, we can approximate
I(x) using the Taylor Series:
![Page 37: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/37.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2DIf we knew the derivatives of
I(x) at x=0, we can approximate
I(x) using the Taylor Series:
?
![Page 38: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/38.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2DIf we knew the derivatives of
I(x) at x=0, we can approximate
I(x) using the Taylor Series:
The approximation is best at the
origin and degrades from there.
![Page 39: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/39.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2D The n-th order Taylor series
expansion of I(x), near the
patch center (x=0) can then
be written in matrix form as:
Note that an
approximated value for
I(x) will depend on n+1
coefficients: the intensity
derivatives at I(0)
![Page 40: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/40.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2DExample: 0th order approximation
![Page 41: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/41.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2DExample: 1st order approximation
![Page 42: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/42.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2DExample: 2nd order approximation
![Page 43: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/43.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2DAnd so on…
![Page 44: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/44.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
But do we know the derivatives?But do we know the derivatives?
![Page 45: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/45.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
But do we know the derivatives?But do we know the derivatives?
No, but we can estimate them!
![Page 46: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/46.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
And can we estimate them for the entire row?And can we estimate them for the entire row?
![Page 47: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/47.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
And can we estimate them for the entire row?And can we estimate them for the entire row?
Yes, but pixel by pixel.
In fact…
![Page 48: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/48.jpg)
Applying the same operation on multiple patchesApplying the same operation on multiple patches
A “sliding A “sliding window” window” algorithm is a algorithm is a common approach to patchcommon approach to patch--based based operationsoperations
The algorithm goes as follows:The algorithm goes as follows:
![Page 49: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/49.jpg)
A “sliding A “sliding window” window” algorithm is a algorithm is a common approach to patchcommon approach to patch--based based operationsoperations
The algorithm goes as follows:The algorithm goes as follows:
1.1. Define a “pixel window” using a Define a “pixel window” using a window size and a window center.window size and a window center.
Applying the same operation on multiple patchesApplying the same operation on multiple patches
![Page 50: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/50.jpg)
A “sliding A “sliding window” window” algorithm is a algorithm is a common approach to patchcommon approach to patch--based based operationsoperations
The algorithm goes as follows:The algorithm goes as follows:
1.1. Define a “pixel window” using a Define a “pixel window” using a window size and a window center.window size and a window center.
2.2. Apply whatever operation in mind Apply whatever operation in mind
Applying the same operation on multiple patchesApplying the same operation on multiple patches
2.2. Apply whatever operation in mind Apply whatever operation in mind to that patchto that patch
3.3. Move the window center one pixel Move the window center one pixel to define a new windowto define a new window
4.4. Repeat steps 1Repeat steps 1--33
![Page 51: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/51.jpg)
A “sliding A “sliding window” window” algorithm is a algorithm is a common approach to patchcommon approach to patch--based based operationsoperations
The algorithm goes as follows:The algorithm goes as follows:
1.1. Define a “pixel window” using a Define a “pixel window” using a window size and a window center.window size and a window center.
2.2. Apply whatever operation in mind Apply whatever operation in mind
Applying the same operation on multiple patchesApplying the same operation on multiple patches
2.2. Apply whatever operation in mind Apply whatever operation in mind to that patchto that patch
3.3. Move the window center one pixel Move the window center one pixel to define a new windowto define a new window
4.4. Repeat steps 1Repeat steps 1--33
![Page 52: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/52.jpg)
A “sliding A “sliding window” window” algorithm is a algorithm is a common approach to patchcommon approach to patch--based based operationsoperations
The algorithm goes as follows:The algorithm goes as follows:
1.1. Define a “pixel window” using a Define a “pixel window” using a window size and a window center.window size and a window center.
2.2. Apply whatever operation in mind Apply whatever operation in mind
Applying the same operation on multiple patchesApplying the same operation on multiple patches
2.2. Apply whatever operation in mind Apply whatever operation in mind to that patchto that patch
3.3. Move the window center one pixel Move the window center one pixel to define a new windowto define a new window
4.4. Repeat steps 1Repeat steps 1--33
![Page 53: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/53.jpg)
A “sliding A “sliding window” window” algorithm is a algorithm is a common approach to patchcommon approach to patch--based based operationsoperations
The algorithm goes as follows:The algorithm goes as follows:
1.1. Define a “pixel window” using a Define a “pixel window” using a window size and a window center.window size and a window center.
2.2. Apply whatever operation in mind Apply whatever operation in mind
Applying the same operation on multiple patchesApplying the same operation on multiple patches
2.2. Apply whatever operation in mind Apply whatever operation in mind to that patchto that patch
3.3. Move the window center one pixel Move the window center one pixel to define a new windowto define a new window
4.4. Repeat steps 1Repeat steps 1--33
![Page 54: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/54.jpg)
A “sliding A “sliding window” window” algorithm is a algorithm is a common approach to patchcommon approach to patch--based based operationsoperations
The algorithm goes as follows:The algorithm goes as follows:
1.1. Define a “pixel window” using a Define a “pixel window” using a window size and a window center.window size and a window center.
2.2. Apply whatever operation in mind Apply whatever operation in mind
Applying the same operation on multiple patchesApplying the same operation on multiple patches
2.2. Apply whatever operation in mind Apply whatever operation in mind to that patchto that patch
3.3. Move the window center one pixel Move the window center one pixel to define a new windowto define a new window
4.4. Repeat steps 1Repeat steps 1--33
![Page 55: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/55.jpg)
Estimating Derivatives For Image Row rEstimating Derivatives For Image Row r
“Sliding window” algorithm:“Sliding window” algorithm:
•• Define a “pixel window” centered Define a “pixel window” centered at pixel (at pixel (w,rw,r) )
•• Fit nFit n--degree poly to window’s degree poly to window’s intensities (usually n=1 or 2) intensities (usually n=1 or 2)
•• Assign the poly’s derivatives at x=0 Assign the poly’s derivatives at x=0 to pixel at window’s center to pixel at window’s center to pixel at window’s center to pixel at window’s center
•• “Slide” window one pixel over, so “Slide” window one pixel over, so that it is centered at pixel (w+1,r)that it is centered at pixel (w+1,r)
•• Repeat 1Repeat 1--4 until window reaches 4 until window reaches right image borderright image border
![Page 56: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/56.jpg)
Estimating Derivatives For Image Row rEstimating Derivatives For Image Row r
“Sliding window” algorithm:“Sliding window” algorithm:
•• Define a “pixel window” centered Define a “pixel window” centered at pixel (at pixel (w,rw,r) )
•• Fit nFit n--degree poly to window’s degree poly to window’s intensities (usually n=1 or 2) intensities (usually n=1 or 2)
•• Assign the poly’s derivatives at x=0 Assign the poly’s derivatives at x=0 to pixel at window’s center to pixel at window’s center to pixel at window’s center to pixel at window’s center
•• “Slide” window one pixel over, so “Slide” window one pixel over, so that it is centered at pixel (w+1,r)that it is centered at pixel (w+1,r)
•• Repeat 1Repeat 1--4 until window reaches 4 until window reaches right image borderright image border
![Page 57: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/57.jpg)
Estimating Derivatives For Image Row rEstimating Derivatives For Image Row r
“Sliding window” algorithm:“Sliding window” algorithm:
•• Define a “pixel window” centered Define a “pixel window” centered at pixel (at pixel (w,rw,r) )
•• Fit nFit n--degree poly to window’s degree poly to window’s intensities (usually n=1 or 2) intensities (usually n=1 or 2)
•• Assign the poly’s derivatives at x=0 Assign the poly’s derivatives at x=0 to pixel at window’s center to pixel at window’s center to pixel at window’s center to pixel at window’s center
•• “Slide” window one pixel over, so “Slide” window one pixel over, so that it is centered at pixel (w+1,r)that it is centered at pixel (w+1,r)
•• Repeat 1Repeat 1--4 until window reaches 4 until window reaches right image borderright image border
![Page 58: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/58.jpg)
Estimating Derivatives For Image Row rEstimating Derivatives For Image Row r
“Sliding window” algorithm:“Sliding window” algorithm:
•• Define a “pixel window” centered Define a “pixel window” centered at pixel (at pixel (w,rw,r) )
•• Fit nFit n--degree poly to window’s degree poly to window’s intensities (usually n=1 or 2) intensities (usually n=1 or 2)
•• Assign the poly’s derivatives at x=0 Assign the poly’s derivatives at x=0 to pixel at window’s center to pixel at window’s center to pixel at window’s center to pixel at window’s center
•• “Slide” window one pixel over, so “Slide” window one pixel over, so that it is centered at pixel (w+1,r)that it is centered at pixel (w+1,r)
•• Repeat 1Repeat 1--4 until window reaches 4 until window reaches right image borderright image border
![Page 59: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/59.jpg)
Estimating Derivatives For Image Row rEstimating Derivatives For Image Row r
“Sliding window” algorithm:“Sliding window” algorithm:
•• Define a “pixel window” centered Define a “pixel window” centered at pixel (at pixel (w,rw,r) )
•• Fit nFit n--degree poly to window’s degree poly to window’s intensities (usually n=1 or 2) intensities (usually n=1 or 2)
•• Assign the poly’s derivatives at x=0 Assign the poly’s derivatives at x=0 to pixel at window’s center to pixel at window’s center to pixel at window’s center to pixel at window’s center
•• “Slide” window one pixel over, so “Slide” window one pixel over, so that it is centered at pixel (w+1,r)that it is centered at pixel (w+1,r)
•• Repeat 1Repeat 1--4 until window reaches 4 until window reaches right image borderright image border
![Page 60: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/60.jpg)
Estimating Derivatives For Image Row rEstimating Derivatives For Image Row r
“Sliding window” algorithm:“Sliding window” algorithm:
•• Define a “pixel window” centered Define a “pixel window” centered at pixel (at pixel (w,rw,r) )
•• Fit nFit n--degree poly to window’s degree poly to window’s intensities (usually n=1 or 2) intensities (usually n=1 or 2)
•• Assign the poly’s derivatives at x=0 Assign the poly’s derivatives at x=0 to pixel at window’s center to pixel at window’s center to pixel at window’s center to pixel at window’s center
•• “Slide” window one pixel over, so “Slide” window one pixel over, so that it is centered at pixel (w+1,r)that it is centered at pixel (w+1,r)
•• Repeat 1Repeat 1--4 until window reaches 4 until window reaches right image borderright image border
![Page 61: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/61.jpg)
Topic 4.1:Topic 4.1:
Local analysis of 1D image Local analysis of 1D image
patchespatchespatchespatches
•• Taylor series approximation of 1D intensity patchesTaylor series approximation of 1D intensity patches
•• Estimating derivatives of 1D intensity patches:Estimating derivatives of 1D intensity patches:
•• LeastLeast--squares fittingsquares fitting
• Weighted least-squares fitting
• Robust polynomial fitting: RANSAC
![Page 62: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/62.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2D
How to estimate the
Taylor series
approximation from
image data?image data?
![Page 63: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/63.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2D
Surprise!
The nth degree Taylor
approximation can be approximation can be
estimated using a linear
system of equations
(which we can represent
in matrix form).
This is Least Squares!
![Page 64: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/64.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2D We know that the
Taylor series is:
![Page 65: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/65.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2D We know that the
Taylor series is:
The derivatives are unknown
![Page 66: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/66.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2D We know that the
Taylor series is:
But the coefficients are known
![Page 67: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/67.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2D The n-th order Taylor series
expansion of I(x), near the
patch center (x=0) can then
be written in matrix form as:
![Page 68: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/68.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2D The n-th order Taylor series
expansion of I(x), near the
patch center (x=0) can then
be written in matrix form as:
![Page 69: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/69.jpg)
TaylorTaylor--Series Approximation of I(x)Series Approximation of I(x)
As graph in 2DAs graph in 2D The n-th order Taylor series
expansion of I(x), near the
patch center (x=0) can then
be written in matrix form as:
2w+1 equations to
estimate n+1
unknowns
![Page 70: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/70.jpg)
As graph in 2DAs graph in 2D
LeastLeast--Squares Polynomial Fitting of I(x)Squares Polynomial Fitting of I(x)
The equations define the system:
![Page 71: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/71.jpg)
LeastLeast--Squares Polynomial Fitting of I(x)Squares Polynomial Fitting of I(x)
We could then do v=Xd to get
an estimate for all pixels in the
patch in (-w, …, 0, …, w)
![Page 72: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/72.jpg)
LeastLeast--Squares Polynomial Fitting of I(x)Squares Polynomial Fitting of I(x)
![Page 73: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/73.jpg)
00thth--Order (Constant) Estimation of I(x)Order (Constant) Estimation of I(x)
![Page 74: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/74.jpg)
00thth--Order (Constant) Estimation of I(x)Order (Constant) Estimation of I(x)
![Page 75: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/75.jpg)
00thth--Order (Constant) Estimation of I(x)Order (Constant) Estimation of I(x)
![Page 76: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/76.jpg)
00thth--Order (Constant) Estimation of I(x)Order (Constant) Estimation of I(x)
![Page 77: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/77.jpg)
00thth--Order (Constant) Estimation of I(x)Order (Constant) Estimation of I(x)
![Page 78: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/78.jpg)
00thth--Order (Constant) Estimation of I(x)Order (Constant) Estimation of I(x)
![Page 79: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/79.jpg)
11stst--Order (Linear) Estimation of I(x)Order (Linear) Estimation of I(x)
![Page 80: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/80.jpg)
22ndnd--Order (Quadratic) Estimation of I(x)Order (Quadratic) Estimation of I(x)
![Page 81: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/81.jpg)
22ndnd--Order (Quadratic) Estimation of I(x)Order (Quadratic) Estimation of I(x)
Note how all pixels in the window contribute
equally to the estimate around the center of
the window!
![Page 82: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/82.jpg)
Topic 4.1:Topic 4.1:
Local analysis of 1D image Local analysis of 1D image
patchespatchespatchespatches
•• Taylor series approximation of 1D intensity patchesTaylor series approximation of 1D intensity patches
•• Estimating derivatives of 1D intensity patches:Estimating derivatives of 1D intensity patches:
• Least-squares fitting
•• Weighted leastWeighted least--squares fittingsquares fitting
• Robust polynomial fitting: RANSAC
![Page 83: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/83.jpg)
Weighted Least Squares Polynomial FittingWeighted Least Squares Polynomial Fitting
Scenario #1:Scenario #1:
•• Fit polynomial to ALL pixel intensities in a patchFit polynomial to ALL pixel intensities in a patch
IntensityIntensity
Pixel (x)Pixel (x)00 ww--ww
![Page 84: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/84.jpg)
Weighted Least Squares Polynomial FittingWeighted Least Squares Polynomial Fitting
Scenario #2:Scenario #2:
•• Fit polynomial to Fit polynomial to all the pixel all the pixel intensities in intensities in the the patchpatch
•• Pixels contribute to estimate of derivative(s) at Pixels contribute to estimate of derivative(s) at center according to a weight function center according to a weight function ΩΩ(x)(x)
IntensityIntensity
Pixel (x)Pixel (x)00 ww--ww
![Page 85: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/85.jpg)
Polynomial Fitting: A Linear FormulationPolynomial Fitting: A Linear Formulation
![Page 86: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/86.jpg)
Polynomial Fitting: A Linear FormulationPolynomial Fitting: A Linear Formulation
![Page 87: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/87.jpg)
Weighted LeastWeighted Least--Squares Estimation of I(x)Squares Estimation of I(x)
![Page 88: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/88.jpg)
Weighted LeastWeighted Least--Squares Estimation of I(x)Squares Estimation of I(x)
![Page 89: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/89.jpg)
Weighted LeastWeighted Least--Squares Estimation of I(x)Squares Estimation of I(x)
![Page 90: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/90.jpg)
Weighted LeastWeighted Least--Squares Estimation of I(x)Squares Estimation of I(x)
We could then do v=Xd to get We could then do v=Xd to get
an estimate of I(x) for all pixels
in the patch in (-w, …, 0, …, w).
![Page 91: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/91.jpg)
Topic 4.1:Topic 4.1:
Local analysis of 1D image Local analysis of 1D image
patchespatchespatchespatches
•• Taylor series approximation of 1D intensity patchesTaylor series approximation of 1D intensity patches
•• Estimating derivatives of 1D intensity patches:Estimating derivatives of 1D intensity patches:
• Least-squares fitting
• Weighted least-squares fitting
•• Robust polynomial fitting: RANSACRobust polynomial fitting: RANSAC
![Page 92: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/92.jpg)
Robust Polynomial FittingRobust Polynomial Fitting
Scenario #3:Scenario #3:
•• Fit polynomial only to SOME pixel intensities in a patchFit polynomial only to SOME pixel intensities in a patch(the “inliers”)(the “inliers”)
IntensityIntensity
Pixel (x)Pixel (x)00ww--ww
![Page 93: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/93.jpg)
Robust Polynomial FittingRobust Polynomial Fitting
But how can we tell between inliers and outliers?But how can we tell between inliers and outliers?
IntensityIntensity
Pixel (x)Pixel (x)ww--ww
![Page 94: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/94.jpg)
Robust Polynomial FittingRobust Polynomial Fitting
We can’t. At least not before we fit a model.We can’t. At least not before we fit a model.
![Page 95: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/95.jpg)
Polynomial Fitting Using RANSACPolynomial Fitting Using RANSAC
Here’s our problem: find the inliers, fit a polynomial to them:Here’s our problem: find the inliers, fit a polynomial to them:
IntensityIntensity
Given:Given:
•• n = degree of polyn = degree of poly
•• p = fraction of inliers p = fraction of inliers
•• t = fit thresholdt = fit threshold
•• pp = success probability= success probability
Pixel (x)Pixel (x)00
•• ppss = success probability= success probability
ww--ww
![Page 96: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/96.jpg)
RANSAC AlgorithmRANSAC Algorithm
Example: Line fitting using RANSAC (i.e., n=2 unknown Example: Line fitting using RANSAC (i.e., n=2 unknown polynomial coefficients)polynomial coefficients)
•• Step 1: Randomly choose n pixels from the patchStep 1: Randomly choose n pixels from the patch
IntensityIntensity
Pixel (x)Pixel (x)00ww--ww
![Page 97: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/97.jpg)
RANSAC AlgorithmRANSAC Algorithm
Step Step 2: Fit the poly using the chosen pixels/intensities2: Fit the poly using the chosen pixels/intensities
IntensityIntensity
Pixel (x)Pixel (x)00ww--ww
![Page 98: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/98.jpg)
RANSAC AlgorithmRANSAC Algorithm
Step Step 3: Count pixels with vertical distance < threshold t3: Count pixels with vertical distance < threshold t
Pixel (x)Pixel (x)00ww--ww
![Page 99: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/99.jpg)
RANSAC AlgorithmRANSAC Algorithm
•• Step 4: If there aren’t “enough” such pixels, REPEATStep 4: If there aren’t “enough” such pixels, REPEAT(not more than K times)(not more than K times)
Pixel (x)Pixel (x)00ww--ww
![Page 100: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/100.jpg)
RANSAC AlgorithmRANSAC Algorithm
How about these two?How about these two?
IntensityIntensity
Pixel (x)Pixel (x)00ww--ww
![Page 101: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/101.jpg)
RANSAC AlgorithmRANSAC Algorithm
Step Step 4: If there are “enough” such pixels, STOP4: If there are “enough” such pixels, STOPLabel them as “inliers” & do a leastLabel them as “inliers” & do a least--squares fitsquares fitto the INLIER pixels onlyto the INLIER pixels only
IntensityIntensity
Pixel (x)Pixel (x)00ww--ww
![Page 102: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/102.jpg)
RANSAC AlgorithmRANSAC Algorithm
Step Step 4: If there are “enough” such pixels, STOP4: If there are “enough” such pixels, STOPLabel them as “inliers” & do a leastLabel them as “inliers” & do a least--squares fitsquares fitto the INLIER pixels onlyto the INLIER pixels only
IntensityIntensity
Pixel (x)Pixel (x)00ww--ww
![Page 103: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/103.jpg)
RANSAC AlgorithmRANSAC Algorithm
EventuallyEventually, after “enough” trials, , after “enough” trials, there must be some there must be some likelihood of having chosen n+1 inlierslikelihood of having chosen n+1 inliers to fit the model.to fit the model.
IntensityIntensity
Pixel (x)Pixel (x)00ww--ww
![Page 104: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/104.jpg)
RANSAC AlgorithmRANSAC Algorithm
EventuallyEventually, after “enough” trials, , after “enough” trials, there must be some there must be some likelihood of having chosen n+1 inlierslikelihood of having chosen n+1 inliers to fit the model.to fit the model.
How many trials are enough then?How many trials are enough then?
![Page 105: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/105.jpg)
RANSAC AlgorithmRANSAC Algorithm
Repeat at most K times:Repeat at most K times:
1.1. Randomly choose n+1 pixelsRandomly choose n+1 pixels
Given:Given:
•• n = degree of polyn = degree of poly
•• p = fraction of inliers p = fraction of inliers
•• t = fit thresholdt = fit threshold
•• ppss = success probability= success probability
1.1. Randomly choose n+1 pixelsRandomly choose n+1 pixels
2.2. Fit nFit n--degree polydegree poly
3.3. Count pixels whose vertical Count pixels whose vertical distance from poly is < tdistance from poly is < t
4.4. If there are at least If there are at least (2w+1)p pixels, EXIT LOOP(2w+1)p pixels, EXIT LOOP
a. Label them as inliersa. Label them as inliers
b. Fit nb. Fit n--degree poly to all degree poly to all inlierinlier pixelspixels 00
![Page 106: Topic 4: Local analysis of image patchesmangas/teaching/320/slides/... · 2014. 1. 22. · Repeat steps 1Repeat steps 1--3. A “sliding A “sliding window” window” algorithm](https://reader036.vdocument.in/reader036/viewer/2022071512/61326434dfd10f4dd73a6b76/html5/thumbnails/106.jpg)
RANSAC AlgorithmRANSAC Algorithm
Repeat at most K times:Repeat at most K times:
1.1. Randomly choose n+1 pixelsRandomly choose n+1 pixels
Given:Given:
•• n = degree of polyn = degree of poly
•• p = fraction of inliers p = fraction of inliers
•• t = fit thresholdt = fit threshold
•• ppss = success probability= success probability
1.1. Randomly choose n+1 pixelsRandomly choose n+1 pixels
2.2. Fit nFit n--degree polydegree poly
3.3. Count pixels whose vertical Count pixels whose vertical distance from poly is < tdistance from poly is < t
4.4. If there are at least If there are at least (2w+1)p pixels, EXIT LOOP(2w+1)p pixels, EXIT LOOP
a. Label them as inliersa. Label them as inliers
b. Fit nb. Fit n--degree poly to all degree poly to all inlier pixelsinlier pixels