image warping (szeliski 3.6.1)

38
Image Warping (Szeliski 3.6.1) cs129: Computational Photography James Hays, Brown, Fall 2012 Slides from Alexei Efros and Steve Seitz tp://www.jeffrey-martin.com

Upload: francis-sargent

Post on 31-Dec-2015

41 views

Category:

Documents


0 download

DESCRIPTION

Image Warping (Szeliski 3.6.1). http://www.jeffrey-martin.com. cs129: Computational Photography James Hays, Brown, Fall 2012. Slides from Alexei Efros and Steve Seitz. f. f. f. T. T. x. x. x. f. x. Image Transformations. image filtering: change range of image g(x) = T(f(x)). - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Image Warping  (Szeliski 3.6.1)

Image Warping (Szeliski 3.6.1)

cs129: Computational PhotographyJames Hays, Brown, Fall 2012

Slides from Alexei Efros and Steve Seitz

http://www.jeffrey-martin.com

Page 2: Image Warping  (Szeliski 3.6.1)

Image Transformations

image filtering: change range of imageg(x) = T(f(x))

f

x

Tf

x

f

x

Tf

x

image warping: change domain of imageg(x) = f(T(x))

Page 3: Image Warping  (Szeliski 3.6.1)

Image Transformations

T

T

f

f g

g

image filtering: change range of image

g(x) = T(f(x))

image warping: change domain of imageg(x) = f(T(x))

Page 4: Image Warping  (Szeliski 3.6.1)

Parametric (global) warpingExamples of parametric warps:

translation rotation aspect

affineperspective

cylindrical

Page 5: Image Warping  (Szeliski 3.6.1)

Parametric (global) warping

Transformation T is a coordinate-changing machine:

p’ = T(p)

What does it mean that T is global and parametric?• Is the same for any point p• can be described by just a few numbers (parameters)

Let’s represent T as a matrix:

p’ = Mp

T

p = (x,y) p’ = (x’,y’)

y

x

y

xM

'

'

Page 6: Image Warping  (Szeliski 3.6.1)

ScalingScaling a coordinate means multiplying each of its components by

a scalarUniform scaling means this scalar is the same for all components:

2

Page 7: Image Warping  (Szeliski 3.6.1)

Non-uniform scaling: different scalars per component:

Scaling

X 2,Y 0.5

Page 8: Image Warping  (Szeliski 3.6.1)

Scaling

Scaling operation:

Or, in matrix form:

byy

axx

'

'

y

x

b

a

y

x

0

0

'

'

scaling matrix S

What’s inverse of S?

Page 9: Image Warping  (Szeliski 3.6.1)

2-D Rotation

(x, y)

(x’, y’)

x’ = x cos() - y sin()y’ = x sin() + y cos()

Page 10: Image Warping  (Szeliski 3.6.1)

2-D Rotation

x = r cos ()y = r sin ()x’ = r cos ( + )y’ = r sin ( + )

Trig Identity…x’ = r cos() cos() – r sin() sin()y’ = r sin() cos() + r cos() sin()

Substitute…x’ = x cos() - y sin()y’ = x sin() + y cos()

(x, y)

(x’, y’)

Page 11: Image Warping  (Szeliski 3.6.1)

2-D RotationThis is easy to capture in matrix form:

Even though sin() and cos() are nonlinear functions of ,• x’ is a linear combination of x and y

• y’ is a linear combination of x and y

What is the inverse transformation?• Rotation by –• For rotation matrices

y

x

y

x

cossin

sincos

'

'

TRR 1

R

Page 12: Image Warping  (Szeliski 3.6.1)

2x2 MatricesWhat types of transformations can be

represented with a 2x2 matrix?

2D Identity?

yyxx

''

yx

yx

1001

''

2D Scale around (0,0)?

ysy

xsx

y

x

*'

*'

y

x

s

s

y

x

y

x

0

0

'

'

Page 13: Image Warping  (Szeliski 3.6.1)

2x2 MatricesWhat types of transformations can be

represented with a 2x2 matrix?

2D Rotate around (0,0)?

yxyyxx

*cos*sin'*sin*cos'

y

x

y

x

cossin

sincos

'

'

2D Shear?

yxshy

yshxx

y

x

*'

*'

y

x

sh

sh

y

x

y

x

1

1

'

'

Page 14: Image Warping  (Szeliski 3.6.1)

2x2 MatricesWhat types of transformations can be

represented with a 2x2 matrix?

2D Mirror about Y axis?

yyxx

''

yx

yx

1001

''

2D Mirror over (0,0)?

yyxx

''

yx

yx

1001

''

Page 15: Image Warping  (Szeliski 3.6.1)

2x2 MatricesWhat types of transformations can be

represented with a 2x2 matrix?

2D Translation?

y

x

tyy

txx

'

'

Only linear 2D transformations can be represented with a 2x2 matrix

NO!

Page 16: Image Warping  (Szeliski 3.6.1)

All 2D Linear Transformations

Linear transformations are combinations of …• Scale,• Rotation,• Shear, and• Mirror

Properties of linear transformations:• Origin maps to origin• Lines map to lines• Parallel lines remain parallel• Ratios are preserved• Closed under composition

y

x

dc

ba

y

x

'

'

yx

lkji

hgfe

dcba

yx''

Page 17: Image Warping  (Szeliski 3.6.1)

Homogeneous CoordinatesQ: How can we represent translation as a 3x3

matrix?

y

x

tyy

txx

'

'

Page 18: Image Warping  (Szeliski 3.6.1)

Homogeneous Coordinates

Homogeneous coordinates• represent coordinates in 2

dimensions with a 3-vector

1

coords shomogeneou y

x

y

x homogenouscoords

Page 19: Image Warping  (Szeliski 3.6.1)

Homogeneous Coordinates

Add a 3rd coordinate to every 2D point• (x, y, w) represents a point at location (x/w, y/w)• (x, y, 0) represents a point at infinity• (0, 0, 0) is not allowed

Convenient coordinate system to

represent many useful transformations

1 2

1

2(2,1,1) or (4,2,2) or (6,3,3)

x

y

Page 20: Image Warping  (Szeliski 3.6.1)

Homogeneous CoordinatesQ: How can we represent translation as a 3x3

matrix?

A: Using the rightmost column:

100

10

01

y

x

t

t

ranslationT

y

x

tyy

txx

'

'

Page 21: Image Warping  (Szeliski 3.6.1)

TranslationExample of translation

11100

10

01

1

'

'

y

x

y

x

ty

tx

y

x

t

t

y

x

tx = 2ty = 1

Homogeneous Coordinates

Page 22: Image Warping  (Szeliski 3.6.1)

Basic 2D TransformationsBasic 2D transformations as 3x3 matrices

1100

0cossin

0sincos

1

'

'

y

x

y

x

1100

10

01

1

'

'

y

x

t

t

y

x

y

x

1100

01

01

1

'

'

y

x

sh

sh

y

x

y

x

Translate

Rotate Shear

1100

00

00

1

'

'

y

x

s

s

y

x

y

x

Scale

Page 23: Image Warping  (Szeliski 3.6.1)

Matrix CompositionTransformations can be combined by

matrix multiplication

wyx

sysx

tytx

wyx

1000000

1000cossin0sincos

1001001

'''

p’ = T(tx,ty) R() S(sx,sy) p

Page 24: Image Warping  (Szeliski 3.6.1)

Affine Transformations

Affine transformations are combinations of …• Linear transformations, and

• Translations

Properties of affine transformations:• Origin does not necessarily map to origin

• Lines map to lines

• Parallel lines remain parallel

• Ratios are preserved

• Closed under composition

• Models change of basis

Will the last coordinate w always be 1?

wyx

fedcba

wyx

100''

Page 25: Image Warping  (Szeliski 3.6.1)

Projective Transformations

Projective transformations …• Affine transformations, and

• Projective warps

Properties of projective transformations:• Origin does not necessarily map to origin

• Lines map to lines

• Parallel lines do not necessarily remain parallel

• Ratios are not preserved

• Closed under composition

• Models change of basis

wyx

ihgfedcba

wyx

'''

Page 26: Image Warping  (Szeliski 3.6.1)

2D image transformations

These transformations are a nested set of groups• Closed under composition and inverse is a member

Page 27: Image Warping  (Szeliski 3.6.1)

Matlab Demo

“help imtransform”

Page 28: Image Warping  (Szeliski 3.6.1)

Recovering Transformations

What if we know f and g and want to recover the transform T?• e.g. better align images from Project 1• willing to let user provide correspondences

– How many do we need?

x x’

T(x,y)y y’

f(x,y) g(x’,y’)

?

Page 29: Image Warping  (Szeliski 3.6.1)

Translation: # correspondences?

How many correspondences needed for translation?

How many Degrees of Freedom?

What is the transformation matrix?

x x’

T(x,y)y y’

?

100

'10

'01

yy

xx

pp

pp

M

Page 30: Image Warping  (Szeliski 3.6.1)

Euclidian: # correspondences?

How many correspondences needed for translation+rotation?

How many DOF?

x x’

T(x,y)y y’

?

Page 31: Image Warping  (Szeliski 3.6.1)

Affine: # correspondences?

How many correspondences needed for affine?

How many DOF?

x x’

T(x,y)y y’

?

Page 32: Image Warping  (Szeliski 3.6.1)

Projective: # correspondences?

How many correspondences needed for projective?

How many DOF?

x x’

T(x,y)y y’

?

Page 33: Image Warping  (Szeliski 3.6.1)

Image warping

Given a coordinate transform (x’,y’) = T(x,y) and a source image f(x,y), how do we compute a transformed image g(x’,y’) = f(T(x,y))?

x x’

T(x,y)

f(x,y) g(x’,y’)

y y’

Page 34: Image Warping  (Szeliski 3.6.1)

f(x,y) g(x’,y’)

Forward warping

Send each pixel f(x,y) to its corresponding location

(x’,y’) = T(x,y) in the second image

x x’

T(x,y)

Q: what if pixel lands “between” two pixels?

y y’

Page 35: Image Warping  (Szeliski 3.6.1)

f(x,y) g(x’,y’)

Forward warping

Send each pixel f(x,y) to its corresponding location

(x’,y’) = T(x,y) in the second image

x x’

T(x,y)

Q: what if pixel lands “between” two pixels?

y y’

A: distribute color among neighboring pixels (x’,y’)– Known as “splatting”– Check out griddata in Matlab

Page 36: Image Warping  (Szeliski 3.6.1)

f(x,y) g(x’,y’)x

y

Inverse warping

Get each pixel g(x’,y’) from its corresponding location

(x,y) = T-1(x’,y’) in the first image

x x’

Q: what if pixel comes from “between” two pixels?

y’T-1(x,y)

Page 37: Image Warping  (Szeliski 3.6.1)

f(x,y) g(x’,y’)x

y

Inverse warping

Get each pixel g(x’,y’) from its corresponding location

(x,y) = T-1(x’,y’) in the first image

x x’

T-1(x,y)

Q: what if pixel comes from “between” two pixels?

y’

A: Interpolate color value from neighbors– nearest neighbor, bilinear, Gaussian, bicubic

– Check out interp2 in Matlab

Page 38: Image Warping  (Szeliski 3.6.1)

Forward vs. inverse warpingQ: which is better?

A: usually inverse—eliminates holes• however, it requires an invertible warp function—not always possible...