1 computer science 631 lecture 4: wavelets ramin zabih computer science department cornell...

1

Computer Science 631Lecture 4: Wavelets

Ramin ZabihComputer Science DepartmentCORNELL UNIVERSITY

2

Outline

Announcement: HW#1 exists! One more try at Powerpoint formulas Wavelet transformation via the Haar basis 2D wavelets

3

Ray-based coordinates

Compute the position of the pixel X w.r.t. an oriented ray PQ• Coordinates are A (along PQ) and B

(perpendicular to PQ)

P

Q X

A

BA

Q P

Q PX P=

--

× -b gb g

BQ P

Q PX P=

--

× -^b gb g

4

Changing units

The problem is that A and B are in units of pixels• Need them in percentages of the length of PQ

uA

Q P

Q P

Q PX P=

-=

-

-× -b gb g2

vB

Q P

Q P

Q PX P=

-=

--

× -^b gb g2 P

Q X

5

General formula

Note that X’ (as well as X) is a point, not a pixel

X’

P

Q X

P’

Q’

a

b

a

b

X P a Q P b Q P' ' ( ' ' ) ' '= + × - + × - ^b g

6

Fast bilinear interpolation

The value at the interior point (x,y) is

To compute this fast:

p10

p00 p01

p11

(x,y)

p x y p x y p x y p xy pxy = - - × + - × + - × + ×1 1 1 100 10 01 11b gb g ( ) ( )

p p x p p

p p x p p

p p y p p

x

x

xy x x x

0 00 10 00

1 01 11 01

0 1 0

= + -= + -= + -

( )

( )

( )

7

Frequency decomposition of an image

An image can be described in several ways• So far, in terms of pixels (= spatial domain)• The frequency decomposition is very useful• Low-frequency components change slowly• High-frequency components change rapidly

If the image has no (little) high-frequency components, then aliasing is not a problem

8

Why use the frequency domain?

By describing an image in terms of the frequency domain, many things become clear• The image formation process itself removes really

high-frequency components– What happens when we take a picture of a checkerboard

where a pixel contains 100 squares?

– Many image operations are naturally viewed in terms of their effects on various frequency components

• Local averaging removes high-frequency components

– Image compression is best viewed in this way

9

Choice of basis

The canonical way to describe the frequency of an image is in terms of its Fourier transform• This involves a number of issues that we don’t have time

to cover in depth Instead, we will use a Wavelet representation called

the Haar basis• Same basic idea, but easier and more intuitive• For example, our basis vectors will be mostly 0

– By contrast, the Fourier basis is sine waves

10

Image representations

Consider a 1-D image (signal) with four elements: I = [9 7 3 5]• Spatial representation:

I = 9*[1 0 0 0]+7*[0 1 0 0]+3*[0 0 1 0]+5*[0 0 0 1]• The basis elements are 1-D (in this case) vectors,

each with a single 1– What are they called for an image?

11

Wavelets and DC components

Our basis vectors will have a scale, which intuitively means how many non-zero elements• To begin with, we will subtract the average value of

I, which is 6 in our exampleI - 6 = [3 1 -3 -1] I = 6*[1 1 1 1] + [3 1 -3 -1]

• [1 1 1 1] is our first (dull) new basis– No zeros, so coarsest possible scale

• The average value of an image is referred to as its DC component, others are AC components

12

Haar basis AC components

For wavelets, the basis functions are shifted and scaled versions of a single function• Like Fourier basis, with sine/cosine• The “generating function” for the Haar basis is

[1 -1] – everywhere else it’s 0

• This is a pretty simple way to describe a signal!

13

Computing detail coefficients

The average of two numbers is halfway between them• Therefore, a pair [a b] can be represented as:

avg*[1 1] + diff*[1 -1], where

avg = (a + b)/2 and diff = a - avg = avg - b• For the Haar basis, we will use this trick

– For a pair of numbers, compute the average and difference

14

Multi-resolution averaging

Given a signal of size 2n, we can average the pixels together at various scales• Compute a pairwise average, to get size 2n-1

• Compute a pairwise average on that, too• Eventually you get a single number

Example (n = 2)

[9 7 3 5][8 4][6]

421

scale average

15

Generating detail coefficients

Each time we compute the average, we can write down the difference too[9 7] -> [8 1], [3 5] -> [4 -1]

We can reconstruct [8 4] from [6] and [2]

[9 7 3 5][8 4][6]

421

scale average

[1 -1][2]

detail

16

Example

Wavelet basis

-2

-1

0

1

2

Goal: represent the input (left) in terms of a DC term, plusscaled and shifted versions of the basis function (right)

AveragesScale: 4

0

2

4

6

8

10

17

One step of average and difference

DifferencesScale:4 - Scale:2

-2

-1

0

1

2

AveragesScale:2

0

2

4

6

8

10

Add these to get our original input [9 7 3 5]

18

Last step of average and difference

AveragesScale:1

0

1

2

3

4

5

6

7

DifferencesScale:2 - Scale:1

-3

-2

-1

0

1

2

Add these to get the previous average [8 8 4 4]

19

Wavelet representation

There are several ways to do the representation• For the Haar basis, we use a single DC component,

and everything else is details

6*[1 1 1 1]+2*[1 1 -1 -1]+1*[1 -1 0 0] +-1*[0 0 1 -1]• Note that you can build the averages at one level from

the averages and differences on the next level down– We can reconstruct [8 4] from [6] and [2]

[8 4] = 6*[1 1] + 2*[1 -1]

20

Given an input signal, we can create the transformed signal by repeated averaging and differencing

[9 7 3 5]

[8 4 1 -1]

[6 2 1 -1]

Computing the transform

Algorithm:Turn each adjacent pair into an (avg,diff) pairStore all the avg’s in the left half of outputStore all the diff’s in the right halfRecurse on the left half of the outputStop when there is a single number left

Inverting is a simple exercise...

21

Scaling the Haar basis vectors

The basis vectors for the Haar basis are shifted and scaled versions of [1 -1]

We’d like the basis to be orthonormal• For any two elements u,v we want the inner

product utv to be 1 if u = v, otherwise 0• What is [1 -1]t[1 -1] ?• Solution: multiple basis vectors by the

appropriate constant

22

What about 2-D images

First issue is that images are not necessarily of size 2n by 2n

• What is the wavelet transform of [1 3 9]? This issue arises in many places

• For example, if you want to smooth an image. • Obvious solution is to replace each pixel by a

local average– But what do you do at the borders?

23

Three obvious solutions

Pad the image with something• Typically, pad with 0• For a local operation, discard the values that are

within radius of the border– Just need to be sure the process doesn’t crash

“Make up” some other stuff around the image• Question is what to make up• Obvious answer is to use the image itself!

24

Standard tiling

25

Reverse tiling

26

Reverse tiling is in general better

Why? The difference between the Fourier

transform and the transform that JPEG compression uses is (to simplify) standard versus reverse tiling• Discrete Fourier Transform = standard• Discrete Cosine Transform = reverse

27

2-D wavelet transform

The standard decomposition first computes the 1D wavelet transform of each row• The result is a row of the same length• Order by decreasing importance (resolution)

– DC coefficient first

• We treat the result as an image, then compute a 1D wavelet transform of each column

• Result is an image with a single DC coefficient– Rest are details (usually, details of details…)

28

Standard decomposition example

29

What are the 2-D basis functions?

Under the standard basis, examples are:

Gray = 0

30

Non-standard 2D Haar basis