sampling schemes for 2-d signals with finite rate of innovation * pancham shukla communications and...

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION*

Pancham Shukla

Communications and Signal Processing GroupImperial College London

This research is supported by EPSRC.

Dr P L Dragotti

by

supervisor

A transfer talk on

1/3/2005

2

SAMPLING SCHEMES FOR 2-D SIGNALS WITH FINITE RATE OF INNOVATION

OUTLINE

1. INTRODUCTION• Sampling: Problem, Background, and Scope

2. SIGNALS WITH FINITE RATE OF INNOVATION (FRI) (non-bandlimited)• Definition, Extension in 2-D• 2-D Sampling setup, Sampling kernels and their properties

3. SAMPLING OF FRI SIGNALS

• SETS OF 2-D DIRACSLocal reconstruction (amplitude and position)

• BILEVEL POLYGONS & DIRACS using COMPLEX MOMENTSGlobal reconstruction (corner points)

• PLANAR POLYGONS using DIRECTIONAL DERIVATIVESLocal reconstruction (corner points)

4. CONCLUSION AND FUTURE WORK

3

• Why sampling? Many natural phenomena are continuous (e.g. Speech, Remote sensing) and required to be observed and processed by sampling.

Many times we need reconstruction (perfect !) of the original phenomena.

Continuous Discrete (samples) Continuous

• Sampling theory by Shannon (Kotel’nikov, Whittaker)

• Why not always ‘bandlimited-sinc’? (Although powerful and widely used since 5 decades)

1. Real world signals are non-bandlimited.2. Ideal low pass (anti-aliasing, reconstruction) filter does not exist. (Acquisition devices)3. Shannon’s reconstruction formula is rarely used in practice with finite length signals (esp. images) due to infinite support and slow decay of ‘sinc’ kernel. (do we achieve PR in practice?)


1. INTRODUCTION

)(tx ][ns )(ˆ tx

‘bandlimited-sinc’ scenario withthe assumption of Perfect Reconstruction !

)()( tSINCt T )()( tSINCt T )()(ˆ txtx

n

nTt )(

sms f

Tff1

,2

mfmf

mff mff )(tx

*PR ?

Motivation:

4

• Extensions of Shannon’s theorySo… many papers but for comprehensive account, we refer to [Jerry 1977, Unser 2000].

• Shift-invariant subspaces [Unser et al.]

The classes of non-bandlimited signals (e.g. uniform splines) residing in the shift-invariant subspaces can be perfectly reconstructed. The other non-bandlimited signals are approximated through their projections.


1. INTRODUCTION contd.

We look into:

Non-bandlimited signals that do not reside in shift-invariant subspace but have a parametric representation.

Non-traditional ways of perfect reconstruction… ….from the projections of such signals in the shift-invariant subspace.

Is it possible to perfectly reconstruct such signals from their samples?

Any examples of such signals ?

What type of kernels ?

Sampling and reconstruction schemes?

e.g. uniform spline

Non-bandlimited signal

projection

Shift-invariant subspace

5

Very recently such signals are identified and termed as

• Signals with Finite Rate of Innovation (or FRI signals) [ Vetterli et al. 2002]Model: Non-bandlimited signals that do not reside in shift-invariant subspace.Examples: Streams of Diracs, non-uniform splines, and piecewise polynomials.

Unique feature: A finite number of degrees of freedom per time (rate of innovation )e.g. a Dirac in 1-D has a rate of innovation = 2 (i.e. amplitude and position).

• The sampling schemes for such signals in 1-D are given by [Vetterli, Marziliano and Blu 2002].• Extensions of these schemes in 2-D are given by [Maravic and Vetterli 2004], however, focusing on

Sampling kernels: as sinc and Gaussian.

Algorithms: Little more involved reconstruction algorithms (solution of linear systems, root finding) based on Annihilating filter method [from Spectral estimation, Error correction coding].Reconstruction: Only a finite number of samples ( ) guarantees perfect reconstruction.


2. FRI SIGNALS

6

• Assortments of kernels [Dragotti, Vetterli and Blu, ICASSP-2005]

For 1-D FRI signals, one can use varieties of kernels such as

1. That reproduce polynomials (satisfy Strang and Fix conditions)

2. Exponential Splines (E-Splines) [Unser]

3. Functions with rational Fourier transforms

• Our FocusSampling extensions in 2-D using above mentioned kernels, in particular, for

Sets of 2-D Diracs Local & Global schemes: Local kernels & Complex moments + (AFM)

Bilevel polygons Global scheme: Complex moments + Annihilating filter method (AFM)

Planar polygons Local scheme: Directional derivatives + Directional kernels


2. FRI SIGNALS contd.

),( yxg ),( yxg),( yxg

7

• Sampling setup

• Properties of sampling kernelsIn current context, any kernel that reproduce polynomials of degrees =0,1,2…-1 such that

Partition of unity:

Reproduction of polynomials along x-axis:

Reproduction of polynomials along y-axis:


2. SAMPLING FRI SIGNALS in 2-D

Zj Zk

xy kyjx 1),(

e.g., B-Splines (biorthogonal) and Daubechies scaling functions (orthogonal) are valid kernels

)/,/(),,(, kTyjTxyxgS yxxykj

Zkxy

Zjj xkyjxC ),(,

1jC ,

Zkxy

Zjk ykyjxC ),(, kC ,

0

Input signal

Sampling kernel

Set of samples in 2-D

8

• Sets of 2-D Diracs: Local reconstructionConsider and with support such that

there is at most one Dirac in an area of size . Assume .

From the partition of unity

(reproducing of polynomial of degree 0),it follows that

),(),( , kjxyZj Zk

kj yyxxayxg


3. SETS OF 2-D DIRACS

yyxx TLTL

),( yxxy yx LL

),(, qpxyqp yyxxa 1, yx TT

x yL

j

L

kkjqp Sa

1 1,,

qp

L

j

L

kqpxyqp

x y

L

j

L

kxyqpxyqp

L

j

L

kxyqpxyqp

L

j

L

kkj

a

kyjxa

dydxkyjxyyxxa

kyjxyyxxaS

x y

x y

x yx y

,

1 1,

1 1,

1 1,

1 1,

),(

),(),(

),(),,(

(property: partition of unity)

This is derived as follows,

B-Splines oforder one

The amplitude

Onlyinner products overlap the unique Dirac

yx LL

3.12.1

55.210 ,, kjqp Sa

9

• Properties of sampling kernelsIn current context, any kernel that reproduce polynomials of degrees =0,1,2…-1 such that

Partition of unity:

Reproduction of polynomials along x-axis:

Reproduction of polynomials along y-axis:


2. SAMPLING FRI SIGNALS in 2-D

Zj Zk

xy kyjx 1),(

e.g., B-Splines (biorthogonal) and Daubechies scaling functions (orthogonal) are valid kernels

Zkxy

Zjj xkyjxC ),(,

1jC ,

Zkxy

Zjk ykyjxC ),(, kC ,

0

10

• Sets of 2-D Diracs: Local reconstruction contd.

… and using polynomial reproduction properties along x and y directions,

the coordinate positions are given by


3. SETS OF 2-D DIRACS contd.

qp

L

j

L

kkjj

p a

SC

x

x y

,

1 1,,1

Above relations are derived as,

Similarly, it is easy to follow that

As long as any two Diracs are sufficiently apart, we can accurately reconstruct a set of Diracs, considering one Dirac per time.

11


4. BILEVEL POLYGONS & DIRACS: Complex-moments

• Complex-moments for polygonal shapes: earlier works

Since decades, moments are used to characterize unspecified objects. [Shohat and Tamarkin 1943, Elad et al. 2004]. Here, we present a sampling perspective to the results of [Davis 1964, Milanfar et al. 1995, Elad et al. 2004] on reconstruction of polygonal shapes using complex-moments.

Milanfar et al. (1995 ) extended the above work using as followsnzzh )(

)()(),(1

i

N

ii zhdydxzhyxg

Result of Davis (1964): For any non-degenerate, simply connected polygon with corner

points ( ) in closure of any analytic function ,following holds

),( yxg N

iz )(zh

where are complex weights that depend on the ordered connection of corner points i iz

O

nsn dydxzyxg ),(

Onw

n dydxzyxgnn 2),()1

Definition: The nth simple and weighted complex-moments of a given function over a

complex Cartesian plane in the closure are given by

),( yxg

yxz 1

Simple moment Weighted moment

12

• Complex-moments for polygonal shapes: a modern connection



wn

sn

n

n

ni

N

ii

nn

dydxzyxgnn

dydxzyxg

dydxzhyxgz

2

)2(

"1

)1(

),()1(

)(),(

)(),(

(simple moment)

(weighted moment)

Results of Milanfar et al. (1995): Milanfar et al. considered a bilevel polygon that is non-

degenerate, simply connected and convex. They showed that when and is ‘1’ in the

closure and ‘0’ out side. It follows that for

nzzh )( ),( yxg

x

y

12....2,1,0 Nn

0n 1n 2n

Theorem [Milanfar et al.]: For a given non-degenerate, simply connected, and convex polygon in the complex Cartesian plane, all its N corner points are uniquely determined by its weighted complex-moments up to order 2N-1.w

n

Now, we will briefly review the annihilating filter method due to its relevance in finding weights and positions of the corner points zi from the observed complex moments.

0n

1n2n

13



• Annihilating filter method (we refer to [Vetterli, Stoica and Moses] for more details)

This method is well known in the field of Error-correcting codes and Spectral estimation. Especially, in second application, it is employed to determine the weights and locations of the spectral components, generally observed in form of

i iu

The annihilating filter method consists of the following steps:

1. Design the annihilating filter A(z): such that for filter

with its z-transform the condition holds. .1][)(1

0

1

0

N

ii

N

l

l zuzlAzA

NllA ,...1,0],[

0][][ nnA

2. Locations: The convolution condition is solved by the following Yule-Walker system

3. Weights: Once the locations are known, eq. (1) is solved for the weights by the following Vandermonde system

iu i

ni

N

ii un

1

][ NnCuR ii ,,where

(1)

]12[

]1[

][

][

]2[

]1[

]1[]32[]22[

]1[]1[][

]0[]2[]1[

N

N

N

NA

A

A

NNN

NN

NN

The roots of the filter are the locations iu

]1[

]1[

]0[111

1

1

0

11

11

10

110

Nuuu

uuu

NNN

NN

N

Gives the weights i

14


4. BILEVEL POLYGONS & DIRACS: Complex-moments• A sampling perspective (using Complex-moments + Annihilating filter method)

Consider g(x,y) as a non-degenerate, simply connected, and convex bilevel polygon with N corner points

Consider g(x,y) as a set of N 2-D Diracs

,)/,/(),,(, kTyjTxyxgS yxxykj

),( yxxy can reproduce polynomials up to degree 2N-1 ( = 0,1...2N-1)

2

),()1(

)(),(

)2(1

n

dydxzyxgnn

dydxzhyxgz

wn

On

O

N

i

nii

n

dydxzyxg

dydxzhyxgz

sn

On

O

N

i

nii

),(

)(),(1

wn

kjn

j k

yk

xj

ni

N

ii SCCnnz

,

)2(,1,1

1

1)1( sn

kjn

j k

yk

xj

ni

N

ii SCCza

,,1,11

1

Then from the complex-moments formulation of Milanfar et al.

Because of the polynomial reproduction property of the kernel, we derive that

where are complex weights and are corner points of the bilevel polygon

i iz where denotes amplitudes of the Diracs and are complex position of the Diracs

ia iz

Now using annihilating filter method, it is straightforward to see that

izwn )(zA )(zA iz

iasn

15


4. BILEVEL POLYGONS & DIRACS: Complex-moments• Simulation results

Bilevel polygon with N=3corner points

A set of N=3 Diracs

),(*),( yxyxg xy ),(*),( yxyxg xy ),( yxg),( yxg

),(),( 5 yxyx xyxy

kjS ,kjS ,

)(zAwn )(zAs

n

16



• Summary

1. A polygon with N corner points is uniquely determined from its samples using a kernel that reproduce polynomials up to degree 2N-1 along both x and y directions.

2. A set of Diracs is uniquely determined from its samples using a kernel that reproduces polynomials up to degree 2N-1 along both x and y directions and that there are at most N Diracs in any distinct area of size .

3. Global reconstruction algorithm

Complexity Complexity of the signal

Numerical instabilities in algorithmic implementations for very close corner points

yyxx TNLTNL

17


5. PLANAR POLYGONS: Directional-derivatives

• Problem formulation Intuitively, for a planar polygon, two successive directional derivatives along two adjacent sides of the polygon result into a 2-D Dirac at the corner point formed by the respective sides.

Continuous model:

Discrete challenge:

Lattice theory: Directional derivatives Discrete differences

• Subsampling over integer lattices and

• Local directional kernels in the framework of 2-D Dirac sampling (local reconstruction)

In present context, we have access to samples only.kjS ,

Planar polygon with

N corner points

N pairs of orientations

N pairs ofdirectional derivatives

N Diracs

Local reconstructionscheme of 2-D Diracs

Amplitudes and positions of the corner points

18


5. PLANAR POLYGONS: Directional-derivatives• Lattice theory We refer to [Cassels, Convey and Sloan] for more detail.

Base lattice: is a subset of points of Z2 (integer lattice)

Each pattern of subsampling (or ) over the integer lattice is characterized by a non-unique

Sampling matrix

2,21,2

2,11,1

2

1

vv

vv

v

vV

For example, by taking

12

21V , the base lattice is illustrated as

2)tan(tan 11,1

2,111

v

v

2/1)tan(tan 21,2

2,212

v

vA

11, v

22 , v

19


5. PLANAR POLYGONS: Directional-derivatives• Proposed sampling schemeConsider a planar polygon with corner points and a sampling kernel that satisfies

partition of unity (reproduces polynomial of degree zero).

The observed samples are given by

),( yxg ),( yxxyN

),(),,(, yxyxgS xykj

Therefore, using lattice theory, apply a pair of

directional differences and along

and over the samples identified

by the base lattice and its sampling matrix

It then follows,

1D 2D

1 2 kjS ,

V

))0(),0(())(),((

))(),(())(),((

),,(

2,11,1

2,21,22,12,21,121

)0(),0()(),()(),()(),(, 2,11,12,21,22,12,21,12112

yxvyvx

vyvxvvyvvx

yxg

SSSSSDD

xyxy

xyxy

vvvvvvvvkj

By using Parseval’s identities and after certain manipulations, we have

),(,),(

)(det 2112

12

,yxyxg

V

SDD kj

)sin(

),(*),(),(

12

021

21

yxyxyx

xy

where

Dirac at A Modified kernel

20



),(,),(

)det( 2121

12

,yxyxg

V

SDD kj

)sin(

),(*),(),(

12

021

21

yxyxyx

xy

where

Dirac at A Modified kernel

• Directional kernels

Modified kernel is a ‘directional kernel’. For each corner point independent directional kernel.

For example,

),( yxxy ),(021

yx ),(21

yx

support: yx LvvLvv 2,22,11,21,1support: yx LL

21


5. PLANAR POLYGONS: Directional-derivatives• Local reconstruction of the corner point

The directional kernel can reproduce polynomials of degrees 0 and 1 in both x and y directions.

Assume that there is only one corner point support of its associated directional kernel. Then from the local reconstruction scheme of Diracs, we can reconstruct theamplitude and the position of an equivalent Dirac at a given corner point (e.g. at point A ) as:

),(21

yx

)(det

,

,

12

V

SDD

aj k

kj

qp

)(det,

,,1 12

Va

SDDC

xqp

j kkj

xj

p

)(det,

,,1 12

Va

SDDC

yqp

j kkj

yk

q

2)tan(tan 11,1

2,111

v

v

2/1)tan(tan 21,2

2,212

v

v

A

11, v

22 , v

22


5. PLANAR POLYGONS: Directional-derivatives• Simulation result

Original polygon with 3 corner points

After first pair of directional difference on samples

Samples of the polygon using Haar kernel

After third pair of directional difference on samples

After second pair of directional difference on samples

Pair of directional differences

Local reconstruction

23



Planar polygon with

N corner points


N pairs ofdirectional derivatives

N Diracs



• Summary: reconstruction algorithm

Initial intuition: Final realization:

Planar polygon with

N corner points


N pairs ofdirectional differences

N Diracs



Q)tan(R

yx LvvLvv 2,22,11,21,1

Only one corner point in the support of its directional kernel

Enough number of samples kjS ,

Advantage:

1. Local reconstruction.

2. Only local reconstruction complexity, irrespective of the number of corner points in a polygon.

24

• Conclusion

• We have proposed several sampling schemes for the classes of 2-D non-bandlimited

signals.

• In particular, sets of Diracs and (bilevel and planar) polygons can be reconstructed from

their samples by using kernels that reproduce polynomials.

• Combining the tools like annihilating filter method, complex-moments, and directional

derivatives, we provide local and global sampling choices with varying degrees of

complexity.


6. CONCLUSION & FUTURE WORK

25


6. CONCLUSION & FUTURE WORK

• Future work

From March 2005 to October 2005:

1. Exploring a different class of kernels, namely, exponential splines (E-Splines).

2. Extending the sampling schemes in higher dimension. For instance, using the notion of complex numbers in 4-D (quaternion).

3. Considering more intricate cases such as piecewise polynomials inside the polygons, and planar shapes with piecewise polynomial boundaries.

We plan to submit a paper for IEEE Transactions on Image Processing by summer 2005.

From November 2005 to June 2006:

1. Studying the wavelet footprints [Dragotti 2003] and then extending them in 2-D

2. Integrating the proposed sampling schemes with the footprints in 2-D

3. Investigating the sampling situations when the signals are perturbed with the noise

4. Developing resolution enhancement algorithms for satellite images.

26

Questions?

sampling schemes for 2-d signals with finite rate of innovation * pancham shukla communications and...

Documents

signals vetterli

scope signals

finite length signals

reconstruction schemes

sampling kernels

finite rate of innovation1

real world signals

properties sampling