End Effects of EMD

An unsolved, and perhaps,

unsolvable problem.

End Effects

• The end effect problem is a self-inflicted wound on EMD.

• Traditional method in dealing with the end is to use window, which would mask the ends and force the ends to be tapered to zero.

End Effects• In EMD, we could do the same, but we decided

to salvage some thing out of the data near the ends.

• We need at least one data point beyond each end to stabilize the spline. This data is an extremum, so we have to predict or extrapolate the sequence of extrema for an extra point or two beyond the end.

• When the end points are not extrema, the spline could swing wildly. The effects are not limited to the neighborhoods of the ends; they could propagate into the interior of the data especially in the low frequency components.

End Effects

• As EMD method is designed for nonlinear and nonstationary data, forecasting is impossible. However, we do have the following alleviating conditions:

• The forecasting of the extra points are all for each IMF, which are relatively narrow band and more stationary than the data.

• As the extrema points determine the envelope, our task is to extend the envelopes, which have much slower variation than the IMF data, and therefore more forgiving as far as error is concerned.

Solutions for End Effects

• Mirror images: simple mirror image, mirror image and rotations.

• Mirror images and tapering: adding a taper function to force the mirrored data decay to zero.

• Adding characteristics waves: the extra points are determined by the average of n-waves (usually n=3) in the immediate neighborhood of the ends.

• Extension with ‘linear spline’ fittings near the boundaries.• Pattern comparison with the interior data points.• Linear predictions that preserve the power spectral

shape.• Extensive search for the points with minimum interior

perturbations.

‘Linear Spline’ near the Boundary

Wu and Huang, 2009: AADA 1, 1-41

maxima minimaImproved end-effects-corrected method

Red point is the determined extrema. The end points are always both maxima and minima with different values.

0 50 100 150 200 250 300-0.4

-0.2

0

0.2

0.4

0.6Envelopes with end effects corrected

dataupperlowermaximaminma

260 265 270 275 280 285 290 295 300-0.04

-0.02

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

Zoom in of end point : The green point is determined by the straight line linking the last two maxima, if it is less than the data at the end point, the maxima at the end point is chosen as the data itself. Envelope is determined using natural spline.

260 265 270 275 280 285 290 295 300-0.04

-0.02

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

Zoom in of end point :Please notice when using Hermite Spline method to determine envelope, all data will be less than the envelope, this fails when using natural spline. Envelope is determined using Hermite spline.

Minima : The red point is determined by the straight green line linking the last two minima, if it is less than the data at the end point, the minima at the end point is chosen as the data itself. Envelope is determined using Natural spline

260 265 270 275 280 285 290 295 300

-0.1

-0.05

0

0.05

0.1

0.15

260 265 270 275 280 285 290 295 300

-0.1

-0.05

0

0.05

0.1

0.15

Notice even using Hermite Spline, the data is still less than the lower envelope. The problem may be because the last minima is chosen too small using the current method. Envelope is determined using Hermite spline.

260 265 270 275 280 285 290 295 300

-0.1

-0.05

0

0.05

0.1

0.15

A Variation: Here, the minima at the end point is determined as the mean of the data and the green point, which is determined the straight line linking the last two minima. Then, the data will all larger than the lower envelope.

End Effects

End Effects : Details Beginning

End Effects : Details End

Linear Prediction

Linear Predictive Code I

N

n 1 n-1 2 n-2 N n-N k n-kk=1

N

n n n n k n-kk=1

Assuming that we can express any signal by its past history:

ˆ x = a x + a x + ... + a x = a x ,

the error we make is given by

ˆ e = x - x = x - a x .

The usu

2M M N2n n k n-k

n=1 n 1 k=1

al criterion for a good fit for the energy

E = e = x - a x

to be minimum.

Linear Predictive Code II

M N

n k n-k n-in=1 k=1i

M M N N M

n n-i k n-k n-i k n k n in=1 n=1 k=1 k= 1

k

1 n

To minimize E, we have to set

E = 2 x - a x x = 0 .

a

This means

x x = a x x = a x x ,

which leads to

R(i) = a

N

k

k=1

.

Thus, we have N-equations for N-unknow

R( k

ns,

i )

a .

Present Status of Solutions

• None of the above methods is totally justifiable.• All of the above methods have been used in one

or the other occasions with decent results.• No solution is in sight; the forecasting problem in

nonlinear and nonstationary processes is ill posed and might not have solution at all.

• A wide open question for foreseeable future.