1 linear prediction. 2 linear prediction (introduction) : the object of linear prediction is to...

1

Linear PredictionLinear Prediction

2

Linear Prediction Linear Prediction (Introduction)(Introduction)::

The object of linear prediction is to The object of linear prediction is to estimate the output sequence from a estimate the output sequence from a linear combination of input samples, linear combination of input samples, past output samples or both :past output samples or both :

The factors The factors a(i)a(i) and and b(j)b(j) are called are called predictor coefficients.predictor coefficients.

p

i

q

j

inyiajnxjbny10

)()()()()(ˆ

3

Linear Prediction Linear Prediction (Introduction)(Introduction)::

Many systems of interest to us are Many systems of interest to us are describable by a linear, constant-describable by a linear, constant-coefficient difference equation :coefficient difference equation :

If If Y(z)/X(z)=H(z),Y(z)/X(z)=H(z), where where H(z)H(z) is a ratio of is a ratio of polynomials polynomials N(z)/D(z),N(z)/D(z), then then

Thus the predicator coefficient given us immediate Thus the predicator coefficient given us immediate access to the poles and zeros of access to the poles and zeros of H(z). H(z).

q

j

p

i

jnxjbinyia00

)()()()(

p

i

iq

j

j ziazDzjbzN00

)()( and )()(

4

Linear Prediction Linear Prediction (Types of (Types of

System Model)System Model):: There are two important variants :There are two important variants :

All-pole model (in statistics, All-pole model (in statistics, autoregressive (AR)autoregressive (AR) model ) : model ) :

The numerator The numerator N(z)N(z) is a constant. is a constant. All-zero model (in statistics, All-zero model (in statistics, moving-moving-

averageaverage (MA) model ) : (MA) model ) : The denominator The denominator D(z)D(z) is equal to unity. is equal to unity.

The mixed pole-zero model is called the The mixed pole-zero model is called the autoregressive moving-averageautoregressive moving-average (ARMA) (ARMA) model.model.

5

Linear Prediction Linear Prediction (Derivation of (Derivation of

LP equations)LP equations):: Given a zero-mean signal Given a zero-mean signal y(n), y(n), in the AR in the AR

model :model :

The error is :The error is :

To derive the predicator we use the To derive the predicator we use the orthogonality orthogonality principleprinciple, the principle states that the desired , the principle states that the desired coefficients are those which make the error coefficients are those which make the error orthogonal to the samples orthogonal to the samples y(n-1), y(n-2),…, y(n-p).y(n-1), y(n-2),…, y(n-p).

p

i

inyiany1

)()()(ˆ

p

i

inyia

nynyne

0

)()(

)(ˆ)()(

6


LP equations)LP equations):: Thus we require that Thus we require that

Or,Or,

Interchanging the operation of averaging and Interchanging the operation of averaging and summing, and representing < > by summing summing, and representing < > by summing over n, we haveover n, we have

The required predicators are found by The required predicators are found by solving these equations.solving these equations.

p..., 2, 1,jfor 0)()( nejny

p1,...,j ,0)()()(0

n

p

i

jnyinyia

7


LP equations)LP equations):: The orthogonality principle also states that The orthogonality principle also states that

resulting minimum error is given byresulting minimum error is given by

Or,Or,

We can minimize the error over all time :We can minimize the error over all time :

wherewhere

Eriap

ii

0

)(

)()()(2 nenyneE

Enyinyian

p

i

)()()(0

, ...,p,jria ji

p

i

21 ,0)(0

8

Linear Prediction Linear Prediction (Applications)(Applications)::

Autocorrelation matching :Autocorrelation matching : We have a signal y(n) with known We have a signal y(n) with known

autocorrelation . We model this autocorrelation . We model this with the AR system shown below :with the AR system shown below :

)(nryy

p

i

ii za

zAzH

1

1)(

)(

)(neσ

1-A(z)

)(nz

9

Linear Prediction Linear Prediction (Order of Linear (Order of Linear

Prediction)Prediction):: The choice of predictor order depends on The choice of predictor order depends on

the analysis bandwidth. The rule of thumb the analysis bandwidth. The rule of thumb is :is :

For a normal vocal tract, there is an average of For a normal vocal tract, there is an average of about one formant per kilohertz of BW.about one formant per kilohertz of BW.

One formant require two complex conjugate One formant require two complex conjugate poles.poles.

Hence for every formant we require two Hence for every formant we require two predicator coefficients, or two coefficients per predicator coefficients, or two coefficients per kilohertz of bandwidth.kilohertz of bandwidth.

10

Linear Prediction Linear Prediction (AR Modeling of (AR Modeling of

Speech Signal)Speech Signal):: True Model:True Model:

DTImpulse

generator

G(z)GlottalFilter

UncorrelatedNoise

generator

H(z)Vocal tract

Filter

R(z)LP

Filter

Voiced

Unvoiced

Pitch Gain

Gain

V

U

U(n)

Voiced

Volume

velocity

s(n)

Speech

Signal

11

Linear Prediction Linear Prediction (AR Modeling of (AR Modeling of

Speech Signal)Speech Signal):: Using LP analysis :Using LP analysis :

DTImpulse

generator

WhiteNoise

generator

All-PoleFilter(AR)

Voiced

Unvoiced

Pitch

Gain

estimate

V

U

H(z)

s(n)

Speech

Signal

12

3.3 LINEAR PREDICTIVE CODING 3.3 LINEAR PREDICTIVE CODING MODEL FOR SREECH MODEL FOR SREECH

RECOGNITIONRECOGNITION

)(nu

G

)(zA)(ns

13

3.3.1 The LPC Model3.3.1 The LPC Model

.)(

1

1

1

)(

)()(

)()()(

,)()()(

),(...)2()1()(

1

1

1

21

zAza

zGU

zSzH

zGUzSzazS

nGuinsans

pnsansansans

p

i

ii

p

i

ii

p

ii

p

Convert this to equality by including an excitation term:

14

3.3.2 LPC Analysis Equations3.3.2 LPC Analysis Equations

.1)(

)()(

)()()()()(

).()(

).()()(

1

1

~

1

~

1

p

k

kk

p

kk

p

kk

p

kk

zazS

zEzA

knsansnsnsne

knsans

nGuknsans

The prediction error:

Error transfer function:

15


.)()(

)(

)()(

)()(

2

1

2

m

p

knknn

mnn

n

n

kmsamsE

meE

mneme

mnsmS

We seek to minimize the mean squared error signal:

16

pikiai

kmSimSki

kmSimSamsims

pka

E

p

knkn

mnnn

m mnn

p

kknn

k

n

,...,2,1),()0,(

)()(),(

)()()()(

,...,2,1,0

1

1

Terms of short-term covariance:

(*)

With this notation, we can write (*) as:

A set of P equations, P unknowns

17


p

knkn

mnn

p

kk

mnn

ka

kmsmsamsE

1

1

2

).,0()0,0(

)()()(

The minimum mean-squared error can be expressed as:

18

3.3.3 The Autocorrelation Method3.3.3 The Autocorrelation Method

.0

1),()(),(

0

1),()(),(

)(

.,0

10),().()(

)(1

0

1

0

1

0

2

pk

pikimsmski

pk

pikmsimski

meE

otherwise

Nmmwnmsms

kiN

mnnn

pN

mnnn

pN

mnn

n

w(m): a window zero outside 0≤m≤N-1The mean squared error is:

And:

19


)(),(

:functionation autocorrel simple toreducesfunction

covariance thek,-i offunction aonly is),( Since

.0

1),()(),(

)(1

0

kirki

ki

pk

pikimsmski

nn

n

kin

mnnn

20


.

)(

)3(

)2(

)1(

)0(...)3()2()1(

)3(...)0()1()2(

)2(...)1()0()1(

)1(...)2()1()0(

:as formmatrix in expressed becan and

1),(|)(|

:)()( i.e.

symmetric, isfunction ation autocorrel theSince

2

1

1

pr

r

r

r

a

a

a

rprprpr

prrrr

prrrr

prrrr

piirakir

sokrkr

n

n

n

n

pnnnn

nnnn

nnnn

nnnn

p

knkn

nn

21


22


23


24

3.3.4 The Covariance Method3.3.4 The Covariance Method

1

1

0

1

0

2

.0

1),()(),(

, variablesof changeby or,

0

1),()(),(

:as defined ),(with

)(

:directlyspeech unweighted theuse to

and 10 error to computing of interval thechange

iN

imnnn

n

N

mnn

n

N

mnn

pk

pikimsmski

pk

pikmsimski

ki

meE

Nm

25

3.3.4 The Covariance Method3.3.4 The Covariance Method

.

)0,(

)0,3(

)0,2(

)0,1(

),()3,()2,()1,(

),3()3,3()2,3()1,3(

),2()3,2()2,2()1,2(

),1()3,1()2,1()1,1(

3

2

1

pa

a

a

a

ppppp

p

p

p

n

n

n

n

pnnnn

nnnn

nnnn

nnnn

The resulting covariance matrix is symmetric, but not Toeplitz,and can be solved efficiently by a set of techniques called Cholesky decomposition

26

3.3.6 Examples of LPC Analysis3.3.6 Examples of LPC Analysis

27

3.3.6 Examples of LPC Analysis3.3.6 Examples of LPC Analysis

28

3.3.7 LPC Processor for Speech Recognition3.3.7 LPC Processor for Speech Recognition

29


).1()()(

.0.19.0,1)(~~

1~

nsansns

azazH

Preemphasis: typically a first-order FIR, To spectrally flatten the signalMost widely the following filter is used:

30

3.3.7 LPC Processor for Speech Recognition3.3.7 LPC Processor for Speech RecognitionFrame Blocking:

.1,...,1,0

1,...,1,0),()(

~

L

NnnMsnx

31


Hamming Window:Hamming Window:

,,...,1,0),()()(

.10,1

2cos46.054.0)(

.10),()()(

~1

0

~

~

pmmnxnxmr

NnN

nnw

Nnnwnxnx

mN

n

WindowingWindowing

Autocorrelation Autocorrelation analysisanalysis

32


1ifor ommitted is (*)in summation the:note

,)1(

1(*),|)(|)(

)0(

)1(2)(

)1()1()(

)(

)1(1

1

)1(

)0(

ii

i

ijii

ij

ij

ii

i

iL

j

iji

EkE

k

k

piEjirirk

rE

LPC Analysis, to find LPC coefficients, LPC Analysis, to find LPC coefficients, reflection coefficients (PARCOR), the log reflection coefficients (PARCOR), the log area ratio coefficients, the cepstral area ratio coefficients, the cepstral coefficients, …coefficients, … Durbin’s methodDurbin’s method

33


.1

1logtscoefficienratioarealog

tscoefficienPARCOR

1,tscoefficienLPC )(

m

mm

m

pmm

k

kg

k

pma

34


LPC parameter conversion to cepstral LPC parameter conversion to cepstral coefficientscoefficients

,

1

model LPCin gain term theis ln

1

1,

,

1

1

220

m

kkmkm

kmk

m

kmm

pmacm

kc

pmacm

kac

c

35


m

mjm

j

m

mjm

j

ecjmes

eceS

)()(log

|)(|log

Parameter weightingParameter weighting Low-order cepstral coefficients are sensitive to overall Low-order cepstral coefficients are sensitive to overall

spectral slopespectral slope High-order cepstral coefficients are sensitive to noiseHigh-order cepstral coefficients are sensitive to noise The weighting is done to minimize these sensitivitiesThe weighting is done to minimize these sensitivities

36


,1

).(

,|)(|log

, Qmcwc

jmcc

eces

mmm

mm

m

mjm

j

QmQ

mQwm

1,sin

21

37


))(),...,(),(),(),...,(),((

Finally

)()()( :optionallyor

)()()(

dowlength win-finite aover fit polynomial orthogonalan by eApproximat

)(|),(|log

:derivative time theoftion representa seriesFourier

2121 tctctctctctco

KtcKtctc

ktctct

tc

et

tctes

t

QQt

mmm

K

Kkmm

m

mj

m

mj

Temporal cepstral derivativeTemporal cepstral derivative

38

3.3.9 Typical LPC Analysis Parameters3.3.9 Typical LPC Analysis Parameters

N number of samples in the analysis frame N number of samples in the analysis frame M number of samples shift between framesM number of samples shift between framesP LPC analysis orderP LPC analysis orderQ dimension of LPC derived cepstral vectorQ dimension of LPC derived cepstral vectorK number of frames over which cepstral K number of frames over which cepstral

time derivatives are computedtime derivatives are computed

39

333333KK

121212121212QQ

1010101088pp

100 (10 100 (10 msec)msec)

80 (10 80 (10 msec)msec)

100 (15 100 (15 msec)msec)

MM

300 (30 300 (30 msec)msec)

240 (30 240 (30 msec)msec)

300 (45 300 (45 msec)msec)

NN

parameter kHzFs 67.6 kHzFs 8 kHzFs 10

Typical Values of LPC Analysis Parameters for Typical Values of LPC Analysis Parameters for Speech-Recognition SystemSpeech-Recognition System

1 linear prediction. 2 linear prediction (introduction) : the object of linear prediction is to...

Documents

linear prediction introduction

linear prediction applications

object of linear prediction

pole model

true model

autoregressive ar model

minimum error

mean squared error signal