1

Spectrum Enhancement

Speech ProcessingDec. 11, 2006

YOUNG-CHAN LEE

2

Spectrum enhancement

Goal : find noiseless spectrum for speech signal

Regard corrupted signal consist of noise, pure signal and background noise.

i

i

i

i

where

x corrupted signal

y pure signal

c noise

b background noise

i i i ix c y b ( ) ( )i ie x e y

Same spectrum components

3

Equation-method(1)

Spectrum enhancement

0

( )

( )

, ( )

where , , , constant

i

i i i

i iN M

i i i j i j i ii j M

j i

i i i i i

xe i G

E N f F

c ye i G

c y c y c y F

x c y f c y estimateof background

and F

Global information (average)

Local information (average)

Estimate of background

4

Spectrogram

0 1 2 3 4 5 6-2

0

2

Time (s)

SPEECH

Fre

quency [

Hz]

Time [s]

SPECTROGRAM

0 1 2 3 4 5 60

2000

4000

Fre

quency [

Hz]

Time [s]

ENHANCED SPECTROGRAM

0 1 2 3 4 5 60

2000

4000

10

where

0.01/128

0.01/10

1

10F

5

New Equation – method-(2)

0

( )

( )

, ( )

where , , , constant

i i i i

i i i

i i i i i i i iN M

i ii i i j i j

i j Mj i

i i i i i

x x x xe i

E N f F

c y c y c y c ye i

c y Fc y c y

x c y f c y estimateof background

and F

Part – 1 (Global energy)Part – 2 (local energy)

Part – 3 (Background estimate)Part – 4 (Floor)

Analysis of each component and find parameters

6

Additive Noise Suppression

2ˆ(1 exp( ))

, where is input and is background estimate.

i i i

ii i i

i

y x x

xx x f

f

Suppress noise compare to speech signal

0 500 1000 1500 2000 2500 3000 3500 40000

5

10

15x 10

4 comparison of input and suppressed signal

frequency[Hz]

mag

nitu

de

spectrum

suppressed

ˆ 0.1

0 500 1000 1500 2000 2500 3000 3500 40000

5

10

15x 10

4 comparison of input and suppressed signal

frequency[Hz]

mag

nitu

de

ˆ 1 ˆ 10

0 500 1000 1500 2000 2500 3000 3500 40000

5

10

15x 10

4 comparison of input and suppressed signal

frequency[Hz]

mag

nitu

de

7

Part - 1

Spectrum and global energy

1

0

( )

where constant

i i iN

ii i

i

i i i

x c ye i

Ec y

x c y

0 500 1000 1500 2000 2500 3000 3500 40000

2000

4000

6000

8000

10000

12000Spectrum and global energy

frequency[Hz]

mag

nitu

de

spectrum

global energy

8

Part - 2

Spectrum and local energy

2 ( )

where is constant

i i iM

ii j i j

j Mj i

i i i

x c ye i

Nc y

x c y

0 500 1000 1500 2000 2500 3000 3500 4000-2000

0

2000

4000

6000

8000

10000

12000Spectrum and local energy

frequency[Hz]

mag

nitu

de

spectrum

local energy

9

Part - 3 Spectrum and background estimate

3 ( )

where is constant

i i i

i i i

i i

x c ye i

f c y

x c y

0 500 1000 1500 2000 2500 3000 3500 4000

0

2000

4000

6000

8000

10000

12000Spectrum and background estimate

frequency[Hz]m

agni

tude

10

Estimate of BackgroundBefore background

estimate

New background estimate

11

Part - 4

Spectrum and floor

4 ( )

where is constant

i i i

i i i

x c ye i

F Fx c y

F

0 1000 2000 3000 40000

1

2

3

4

5

6

7

8

9

10

11x 10

4 Spectrum and floor

frequency[Hz]

magnitude

0 1000 2000 3000 40000

1

2

3

4

5

6

7

8

9

10x 10

5

frequency[Hz]

magnitude

Spectrum and floor

12

Overall spectrum

4

where

0.1/128

0.01/10

0.7

10F

PART-1 SPECTROGRAM(Global Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-2 SPECTROGRAM(Local Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-3 SPECTROGRAM(Background Estimate)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-4 SPECTROGRAM(Floor)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

13

Comparison of enhanced spectrum

0 1 2 3 4 5 6-202

SPEECH

Time (s)SPECTROGRAM

Fre

quency [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

ENHANCED SPECTROGRAM

Fre

quency [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

New ENHANCED SPECTROGRAM

Fre

quency [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

1 2 3 4( ) ( ) ( ) ( ) ( )e i e i e i e i e i

14

File list (used same parameters)1. CCW170007_000.ulaw2. CCW170007_001.ulaw3. CCW170007_002.ulaw4. CCW170007_003.ulaw5. CCW170007_004.ulaw6. CCW170007_005.ulaw7. CCW170007_006.ulaw8. CCW170007_007.ulaw9. CCW170007_008.ulaw10. CCW170007_009.ulaw

Examples

15

Example1(CCW1700007_000.ulaw)

0 1 2 3 4 5 6-202

SPEECH

Time (s)SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

New ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

16

Continue (spectrum)PART-1 SPECTROGRAM(Global Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-2 SPECTROGRAM(Local Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-3 SPECTROGRAM(Background Estimate)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-4 SPECTROGRAM(Floor)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

17

Example2(CCW1700007_001.ulaw)

0 1 2 3 4 5 6 7-202

SPEECH

Time (s)SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

New ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

18

Continue (spectrum)PART-1 SPECTROGRAM(Global Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

PART-2 SPECTROGRAM(Local Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

PART-3 SPECTROGRAM(Background Estimate)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

PART-4 SPECTROGRAM(Floor)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

19

Example3(CCW1700007_002.ulaw)

0 1 2 3 4 5 6-202

SPEECH

Time (s)SPECTROGRAM

Fre

quency [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

ENHANCED SPECTROGRAM

Fre

quency [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

New ENHANCED SPECTROGRAM

Fre

quency [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

20

Continue (spectrum)PART-1 SPECTROGRAM(Global Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-2 SPECTROGRAM(Local Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-3 SPECTROGRAM(Background Estimate)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-4 SPECTROGRAM(Floor)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

21

Example4(CCW1700007_003.ulaw)

0 1 2 3 4 5 6 7-202

SPEECH

Time (s)SPECTROGRAM

Fre

quency [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

ENHANCED SPECTROGRAM

Fre

quency [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

New ENHANCED SPECTROGRAM

Fre

quency [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

22

Continue (spectrum)PART-1 SPECTROGRAM(Global Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

PART-2 SPECTROGRAM(Local Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

PART-3 SPECTROGRAM(Background Estimate)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

PART-4 SPECTROGRAM(Floor)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

23

Example5(CCW1700007_004.ulaw)

0 1 2 3 4 5 6-202

SPEECH

Time (s)SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

New ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

24

Continue (spectrum)PART-1 SPECTROGRAM(Global Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-2 SPECTROGRAM(Local Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-3 SPECTROGRAM(Background Estimate)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-4 SPECTROGRAM(Floor)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

25

Example6(CCW1700007_005.ulaw)

0 1 2 3 4 5 6-202

SPEECH

Time (s)SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

New ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

26

Continue (spectrum)PART-1 SPECTROGRAM(Global Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-2 SPECTROGRAM(Local Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-3 SPECTROGRAM(Background Estimate)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-4 SPECTROGRAM(Floor)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

27

Example7(CCW1700007_006.ulaw)

0 1 2 3 4 5 6-202

SPEECH

Time (s)SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

New ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

28

Continue (spectrum)PART-1 SPECTROGRAM(Global Average)

Fre

quency [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-2 SPECTROGRAM(Local Average)

Fre

quency [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-3 SPECTROGRAM(Background Estimate)

Fre

quency [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-4 SPECTROGRAM(Floor)

Fre

quency [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

29

Example8(CCW1700007_007.ulaw)

0 1 2 3 4 5 6-202

SPEECH

Time (s)SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

New ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

30

Continue (spectrum)PART-1 SPECTROGRAM(Global Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-2 SPECTROGRAM(Local Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-3 SPECTROGRAM(Background Estimate)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-4 SPECTROGRAM(Floor)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

31

Example9(CCW1700007_008.ulaw)

0 1 2 3 4 5 6-202

SPEECH

Time (s)SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

New ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

32

Continue (spectrum)PART-1 SPECTROGRAM(Global Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-2 SPECTROGRAM(Local Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-3 SPECTROGRAM(Background Estimate)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

PART-4 SPECTROGRAM(Floor)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6

020004000

33

Example10(CCW1700007_009.ulaw)

0 1 2 3 4 5 6 7-202

SPEECH

Time (s)SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

New ENHANCED SPECTROGRAM

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

34

Continue (spectrum)PART-1 SPECTROGRAM(Global Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

PART-2 SPECTROGRAM(Local Average)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

PART-3 SPECTROGRAM(Background Estimate)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

PART-4 SPECTROGRAM(Floor)

Fre

quen

cy [

Hz]

Time [s]0 1 2 3 4 5 6 7

020004000

35

Conclusions

Global energy, local energy, background estimate greatly affected to build enhanced spectrum.

Part3 and part4 showed good spectrum estimation.

In new method, we don’t need to change parameter values according to input files.

36

MATLAB Demo

37

Thank you!

Merry Christmas