prosody modification in speech signals project by edi fridman & alex zalts supervision by yizhar...
TRANSCRIPT
Prosody modification in speech signals
Project by
Edi Fridman & Alex Zalts
supervision by
Yizhar Lavner
Prosody: the "non-textual" aspects of the speech signal
”Segmental" aspects: timing, duration, rhythm, stress, and metrical structure. The duration of each individual "segment"
is under the control of the speaker to varying degrees, and varies with stress and rate.
The relative strength of an individual syllable, word, or phrase may be realized in a number of ways, including
lengthening (or shortening and cliticization), changes in pitch, and amplitude, and spectral character.
Project goals
• Prosody modification with TDPSOLA algorithm
• Prosody modification with HNM model
• Conversion of male voice to female voice & vice versa
Four steps in prosody modification• Time-scale
modification
• Pitch-scale modification
• Energy envelope modification
• Modification of distribution of utterancers
0 50 100 150 200 250 300 350-0.04
-0.02
0
0.02
0.04
0 50 100 150 200 250 300 350 400 450 500-0.04
-0.02
0
0.02
0.04
0 50 100 150 200 250 300 350-0.04
-0.02
0
0.02
0.04
0 50 100 150 200 250 300 350-0.03
-0.02
-0.01
0
0.01
0.02
0 50 100 150 200 250 300 350-0.04
-0.02
0
0.02
0.04
0 50 100 150 200 250 300 350-0.04
-0.02
0
0.02
0.04
3500 4000 4500 5000 5500-0.02
-0.01
0
0.01
0.02
2500 3000 3500 4000 4500-0.02
-0.01
0
0.01
0.02
TDPSOLA Approach
(*) Based on Overlapp-and-Add idea
(*) Synchronization with original pitch by:1) Setting up pitch marks in analysis signal2) Setting up new pitch marks in synthesis signal according to time-scale and pitch-scale factors (0.6 for pitch 1.3 for time)
(*) Building synthesis signal using OLA
Let us define time instants in analysis signal ta(s) as original pitch marks and pitch contour as P(t)
The stream of synthesis pitch-marks ts(u) is determined from ta(s) according to desired time-scale modification (tD(t)) and pitch-
scale modification Fp(P) by:
ts(u+1)-ts(u) = P`(t) dtts`(u+1)-ts`(u)
ts`(u+1)
ts`(u)
1
with
ts(u+1) = D(ts`(u+1))
Setting up new pitch marks
P`(t) = Fp (P(t))
Problem of TDPSOLA:
Impossible to change pitch contour
because algorithm is based on original pitch marks
0 50 100 150 200 250 300 350 4000
0.2
0.4
0.6
0.8
1
0 50 100 150 200 250 300 350 4000
0.2
0.4
0.6
0.8
1
Problem: too many pitch marks are not counted in, resulting bad sound quality
original pitch-marks
new pitch-marks
HNM Approach
• Speech signal is modeled as harmonics of pitch plus noise
• Harmonics and noise are treated in different
• ways
• Synthesis and analysis are performed in pitch synchronous way
Let X(n) be the speech segment. According to HNM model can be found and written as:
)()(1
1 nwzhnXp
k
nkk
where the complex constants hk and zk are defined as:
)exp( kkk jAh Tfjz kk )2exp( hk - complex amplitude of harmonic K
fk - frequency of harmonic KT - sampling period
W(n) - noise
)(nX
To minimize error )()( nXnX
Amplitudes and phases of pitch-harmonics computed with Prony algorithm by minimizing least square error between harmonics and original signal yielding:
Harmonic K is set to be K*F0 where F0 is pitch that found by PDA
][*][ xZhZZ HH
In each voiced speech fragment maximum voiced frequency Fm is calculated and noise part obtained by filtering signal with HP filter with cutoff frequency Fm
In unvoiced fragments signal’s specturm is modeled by pth-order all-pole filter H(z). The noise is synthesized by filtering a unit variance gaussian noise through H(z)
When pitch scaling is done there is a need to re-compute amplitudes and phases of modified pitch-harmonics.
For this purpose a frequency-continuous spectral and phase envelope is
necessary.
Comparing between TDPSOLA & HNM
TDPSOLA HNM
Sound quality very good very goodwith possiblebuzziness
Pitch contourmodification
can be donewith a lot ofcomputationalload
can be donein easy way
Computational load low high
The only target in pitch-scaling was to change F0 preserving other
formants
There was an attempt to change spectral envelope in order to change male voice to female voice and vice
versa
New algorithm was proposed