di ss2013 hesita_presentation_final
Post on 24-Jun-2015
47 Views
Preview:
TRANSCRIPT
The 6th Workshop on Disfluency in Spontaneous Speech
Stockholm, Sweden August 21-23, 2013
Sara Candeias 1
Dirce Celorico 1
Jorge Proença 1
Arlindo Veiga 1,2
Fernando Perdigão 1,2
1Instituto de Telecomunicações, Coimbra, Portugal2University of Coimbra, DEEC, Portugal
HESITA(tions) in Portuguese Database
2
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Scope
Goal
Description of the HESITA Database Hesitation Patterns Hesitations across speaking styles Phonetic form of filled pauses Segmentation of hesitations
Technical Information
Future
SUMMARY
3
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Scope
Goal
Description of the HESITA Database Hesitation Patterns Hesitations across speaking styles Phonetic form of filled pauses Segmentation of hesitations
Technical Information
Future
SUMMARY
4
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
LINGUISTIC or CLINICAL/THERAPEUTIC areas
more directly interested in gathering knowledge for better
identifying salient information in human speech
communication
Various scientific domains can beneficiate of the analysis of the
hesitation distribution along the speech:
SCOPE
5
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
LINGUISTIC or CLINICAL/THERAPEUTIC areas
more directly interested in gathering knowledge for better
identifying salient information in human speech
communication
Various scientific domains can beneficiate of the analysis of the
hesitation distribution along the speech:
SCOPE
SPEECH TECHNOLOGY
to increase the usability of speech systems, by overpassing the challenges proposed by the presence of such phenomena.
6
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
AUTOMATIC LANGUAGE PROCESSING
could benefit from a richer representation of the audio signal that
incorporates speaking styles information (hesitations),
to reduce errors in the automatic speech recognition,
to improve automatic conversational speech systems.
DETECTION OF HESITATION EVENTS
provides the segmentation of multimedia data into consistent
parts,
leads to important applications : identification of the speech
segments to train acoustic models for speech recognition in
spontaneous speech.
SCOPE
7
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
SCOPE
No database of hesitation events for European Portuguese is freely
available so far !
8
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Scope
Goal
Description of the HESITA Database Hesitation Patterns Hesitations across speaking styles Phonetic form of filled pauses Segmentation of hesitations
Technical Information
Future
SUMMARY
9
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Database for European Portuguese,
mainly focused on the hesitation events,
containing a large and rich variety of speech data events.
GOAL
HESITA database
Available through:
Meta-Net: http://metanet4u.l2f.inesc-id.pt/repository/search/
Project page: http://lsi.co.it.pt/spl/hesitation/downloads.html
10
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Scope
Goal
Description of the HESITA Database Hesitation Patterns Hesitations across speaking styles Phonetic form of filled pauses Segmentation of hesitations
Technical Information
Future
SUMMARY
11
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
HESITA Database 30 daily news programs
collected from podcasts of a European Portuguese television channel
~ 27 hours of speech
audio downsampled from 44.1 kHz to 16 kHz sampling rate,
video information discarded,
studio and out of studio recordings, some telephone sessions.
DESCRIPTION OF THE HESITA DATABASE
12
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
HESITA Database prepared (read) speaking style is dominant:
most of the speech encompasses utterances of anchors and professional speakers (14 hours),
spontaneous speech segments present:
in commentators, reporters, interviewers and interviewees (10 hours),
Lombard speech appears with low representativeness (18 minutes).
DESCRIPTION OF THE HESITA DATABASE
13
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
HESITA Database
Manually identified and annotated hesitation events:
DESCRIPTION OF THE HESITA DATABASE
patterns closely following the notation presented in E. Shriberg
14
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
HESITA Database
Manually identified and annotated hesitation events:
DESCRIPTION OF THE HESITA DATABASE
repetitions (r),
substitutions (s),
filler words (p),
deletions (d) and
insertions (i).
Only the speech segments were annotated in terms of hesitations,
Filled pause vocalizations were transcribed using the SAMPA phonetic alphabet for European Portuguese.
15
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
HESITA Database
Annotation encompasses information regarding to:
audio characteristics - background environments:
studio, street, speech overlapping, noise and music,
DESCRIPTION OF THE HESITA DATABASE
acoustic events - non-speech events:
music, jingles, laughter, coughing or clapping.
respiratory and other events:
noise from cars or wind,
speaking style and speaker information.
16
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
HESITA Database
All the annotations were performed by using the Transcriber software tool.
DESCRIPTION OF THE HESITA DATABASE
17
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
DESCRIPTION OF THE HESITA DATABASE
SP_STU_E3_M represents:
an annotation of speech segment (SP),
in a noise-free environment, studio, (STU),
with high level of spontaneity (E3), and
from a male speaker (M).
18
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
DESCRIPTION OF THE HESITA DATABASE
SP_OVR_E3_M represents:
the annotation of an speech segment (SP)
with overlapping speech (OVR),
in a spontaneous speaking style with high level of spontaneity (E3), and
from a male speaker (M).
19
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
DESCRIPTION OF THE HESITA DATABASE
(r.r) - repetitions (r),
(.w+) - extensions within a word (w+)
(f.) - filled pauses (f).
[6~]: (f.) - phonetic symbols attest extended vowel sounds or vocalic fillers.
res - presence of a respiratory event.
20
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Scope
Goal
Description of the HESITA Database Hesitation Patterns Hesitations across speaking styles Phonetic form of filled pauses Segmentation of hesitations
Technical Information
Future
SUMMARY
21
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Considering the segments annotated accordingly to the presence of hesitations, we can see how the hesitation patterns are distributed.
HESITATION PATTERNS
Top 10 most frequent hesitation patterns.
total of 4608 events observed,
filled pauses (f.) and vocalic extensions within a word (.w+) are the most common.
22
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Pattern models display the way that the hesitation occurs, indicating the order of the words before and after the so-called “repair-point”.
HESITATION PATTERNS
Top 10 most frequent hesitation patterns.
The repair point (as in (f.)) marks the place from which the hesitation is repaired and the fluency is restored.
pattern (r.r) indicates that a word r was repeated as repair or reinforcement ("de.de");
pattern (s-.s), the word s was cut and then substituted ("qua-.quantas");
in (r2.r) the same word r was repeated twice and finally restored ("com.com.com");
in (rs-.rs) the word r was repeated and word s was cut and, then substituted with correction ("da tu-.da totalidade").
23
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
More complex hesitation patterns are present…
HESITATION PATTERNS
Embedded hesitations:
"que vo-.que.que.que voltam.que.que possam"
" that re-.that. that. that return. that. that could "
( ( r s- .(r 2 . r) s) . ( r . r ) s).
24
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Scope
Goal
Description of the HESITA Database Hesitation Patterns Hesitations across speaking styles Phonetic form of filled pauses Segmentation of hesitations
Technical Information
Future
SUMMARY
25
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
In general hesitation events occur mainly in spontaneous speech
4406 against 188 in read (prepared) speech and 12 in Lombard speech,
total of 188 hesitations observed in 14 hours for read (prepared) speaking style results in a rate of 0.22 hesitations per minute,
4406 hesitation events in 10 hours of spontaneous speech result in a rate of 7.34 hesitations per minute.
HESITATIONS ACROSS SPEAKING STYLES
The density of hesitations in speech varies with the speaking style
26
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
HESITATIONS ACROSS SPEAKING STYLES
Distribution of the 5 most common hesitation patterns in the read (prepared) speech
High frequency of vocalic extensions (.w+) (39.36%) just followed by filled pauses (f.) (32.45%).
Top 5 most frequent hesitation patterns for read (prepared) speech.
Although the difference between those two occurrences is not so expressive, it is possible that the choice for the extensions reflects the fact that vocalic fillers tend to be more stigmatized in a prepared speech context.
27
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
HESITATIONS ACROSS SPEAKING STYLES
Distribution of the 5 most common hesitation patterns in the read (prepared) speech
High frequency of vocalic extensions (.w+) (39.36%) just followed by filled pauses (f.) (32.45%).
Top 5 most frequent hesitation patterns for read (prepared) speech.
Repetitions in read or prepared speech become residual.
The occurrence of substitutions are higher in the prepared speech than in spontaneous speech (9.57% vs. 3.61%).
‘proving’ that they are more adequate for communicative strategy mainly in what the fluency of speaking is concerned.
28
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Scope
Goal
Description of the HESITA Database Hesitation Patterns Hesitations across speaking styles Phonetic form of filled pauses Segmentation of hesitations
Technical Information
Future
SUMMARY
29
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
PHONETIC FORM OF FILLED PAUSES
The two most common phonetic forms for filled pauses:
the near-open central vowel [ɐ] ([6] in SAMPA),
the mid-central vowel [ə] ([@] in SAMPA).
Phone distribution of filled pauses (top10 most frequent).
This distribution supports the view that the vocalizations preferred by Portuguese speakers are around central vowels, corresponding to the reduced vowels in an unstressed position.
30
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
PHONETIC FORM OF FILLED PAUSES
The two most common phonetic forms for filled pauses:
the near-open central vowel [ɐ] ([6] in SAMPA),
the mid-central vowel [ə] ([@] in SAMPA).
Phone distribution of filled pauses (top10 most frequent).
Slight inclination for the high back rounded nasal vowel [u] as well (around 3%).
A nasal preference is also evident : see [ɐT ], [ɐm] and [ɐT m] or [u].
Our point here is not to associate a meaning to the filler sounds. However, there is strong empirical evidence that speakers use all of them for playing a structuring role in the speech.
31
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
PHONETIC FORM OF FILLED PAUSES
The two most common phonetic forms for filled pauses:
the near-open central vowel [ɐ] ([6] in SAMPA),
the mid-central vowel [ə] ([@] in SAMPA).
Phone distribution of filled pauses (top10 most frequent).
Slight inclination for the high back rounded nasal vowel [u] as well (around 3%).
A nasal preference is also evident : see [ɐT ], [ɐm] and [ɐT m] or [u].
The choice for a vocalic sound rather than other appears to be, at least in some contexts, motivated by the behavior of neighbor phonetic segments, neutralizing in some way the phonetic difference of the vocalic fillers.
32
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Scope
Goal
Description of the HESITA Database Hesitation Patterns Hesitations across speaking styles Phonetic form of filled pauses Segmentation of hesitations
Technical Information
Future
SUMMARY
33
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Annotation of patterns closely follows E. Shriberg methodology,
Encompasses the initial and final temporal marks,
Corresponding label contains the pattern and the orthographic transcription,
Repair-point marked temporally, showing the instant where the hesitation is corrected and when the fluency on speech is recovered.
SEGMENTATION OF HESITATIONS
The period of time that corresponds to the beginning of the hesitation to its repair-point is much larger (0.61 seconds in average) than the period of time between the repair point and the end of the hesitation correction (0.34 seconds in average).
These trends concerning the distribution and duration of hesitation events may be analyzed as manifestations of planning effort as well.
34
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Scope
Goal
Description of the HESITA Database Hesitation Patterns Hesitations across speaking styles Phonetic form of filled pauses Segmentation of hesitations
Technical Information
Future
SUMMARY
35
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Directories and files:
The archive that can be uploaded contains 58 audio files and the corresponding TRS files, that enclose the two parts of the 30 daily newsprograms.
Data structure of an entry:
The TRS files have a data type definition file associated: trans-14.dtd that is provided in the archive.
TECHNICAL INFORMATION
Corpora size:
TRS files have a total of 4608 hesitation events.
The whole resource occupies 3GB, mainly due to the audio files.
36
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
Thank You
FUTURE...
We really expect that this database can be a relevant base of work for further studies regarding a variety of speech phenomena.
The 6th Workshop on Disfluency in Spontaneous Speech
Stockholm, Sweden August 21-23, 2013
Sara Candeias 1
(saracandeias@co.it.pt)Dirce Celorico 1
Jorge Proença 1
Arlindo Veiga 1,2
Fernando Perdigão 1,2
1Instituto de Telecomunicações, Coimbra, Portugal2University of Coimbra, DEEC, Portugal
HESITA(tions) in Portuguese Database
top related