Download - The role of the prenuclear F0 region in the perception of German questions and statements
1
The role of the prenuclear F0 The role of the prenuclear F0 region in the perception of German region in the perception of German
questions and statementsquestions and statements
Caterina Petrone* & Oliver Niebuhr***Zentrum für Allgemeine Sprachwissenschaft, Berlin** Laboratoire Parole et Langage, Aix-en-Provence
2
What´s a tune?What´s a tune?• Sequence of static tones (cf. AM model,
Pierrehumbert,1980) or contour elements (KIM, Kohler, 1991)
- No structural (only semantic) restrictions for combinations of pitch accents + edge components
- For pitch accents: nuclear = prenuclearIntroduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
• Compositionality -> Intonational meaning given by the independent contribution of individual tones (Pierrehumbert & Hirschberg, 1990, inter alia)
H* L- L%+ + = Statement
L* H- H%+ + = Yes/No question
Tunes Sentence mode
‘Nucleus’ (nuclear accent + terminal edgecomponents) essential in conveying meaning
Prenuclear region?
BUT:
English
3
Q/S in NeapolitanQ/S in Neapolitan
Tr. “The mom wants to see the frog ”
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
La mam vuo le ve de re la ra na?ma
Lamam vuole ve de re la ra nama
• Intonation only mean to distinguish Yes/No Q vs.S:Intonation only mean to distinguish Yes/No Q vs.S: -- Late vs. Early nuclear rise Late vs. Early nuclear rise (D’Imperio, 2000)(D’Imperio, 2000)
- Convex vs. Concave prenuclear fall - Convex vs. Concave prenuclear fall (Petrone, 2008)(Petrone, 2008)L L
H H
time
F 0
AM
(LH)*
(LH)* L+H*
L*+H
Q
S
L-L%
L-L%
+ steep+ convex
+ shallow+ concave
4
0
10
20
30
40
50
60
70
80
90
100
short medium long
Fragments Length
Qu
esti
on
s s
co
re (
%)
Question baseStatement base
Pren/Rise Pren/Fall Nucl/Rise
Tune composition
ResultsResults
• Q/S effect already in the prenuclear accent region• Score decreases for statement-base when a
steep region is heard
**
*
• Score increases for question-base stimuli only when the nuclear accent is present
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
50%
pre AP nucl
HØ
5
Q/S in GermanQ/S in German• Marked by syntactic (subject-verb inversion),
lexical (e.g., wh-words) and/or intonational means (final F0 rise/fall)
H* (L)H* L- L%
H* (L)H* H-H%
Statement
Yes/No question
:
:
Katherina sucht ‘ne Wonhung(“Katherina searches for a flat”)
Sucht Katherina ‘ne Wonhung?(“Does Katherina search for a flat”)
Sucht Katherina ‘ne Wonhung?(“Does Katherina search for a flat”)
AND:
H* (L)H* L- L%
• Questions with final fall: Dialogue partner is supposed to give a short answer according to the speaker’s expectation (cf. Stock 1996; Kohler 2004; Peters 2005).
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
6
Q/S in GermanQ/S in German• Peters (2005):
Sind sie Heidelbergerin H* (L)H* L- L%
(“Are you from Heidelberg”)
= Give me just a short answer (Yes/No), pleaseIntroduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
7
Q/S in GermanQ/S in German• Peters (2005):
Sind sie Heidelbergerin H* (L)H* L- L%
(“Are you from Heidelberg”)
= Give me just a short answer (Yes/No), please
Sind sie Heidelbergerin H* (L)H* H-H%
(“Are you from Heidelberg”)
= Answer Yes/No + Tell me a bit more about you!
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
8
Q/S in GermanQ/S in German• Peters (2005):
Sind sie Heidelbergerin H* (L)H* L- L%
(“Are you from Heidelberg”)
= Give me just a short answer (Yes/No), please
Sind sie Heidelbergerin H* (L)H* H-H%
(“Are you from Heidelberg”)
= Answer Yes/No + Tell me a bit more about you!
• Kohler (2004):
Würde Ihnen das passen H* (L)H* L- L%
(“Would that suit you”)
= Say ‘yes’, please
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
9
Q/S in GermanQ/S in German• Peters (2005):
Sind sie Heidelbergerin H* (L)H* L- L%
(“Are you from Heidelberg”)
= Give me just a short answer (Yes/No), please
Sind sie Heidelbergerin H* (L)H* H-H%
(“Are you from Heidelberg”)
= Answer Yes/No + Tell me a bit more about you!
• Kohler (2004):
Würde Ihnen das passen H* (L)H* L- L%
(“Would that suit you”)
= Say ‘yes’, please
Würde Ihnen das passen H* (L)H* H-H%
(“Would that suit you”)
= The final choice is up to you!
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
10
Q/S in GermanQ/S in German• Also questions with declarative syntax are possible:
Katherina sucht ‘ne Wonhung
H* (L)H* H- H%
(“Katherina searches for a flat”)
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
11
Q/S in GermanQ/S in German• Also questions with declarative syntax are possible:
• In such cases the intonation is crucial, i.e. It MUST be raising. Isacenko & Schädlich (1970): “If an utterance contains no other syntactic or lexical cue to identify it as a question […] then only the last […] rising tone-switch provides the necessary information to allow the hearer to identify it as a question" (p.32) (Cf. also Huddleston (1994) for English and Haeseryn et al. (1997) for Dutch)
=> known as ‘intonation question’ or ‘queclarativ’ (Sadock 1974)
Katherina sucht ‘ne Wonhung
H* (L)H* H- H%
(“Katherina searches for a flat”)
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
12
Q/S in GermanQ/S in German
BUT:
• Characteristic function of such questions: request for confirmation
(a) with regard to the assumed functional differences between final rises/falls in German questions
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
Q1: Is the final rise actually mandatory in German ‘intonation questions’, i.e. does it mean
‘question’ ?
(b) …and in view of the domain and the cues for ‘question’ in Neapolitan Italian
Q2: What role plays the prenuclear region?
Our study started from observations in naturalutterances
13
Q/S in GermanQ/S in GermanStatement
Yes/no Question
Ka the ri na sucht ‘ne Woh nung
Ka the ri na sucht ‘ne Woh nung?
H* L-L%H*
L+H* H-H%H*
+ steep+ convex
+ shallow+ concave
14
Q/S in GermanQ/S in GermanStatement
Yes/no Question
Intonation
Question?
Ka the ri na sucht ‘ne Woh nung
Ka the ri na sucht ‘ne Woh nung?
Ka the ri na sucht ‘ne Woh nung?
H* L-L%H*
L+H* H-H%H*
L*+H L-L%H*
+ steep+ convex
+ shallow+ concave
+ shallow+ concave
15
Experiment IExperiment I
• Similarly to Neapolitans, German listeners are able to identify Q/S sentences, i.e. asserting and questioning speech acts, well before hearing the ‘nucleus’• Differences in the prenuclear rise/fall cue questionhood independent of the presence of a terminal rise and interrogative syntax
+ early rise-fall alignment
+ convex fall
« Assertion »+ late rise-fall
alignment+ concave fall
« Questionhood »
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
16
Q
CorpusCorpus
• 5 intonation Bases: Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
(1)(2)(3)
H* H+L* L-%H* H* L-%H* L*+H L-%
+ early rise-fall alignment+ convex fall
+ late rise-fall alignment+ concave fall
- (1)-(3): nuclear pitch-accent difference with final fall. Known to signal meaning differences within statements, i.e. ‘settled’, ‘open’, ‘astonished’ (Niebuhr 2007; Grice & Baumann 2000; Kohler 1987)
- (4): tune with final rise. Known to signal questions. The contour preceding the terminal mouvement is constant for
(3) & (4) => clear Q/S difference - (3)+(5): difference in prenuclear region (H* alignment and
shape/alignment of subsequent fall) => yields Q/S difference ? - (4)+(5): difference in final fall vs. final rise with constant
preceding tune => yields Q/S difference ?
S
(4)
(5) H* L*+H L-%
H* L*+H H-%
• Natural Utterances: Katherina sucht ‘ne Wohnung
17
CorpusCorpus
• 5 intonation Bases:
• 3 tune fragments: short
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
Ka the ri na sucht ‘ne Woh nung?
(1)(2)(3)
H* H+L* L-%H* H* L-%H* L*+H L-%
+ early rise-fall alignment+ convex fall
+ late rise-fall alignment+ concave fallQ
S
(4)
(5) H* L*+H L-%
H* L*+H H-%
• Natural Utterances: Katherina sucht ‘ne Wohnung
18
CorpusCorpus• Natural Utterances: Katherina sucht ‘ne Wohnung
• 5 intonation Bases:
• 3 tune fragments: medium
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
Ka the ri na sucht ‘ne Woh nung?
(1)(2)(3)
(4)
(5)
H* H+L* L-%H* H* L-%H* L*+H L-%
H* L*+H L-%
+ early rise-fall alignment+ convex fall
+ late rise-fall alignment+ concave fall
H* L*+H H-%
S
Q
19
CorpusCorpus
• 5 intonation Bases:
• 3 tune fragments: long
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
Ka the ri na sucht ‘ne Woh nung?
(1)(2)(3)
(4)
(5)
H* H+L* L-%H* H* L-%H* L*+H L-%
H* L*+H L-%
+ early rise-fall alignment+ convex fall
+ late rise-fall alignment+ concave fall
H* L*+H H-%
S
Q
• Natural Utterances: Katherina sucht ‘ne Wohnung
20
MethodsMethods• Semantic differential task:
- Three 7-point scales aiming at the Q-S difference:
(1) astonished - not astonished; (2) questioning - not questioning; (3) uncertain - certain
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
3210-1-2-3uncertain certain
The speaker sounds…
21
MethodsMethods• Semantic differential task:
- Three 7-point scales aiming at the Q-S difference:
(1) astonished - not astonished; (2) questioning - not questioning; (3) uncertain - certain
• Procedure- One randomized block containing short and
medium tune fragments- Long sentences at the end of the session - 11 German subjects X 9 repetitions (1782 obs.)
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
22
MethodsMethods• Semantic differential task:
- Three 7-point scales aiming at the Q-S difference:
(1) astonished - not astonished; (2) questioning - not questioning; (3) uncertain - certain
• Procedure- One randomized block containing short and
medium tune fragments- Long sentences at the end of the session - 11 German subjects X 9 repetitions (1782 obs.)
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
• Statistical analysis: Linear mixed model- Fixed: Scales; Tune fragments- Random: Subjects- p < .05
23
ResultsResultsH* H+L* L-% H* H* L-%
H* L*+H L-%
* * * n.s. n.s.
(1) (2)
(3)
n.s.
* * *
• Judged as “assertive” already in the prenuclear region
• Assertiveness increases as the terminal fall is heard with H+L* (1), but NOT with H* (2)
• In (3), the L*+H accent conveys more astonishment
In line with Niebuhr (2007), stimuliwith early pren. rise/fall+convex fall:
24
ResultsResults
Stimuli with late prenuclear rise + concave fall :
• Judged as “questioning” already in the prenuclear region, i.e. independent of the presence the terminal rise
• Overall significant difference in responses between (1)-(2)-(3) vs. (4)-(5) base types for both short and long stimuli
• Adding the “nucleus” increases “questioning” in long stimuli
• Effects stronger in intonation base H-H% (4) than in L-L% (5)
H* L*+H H-% H* L*+H L-%(4) (5)
* * * * * *
25
DiscussionDiscussion
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
The intonational signalling of pragmatic functions of sentences in German is not bound to the “nucleus” and the subsequent final F0 pattern:
1. Questioning vs. assertive sentences are well discriminable when only the prenuclear accent region is left in the stimulus
This cannot depend on the phonological specification of the prenuclear accent -(H)* for the 5 intonation types- though phonetic factors such as speech rate or intensity might have affected listeners’ judgements
26
DiscussionDiscussion
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
The intonational signalling of pragmatic functions of sentences in German is not bound to the “nucleus” and the subsequent final F0 pattern:
1. Questioning vs. assertive sentences are well discriminable when only the prenuclear accent region is left in the stimulus
This cannot depend on the phonological specification of the prenuclear accent -(H)* for the 5 intonation types- though phonetic factors such as speech rate or intensity might have affected listeners’ judgements
2. Contrast in listeners’ judgement for stimuli (1)-(2)-(3)vs. (4)-(5) already in the “short” condition This might be due to the differences in the alignment of the prenuclear rise and/or the shape of the fall between the two intonation groups
27
DiscussionDiscussion3. Terminal falling intonations can signal a question, even when not marked syntactically
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
Stimuli with patterns (5), ie. containing a terminal fall but preceded by a late rise-fall/concave fall, shifted the judgements towards more “astonished”, “uncertain” and “questioning”
28
DiscussionDiscussion3. Terminal falling intonations can signal a question, even when not marked syntactically
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
Stimuli with patterns (5), ie. containing a terminal fall but preceded by a late rise-fall/concave fall, shifted the judgements towards more “astonished”, “uncertain” and “questioning”
4. The perception of intonation modality is improved when the nucleus is also available, especially in ‘questioning’ stimuli (4)-(5)
This suggests that prosodic cues in the prenuclear region are less stronger for Q than for S modality, so that listeners have to rely more on the nuclear pattern when perceiving Q (see also Petrone & D’Imperio, 2008 for Neapolitan)
29
Experiment IIExperiment IIGerman listeners capitalize on differences in the prenuclear F0 region in the perception of Q/S utterances
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
30
Experiment IIExperiment IIGerman listeners capitalize on differences in the prenuclear F0 region in the perception of Q/S utterances
+Q-Q
time
f0
Rise timing?
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
31
Experiment IIExperiment IIGerman listeners capitalize on differences in the prenuclear F0 region in the perception of Q/S utterances
+Q-Q
time
f0
+Q-Q
time
f0
Rise timing? Fall timing?
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
32
Experiment IIExperiment IIGerman listeners capitalize on differences in the prenuclear F0 region in the perception of Q/S utterances
+Q-Q
time
f0
+Q-Q
time
f0
Rise timing? Fall timing?
Fall slope?
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
time
f0
+Q -Q
33
Experiment IIExperiment IIGerman listeners capitalize on differences in the prenuclear F0 region in the perception of Q/S utterances
+Q-Q
time
f0
+Q-Q
time
f0
time
f0
-Q
+Q
Rise timing? Fall timing?
Fall slope? Fall shape?
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
time
f0
+Q -Q
34
CorpusCorpus• 2 base types: Resyntheses based on astonished
statement & intonation question (H * L*+H L-%)• F0 (nuclear): F0 peaks of the 2 base types merged
into a constant, intermediate pattern for all stimuli.• F0 (prenuclear): 2 rise alignments (early/late) X 4 fall
slopes (or fall alignments) X 3 fall shapes (linear/concave/convex)
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
time
Early peak Late peak80 ms
35
CorpusCorpus
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
time
Early peak Late peak
T1 T2 T3 T450 ms
• 2 base types: Resyntheses based on astonished statement & intonation question (H * L*+H L-%)
• F0 (nuclear): F0 peaks of the 2 base types merged into a constant, intermediate pattern for all stimuli.
• F0 (prenuclear): 2 rise alignments (early/late) X 4 fall slopes (or fall alignments) X 3 fall shapes (linear/concave/convex)
36
CorpusCorpus
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
time
Early peak Late peak
T1 T2 T3 T4
50 Hz
• 2 base types: Resyntheses based on astonished statement & intonation question (H * L*+H L-%)
• F0 (nuclear): F0 peaks of the 2 base types merged into a constant, intermediate pattern for all stimuli.
• F0 (prenuclear): 2 rise alignments (early/late) X 4 fall slopes (or fall alignments) X 3 fall shapes (linear/concave/convex)
(see D’Imperio & Cangemi, PAPI 2009)
37
MethodsMethods• Procedure
- Indirect identification test: “Does it match?”Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
Test sentence
Context
“Katherina wants to become a painter”
“Really? That’s a risky step”
YES = astonished statement
NO = intonational question
- 11 German listeners x 5 repetitions (2460 obs.)
38
MethodsMethods
• Statistical analysis: Generalized Mixed Model- Fixed: rise and fall alignment, slope, shape, base
type- Random: Subjects- p < .05
• Procedure- Indirect identification test: “Does it match?”
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
Test sentence
Context
“Katherina wants to become a painter”
“Really? That’s a risky step”
YES = astonished statement
NO = intonational question
- 11 German listeners x 5 repetitions (2460 obs.)
39
Results: Timing & SlopeResults: Timing & Slope
• Perception of ‘astonished statement’ decreases around the chance level as the end of the fall is shifted later
0.5
• This effect is stronger for earlier peak alignment at T3 & T4
• Effects of the slope indistinguishable from those of the timing• Small Base Type effect
* *
40
Results: ShapeResults: Shape
EarlyEarly Late Late
• Strong interaction shape by fall alignment in early peak: the perception of ‘astonished statement’ goes much below the chance level for concave shape stimuli at late fall alignments
0.5
• Small shape effects in late peak: Why?
* ** *
41
DiscussionDiscussion
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
German listeners exploit the F0 prenuclear region forQ/S perception, even in absence of a clear informationfrom the ‘nucleus’:
1. Alignment and dynamic cues distributed in the prenuclear F0 region seems to be at work when perceiving the Q/S contrast in GermanThe early fall alignment is a robust cue for ‘astonished statements’ perception. The late fall alignment is more ambiguous, and additional differences in the shape of the fall become crucial for ‘question’ perception.
Late + concave fall
Early fall S
Q
=
=
42
DiscussionDiscussion
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
2. The fall alignment/shape manipulation affected listeners’ judgment only when the prenuclear rise is earlyThis can be due to our manipulation: the shift from the early to the late alignment could have been too “far”. The shape difference is more salient when the rise is earlier
43
DiscussionDiscussion2. The fall alignment/shape manipulation affected
listeners’ judgment only when the prenuclear rise is earlyThis can be due to our manipulation: the shift from the early to the late alignment could have been too “far”. The shape difference is more salient when the rise is earlier
3. The base stimulus produce a small but significant
effect. This means that cues other than F0 (voice quality, speech rate, etc.) might have been exploited by listeners
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
44
ConclusionConclusion• Results from (Neapolitan and) German indicate
that the prenuclear F0 region is relevant in conveying pragmatic functions, and thus it should be taken into account by theories of intonational meaning
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
45
ConclusionConclusion• Results from (Neapolitan and) German indicate
that the prenuclear F0 region is relevant in conveying pragmatic functions, and thus it should be taken into account by theories of intonational meaning
• The influence of the prenuclear F0 region is accounted for by the interaction of multiple F0 dimensions (alignment, shape), thus suggesting that dynamic properties might help in interpreting linguistic information
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
46
ConclusionConclusion• Results from (Neapolitan and) German indicate
that the prenuclear F0 region is relevant in conveying pragmatic functions, and thus it should be taken into account by theories of intonational meaning
• The influence of the prenuclear F0 region is accounted for by the interaction of multiple F0 dimensions (alignment, shape), thus suggesting that dynamic properties might help in interpreting linguistic information
• Our results suggest that intonation meaning is defined by the contour as a whole : the interrelation between tones in a tune cannot be captured by a strict compositional approach
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
47
Grazie/Danke!
48
CorpusCorpus• 2 Base Types: Resynthesized stimuli from an
astonished statement & an intonational question (H * L*+H L-%)Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
Ka the ri na will ‘ne Ma le rin werden?
Ka the ri na will ‘ne Ma le rin werden
49
What´s a tune?What´s a tune?Tunes of utterances can be decomposed into sequenc-es of static tones (cf. AM model, Pierrehumbert,1980)or contour elements (KIM, Kohler 1991)
• No structural (only semantic) restrictions for com-binations of pitch accents + edge components
• For pitch accents: nuclear = prenuclearIntroduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
(Pierrehumbert, 1980)
For example: AM approach (Pierrehumbert 1980)
50
Meaning compositionalityMeaning compositionality
• Intonational meaning given by the independent contribution of individual tones
H* L- L%+ + = Statement
L* H- H%+ + = Yes/No question
English
H* =
L- =
Tunes Sentence mode
(Pierrehumbert & Hirschberg, 1990)
The accented item is instantiated in the openexpression to be added to Hearer’s mutual belief
The interpretation of the current ip does notdepend on that of subsequent ips
• « Nucleus » (nuclear accent + terminal edge components) is essential for conveying meaning
Prenuclear region?
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
51
Tones-F0 mappingTones-F0 mapping• In their phonetic manifestation tonal targets are
characterized by temporal alignment and F0 scaling
ShapeSlope Duration
L
H
L
H
L
H
L
H
L
H
L
H
• F0 between targets defined by interpolation rules
ATime
F0
M r
H
L
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
52
(LH*)
L
H*
La mam vuole ve de re la Dinama
(LH*)
L*
H
La mam vuolevede re la Dina?ma
Speaker OM
Tunes in Neapolitan ItalianTunes in Neapolitan Italian• Intonation contrast:Intonation contrast: AP-tone…AP-tone…
Tr. “The mom wants to see (the) Dina”
• Same phrasing & tonal composition but Same phrasing & tonal composition but different slope!!
Introduction
Experiment I
Discussion
Experiment II
Discussion
Summary & Conclusion
S
Q