entrainment on backchannel‐ preceding cuesrlevitan/papers/acl_talk.pdf · • does the way one...
TRANSCRIPT
EntrainmentonBackchannel‐PrecedingCues
RivkaLevitan Agus;nGravano JuliaHirschberg
1
Entrainment
'Aretheirheadsoff?'shoutedtheQueen.
'Theirheadsaregone, if itpleaseyourMajesty!'thesoldiersshoutedinreply.
'That's right!' shouted theQueen. 'Can youplaycroquet?’
'Yes!'shoutedAlice.'Come on, then!' roared the Queen, and Alice
joined the procession, wondering verymuchwhatwouldhappennext.
‐‐Alice’sAdventuresinWonderland
2
Entrainment
'Aretheirheadsoff?'shoutedtheQueen.
'Theirheadsaregone, if itpleaseyourMajesty!'thesoldiersshoutedinreply.
'That's right!' shouted theQueen. 'Can youplaycroquet?’
'Yes!'shoutedAlice.'Come on, then!' roared the Queen, and Alice
joined the procession, wondering verymuchwhatwouldhappennext.
‐‐Alice’sAdventuresinWonderland
3
Backchannels
Speaker A: All right so I have a- a a nail on top Speaker B: okay Speaker A: with an owl in the lower left.
Backchannel:ashortsegmentofspeechuWeredbyalistenertoconveythatsheispayingaWenXon.
4
A.Gravano&J.Hirschberg,2009Backchannel‐invi,ngcuesintask‐orienteddialogue
T1: speech <pause> speech <pause> speech T2: <backchannel>
5
A.Gravano&J.Hirschberg,2009Backchannel‐invi,ngcuesintask‐orienteddialogue
T1: IPU <pause> IPU <pause> IPU T2: <backchannel>
6
Entrainmentonbackchannel‐precedingcues
• DifferentspeakersdisplaydifferentcombinaXonsofcues.
• Differentspeakersmayrealizeacueindifferentways.
7
Entrainmentonbackchannel‐precedingcues
• DifferentspeakersdisplaydifferentcombinaXonsofcues.
• Differentspeakersmayrealizeacueindifferentways.
• Dospeakerpairsentrainonbackchannel‐precedingcues?
8
Entrainmentonbackchannel‐precedingcues
• Threemeasuresofbackchannelcueentrainment:– Measure1:Cuesets– Measure2:CuerealizaXon– Measure3:LocalcuerealizaXon
9
10
TheColumbiaGamesCorpus• Twelvespontaneoustask‐orientedconversaXons• IneachconversaXon,twopeople,separatedbyacurtain,playedcomputergames.• ThirteenpeopleparXcipated;elevenparXcipatedintwosessionseach.• AllfeaturesextractedautomaXcallyinPraat.
Measure1:Similarcuesets
• Dospeakersusesimilarsetsofcues?
Speaker1RisingintonaXonHigherpitch
LongerduraXon
Speaker2RisingintonaXonHigherintensityLongerduraXon
Changeinvoicequality____________________________________
2cuesincommon
11
Measure1:Similarcuesets
• Dospeakersusesimilarsetsofcues?
Speaker1RisingintonaXonHigherpitch
LongerduraXon
Speaker2RisingintonaXonHigherintensityLongerduraXon
Changeinvoicequality____________________________________
2cuesincommon
12
Measure1:Similarcuesets
• Dospeakersusesimilarsetsofcues?
Yes(pairedt‐test,p<0.05)
Speaker1RisingintonaXonHigherpitch
LongerduraXon
Speaker2RisingintonaXonHigherintensityLongerduraXon
Changeinvoicequality____________________________________
2cuesincommon
13
Measure2:CuerealizaXon
• Dospeakersrealizetheircuesinsimilarways?
Foranyfeaturefmodelingcuec:
thissessionexhibitsentrainmentonc.
14
€
d1,2f < min(d1,*
f , d*,2f )
Measure2:CuerealizaXon
15
Measure2:CuerealizaXon
16
Measure2:CuerealizaXon
17
Measure2:CuerealizaXon
18
Measure2:CuerealizaXon
19
Measure2:CuerealizaXon
20
Measure2:CuerealizaXon
• Pitch,intensityandvoicequalityshowsignificantlysmallerdifferencesbetweenpartnersthanbetweennon‐partneredpairs.
21
Measure2:CuerealizaXon
• Pitch,intensityandvoicequalityshowsignificantlysmallerdifferencesbetweenpartnersthanbetweennon‐partneredpairs.
• Convergence:ThedifferencesbetweenpartnersinintensityandpitcharesmallerinthelasthalfofaconversaXonthanthefirst.
22
Measure3:Localentrainment
• Doesthewayonespeakerinvitesabackchannelaffecthowherpartnerinviteshisnextbackchannel?
23
IPU backchannel IPU backchannel
IPU backchannel IPU backchannel
IPU backchannel IPU backchannel
Measure3:Localentrainment
• Doesthewayonespeakerinvitesabackchannelaffecthowherpartnerinviteshisnextbackchannel?
24
IPU
IPU
IPU
Pearson’scorrelaXon
IPU
IPU
IPU
Measure3:Localentrainment
• Doesthewayonespeakerinvitesabackchannelaffecthowherpartnerinviteshisnextbackchannel?
25
IPU
IPU
IPU
Pearson’scorrelaXon
IPU
IPU
IPU
SignificantcorrelaXonsformeanpitchandintensity(r=0.3).
Entrainmentonbackchannel‐precedingcues
• Threemeasuresofbackchannelcueentrainment:– Measure1:Cuesets– Measure2:CuerealizaXon– Measure3:LocalcuerealizaXon
26
Entrainmentonbackchannel‐precedingcues
• Threemeasuresofbackchannelcueentrainment:– Measure1:Cuesets✓– Measure2:CuerealizaXon– Measure3:LocalcuerealizaXon
27
Entrainmentonbackchannel‐precedingcues
• Threemeasuresofbackchannelcueentrainment:– Measure1:Cuesets✓– Measure2:CuerealizaXon✓– Measure3:LocalcuerealizaXon
28
Entrainmentonbackchannel‐precedingcues
• Threemeasuresofbackchannelcueentrainment:– Measure1:Cuesets✓– Measure2:CuerealizaXon✓– Measure3:LocalcuerealizaXon✓
29
AssociaXonwithdialoguesuccess
• Threemeasuresofdialoguesuccess:– Score– ProporXonofinterrupXonsS1: <unfinished speech> S2: <speech>
– MeanbackchannellatencyS1: <speech> S2: <pause> <backchannel>
30
AssociaXonwithdialoguesuccess
31
AssociaXonwithdialoguesuccess
32
AssociaXonwithdialoguesuccess
33
Conclusions
Speakersusesimilarsetsofcues. Theyrealizethecuesinsimilarways.
Thisentrainmentislocalaswellasglobal.
ThisentrainmentisposiXvelyassociatedwithtasksuccessandnegaXvelyassociatedwithpoorcoordinaXon.
34
FutureWork
• Whoentrainstowhom?• WhatistherelaXonshipbetweendifferenttypesofentrainment?
• Isthisentrainmentassociatedwithmoreperceptualmeasuresoftasksuccess?
35
Thankyou!