ryan kilgore mark chignell university of toronto ibm cas, kmdi 03 | 22 | 06

Ryan KilgoreMark ChignellUniversity of Toronto IBM CAS, KMDI

03 | 22 | 06

seeing unfamiliar voicesdoes visualization of spatial position enhance voice identification?

Kilgore & Chignell, Seeing Unfamiliar Voices | HFT 2006

• Voice collaboration and spatial audio

• Visualizing audio spaces

• Experimental methodology

• Results

• Discussion

presentation overview

02 | 19

• Traditional methods of synchronous communication do not adequately support large groups

• Monaural audio, lack of visual feedback, and poor audio quality make it difficult to determine:

• Who is present?

• Who is speaking?

• What is being said?

problems with voice collaboration

03 | 19Kilgore & Chignell, Seeing Unfamiliar Voices | HFT 2006

spatial audio | overview

Free-Field Acoustics

S1 S2

S

Virtual Acoustics

S1 S2


spatial audio | benefits (1 of 2) • Reduction in masking; facilitation of auditory scene

analysis (Bregman, 1990; Gilkey & Anderson, 1997)

• Increased speech intelligibility in noisy environments (Ericson & McKinley, 1997)

• Increased speech intelligibility in multi-talker listening tasks (Drullman & Bronkhorst, 2000; Abouchacra, 2001; Bolia, 2001)


• Distinct voice locations aid in cognition of audio conference events (Baldis, 2001; Kilgore et al, 2003)

• Significantly preferred to traditional, monaural voice presentation

• Reduced perception of attention required for speaker identification task

• Increased speaker identification performance, particularly for ‘personalized’ audio spaces

spatial audio | benefits (2 of 2)


Vocal Village interface


• Early Vocal Village field trials indicated users want GUI for monitoring and controlling audio space

• Participants in audio-only field trials have highlighted the difficulty of knowing who was present in the audio space (Singer et al, 1999)

• Visual modality can convey awareness-supporting information parallel to audio communication

Will increased awareness of voice locations aid listeners in learning to identify completely unfamiliar voices?

visualization | audio spaces


visualization | previous studies

• Spatially arranged photos of speakers showed no performance benefits but preference (Baldis, 2001)

• Graphic insert w/ voice names and locations showed no benefit to voice identification in an ATC task (MacDonald, 2002)

• HOWEVER: These studies used familiar collaborators, or were limited to only four voices


Determine if visual representation of voice locations will aid listeners in learning to recognize voices that are completely unfamiliar

Dependent variables:

• Accuracy and response time for voice identification task

• Confidence in voice identification task performance

• Mental workload (NASA-TLX) (Hart & Staveland, 1998)

experiment | overview


Ryan Kilgore

e.g. Does adding a picture of the spatial audio environment result in benefits in addition to those of just using spatial audio

• Modified Coordinate Response Measure (CRM) listening task (Bolia et al, 2000)

“Ready [call sign], go to [color] [number] now”

• 4 male, 4 female voices

• Response to target with color, number, speaker’s name

• 27 Participants, no voice training

• Provided performance feedback (w/ correct answer)

experiment | methodology (1 of 2)


• Two independent variables:

• 4 experimental blocks

• 40 stimuli per block (160 total)

experiment | methodology (2 of 2)

Presentation Format

• Mono, Spatial, Spatial+Visual• Varied between subjects

Number of Voices

• 4 Voices, 8 Voices• Varied between subjects


experiment | stimuli


experiment 3 | results (1 of 3)

Correct Voice Identifications by Experimental Block

Experimental BlockF[3, 30] = 61.15, p < .001

Number of VoicesF[1, 30] = 68.21, p < .001

FormatF[2, 30] = 1.39, p = .27

Number × FormatF < 1

4 Voices

Block 1 Block 2 Block 3 Block 4

Per

cent

Cor

rect

Voi

ce I

dent

ifica

tions

by

Blo

ck

0.0

0.2

0.4

0.6

0.8

1.0

8 Voices

Block 1 Block 2 Block 3 Block 40.0

0.2

0.4

0.6

0.8

1.0

MonoSpatialSpatial+Visual


4 Voices

Block 1 Block 2 Block 3 Block 4

Pe

rcen

t C

orr

ect

Vo

ice

Iden

tific

atio

ns b

y B

lock

0.0

0.2

0.4

0.6

0.8

1.0

8 Voices

Block 1 Block 2 Block 3 Block 40.0

0.2

0.4

0.6

0.8

1.0



Kilgore & Chignell, Seeing Unfamiliar Voices | HFT 2006 15 | 19


Removed data for low-learning participants:

• Excluded subjects that showed no improvement in voice identification over duration of experiment

• 2 Mono participants removed3 Spatial participants removed3 Spatial+Visual participants removed



Correct Voice Identifications(low-learning subjects removed)

4V: Format × BlockF < 1

8: Format × BlockF[2,10] = 5.43, p = .025

4 Voices

Mono Spatial Spatial+Visual

% C

orr

ect R

esp

on

ses

0.0

0.2

0.4

0.6

0.8

1.0

8 Voices

Mono Spatial Spatial+Visual0.0

0.2

0.4

0.6

0.8

1.0

Trials 1-120Trials 121-160


4 Voices

Mono Spatial Spatial+Visual

% C

orr

ect

Re

spo

nse

s

0.0

0.2

0.4

0.6

0.8

1.0

8 Voices

Mono Spatial Spatial+Visual0.0

0.2

0.4

0.6

0.8

1.0




discussion

• Simple visual representation of voice locations improves the learning of completely unfamiliar voices in larger audio spaces (8 talkers)

• Visualizations continue to support identification as voices become increasingly familiar

• Spatial presentation of voice, coupled with low-cost visualization methods, may be particularly useful in supporting:

• Large collaborative groups• Groups with limited familiarity


current work – visual scale

19 | 19

questions?

fin

references (1 of 2)

Abouchacra, K., (2001). Binaural Helmet: Improving speech recognition in noise with spatialized sound. Human Factors, 43 (4), 584.

Baldis, Jessica., (2001). Effects of spatial audio on memory, comprehension, and preference during desktop conferences. Proceedings of the SIGCHI conference on human factors in computing systems, Vol. 3, 166-173.

Bregman, A. S., (1990). Auditory Scene Analysis. Cambridge: MIT Press.

Bolia, Robert S., W. Todd Nelson, Mark A. Ericson, and Brian D. Simpson, (2000). A speech

corpus for multitalker communication research. J. Acoust. Soc. Am. 107 (2) 1065-1066.

Bolia, R., (2001). Asymmetric performance in the cocktail party effect: implications for the design of Spatial Audio Displays. Human Factors, 43 (2), 208.

Drullman, Rob and Adelbert W. Bronkhorst, (2000). Multichannel speech intelligibility and talker recognition using monaural, binaural, and three-dimensional auditory presentation. J. Acoust. Soc. Am. 107(4), 2224-2235.

Ericson, M.A., and R. L. McKinley, (1997). The intelligibility of multiple talkers separated spatially in noise. In Binaural and Spatial Hearing in Real and Virtual Environments, Gilkey, Robert H. and Timothy R. Anderson Eds., NJ, Lawrence Erlbaum Associates, 701-724.

ref1

ref2

Gilkey, Robert H. and Timothy R. Anderson Eds., (1997). Binaural and Spatial Hearing in Real and Virtual Environments, New Jersey: Lawrence Erlbaum Associates.

Hart, S.G., and Staveland, L.E., (1988). Development of the NASA-TLX (Task Load Index): results of empirical and theoretical research. In P.A. Hancock, and N. Meshkati (Eds.), Human Mental Workload. North Holland: Elsevier Science Publishers, 139-183.

Kilgore, Ryan M., Mark Chignell and Paul W. Smith, (2003). Spatialized audioconferencing: what are the benefits? Proceedings of the 2003 conference of the Centre for Advanced Studies Conference on Collaborative Research, 111-120.

MacDonald, J. (2002). Intelligibility of speech in a virtual 3-D environment. Human Factors, 44(2), 272.

Singer, A., Hindus, D., Stifelman, L., and S. White, (1999),.Tangible progress: less is more in Somewire audio spaces, Proceedings of the SIGCHI conference on

human factors in computing systems, 104-111.

references (2 of 2)

spatial audio | explanation

• Perception of relative differences between signals picked up by the left and right ears

• Allows people with binaural hearing to locate sound sources in three-dimensional space

• Product of multiple interaural cues: IID, ITD, HRTFs

experimental interface

Vocal Village interface

experiment 4 | visual stimuli (1 of 2)

experiment 4 | visual stimuli (2 of 2)

25.5”

10”

2”

experiment 4 | audio stimuli

Full-Scale (24°) Half-Scale (12°)

thanks

• UofT Interactive Media Lab and the Vocal Village development team

• My committee (Mark Chignell, Greg Jamieson, Ron Baecker)

• IBM Centre for Advanced Studies, Toronto

• Knowledge Media Design Institute

ryan kilgore mark chignell university of toronto ibm cas, kmdi 03 | 22 | 06

Documents

unfamiliar voices hft

spatial audio benefits

spatial audio overview04

kilgore et

audio space participants

personalized audio spaces06

large groupsmonaural

poor audio quality