![Page 1: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/1.jpg)
Experiment by Sekuler et al.Balls moving without sound appeared to move past each otherBalls with an added “click” appeared to collide
Interactions between sight and sound
![Page 2: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/2.jpg)
http://shamslab.psych.ucla.edu/demos/
Sound-induced Illusory Flashing
Auditory clicks can influence perceived number of visual flashes.
![Page 3: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/3.jpg)
Using auditory stimuli to replace sight
http://www.senderogroup.com/
![Page 4: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/4.jpg)
Chapter 13: Speech Perception
![Page 5: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/5.jpg)
Overview of Questions
• Can computers perceive speech as well as humans?• Why does an unfamiliar foreign language often sound like a continuous
stream of sound, with no breaks between words?
• Does each word that we hear have a unique pattern of air pressure changes associated with it?
• Are there specific areas in the brain that are responsible for perceiving speech?
![Page 6: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/6.jpg)
Can computers perceive speech as well as humans?
What I talk now it will start adding text into my Microsoft PowerPoint document .
Sunday’s that’s pretty well in a quiet room with simple words in my own voice speaking pretty slowly.
The button often a silly mistakes to specify start speaking more more quickly to start using very complicated words like phenomenal the psychophysics on auditory cortex what a fine when stream January the cuts psychophysics in a while I probably say psychophysics lot more than most people doing an anxiety why he conceded that his media scene gloves (a) well into the wind is better than my dad in the ring was later stages 10 point 463 impressive but now it’s doing it since hitting at all since when is the loss was doing some tenants find In a second save the quick brown fox typically do not some tenants find In the second save the queen brown fox typically do not something that’s fine in the second save the queen brown fox site next line In the second save the queen brown fox and it’s like not having a meeting of the amount of space at the time the delay and of all citizens of the game is morning I had a lot of my life and he started the year ended sounds and started making of words on the Internet has been at least in house on a one-on and so the
![Page 7: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/7.jpg)
The Speech Stimulus• Phoneme – the smallest unit of speech that changes meaning in a word
– In English there are 47 phonemes:
• 13 major vowel sounds
• 24 major consonant sounds
– Number of phonemes in other languages varied—11 in Hawaiian and 60 in some African dialects
![Page 8: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/8.jpg)
The Acoustic Signal• Produced by air that is pushed up from the lungs through the
vocal cords and into the vocal tract• Vowels are produced by vibration of the vocal cords and
changes in the shape of the vocal tract
![Page 9: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/9.jpg)
Time
Freq
uenc
y (H
z)
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90
500
1000
1500
2000
2500
3000
The Sound Spectrogram
‘frequency sweep’
![Page 10: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/10.jpg)
The Sound Spectrogram
my (lame) attempt at a ‘frequency sweep’
Time
Freq
uenc
y (H
z)
0.2 0.4 0.6 0.8 10
500
1000
1500
2000
2500
3000
Resonant frequencies, or ‘formants’
![Page 11: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/11.jpg)
Time
Freq
uenc
y (H
z)
0.2 0.4 0.6 0.8 10
500
1000
1500
2000
2500
3000
Vowel sounds are caused by a resonant frequency of the vocal cords and produce peaks in pressure at a number of frequencies called formants
The first formant has the lowest frequency, the second has the next highest, etc.
‘ah’
![Page 12: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/12.jpg)
The Acoustic Signal
• Consonants are produced by a constriction of the vocal tract
Time
Freq
uenc
y (H
z)
0 0.2 0.4 0.6 0.8 10
500
1000
1500
2000
2500
3000‘hit’
![Page 13: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/13.jpg)
Time
Freq
uenc
y (H
z)
0.2 0.4 0.6 0.8 10
500
1000
1500
2000
2500
3000
‘chew it’
The segmentation problem: There are no physical breaks in the continuous acoustic signal.
![Page 14: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/14.jpg)
0
500
1000
1500
2000
2500
3000
3500
4000
The segmentation problem
![Page 15: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/15.jpg)
The segmentation problem
![Page 16: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/16.jpg)
Time
Fre
que
ncy
(Hz)
0.2 0.4 0.6 0.8 10
100
200
300
400
500
600
700
800
The variability problem
There is no simple correspondence between the acoustic signal and individual phonemes:
Coarticulation - overlap between articulation of neighboring phonemes: ‘d’ looks different depending on the vowel sound that follows it.
/di/ /du/
![Page 17: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/17.jpg)
The variability problem
There is no simple correspondence between the acoustic signal and individual phonemes:
1) Coarticulation - overlap between articulation of neighboring phonemes
![Page 18: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/18.jpg)
Time
Freq
uenc
y (H
z)
0.5 1 1.5 20
500
1000
1500
2000
2500
3000
Time
Freq
uenc
y (H
z)
0.5 1 1.5 20
500
1000
1500
2000
2500
3000
‘Ollie come here’ (Ione) ‘Ollie come here’ (Geoff)
2) Variability across different speakers:
Speakers differ in pitch, accent, speed in speaking, and pronunciation
The variability problem
![Page 19: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/19.jpg)
The variability problem
3) Different pronunciations have the same meaning, but very different spectrograms
![Page 20: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/20.jpg)
0.2 0.4 0.6 0.8 10
500
1000
1500
2000
2500
3000
Time
Freq
uenc
y (H
z)
0.2 0.4 0.6 0.8 10
500
1000
1500
2000
2500
3000
‘hello’ (Ione) ‘hello’ (Geoff)
But there are some ‘invariances’ in speech perception.
These spectrograms look similar.
![Page 21: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/21.jpg)
Invariant acoustic cues:
Some features of phonemes remain constant
Short-term spectrograms are used to investigate invariant acoustic cues.
Sequence of short-term spectra can be combined to create a running spectral display.
From these displays, there have been some invariant cues discovered
![Page 22: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/22.jpg)
Categorical Perception
• This occurs when a wide range of acoustic cues results in the perception of a limited number of sound categories
• An example of this comes from experiments on voice onset time (VOT) - time delay between when a sound starts and when voicing begins
– Stimuli are da (VOT of 17ms) and ta (VOT of 91ms)
![Page 23: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/23.jpg)
Time
Freq
uenc
y (H
z)
0.2 0.4 0.6 0.8 1 1.2 1.40
500
1000
1500
2000
2500
3000‘too’ ‘doo’
Voice onset time (VOT)
Delay between when the sound begins and the onset of vocal cords.
Distinguishes between ‘ta’ vs. ‘da’, and ‘pa’ vs. ‘pa’.
![Page 24: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/24.jpg)
![Page 25: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/25.jpg)
‘Categorical perception’
Despite the continuous variation of VOT, we only hear one phoneme or the other.
![Page 26: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/26.jpg)
![Page 27: Interactions between sight and soundcourses.washington.edu/psy333/lecture_pdfs/Week9_Day4.pdf · Experiment by Sekuler et al. Balls moving without sound appeared to move past each](https://reader036.vdocument.in/reader036/viewer/2022070920/5fb957a5765ef826747d0a4e/html5/thumbnails/27.jpg)
Cognitive Dimensions of Speech Perception
• Top-down processing, including knowledge a listener has about a language, affects perception of the incoming speech stimulus
• Segmentation is affected by context and meaning
– I scream you scream we all scream for ice cream
I screamed you screen we all screen for high screen