introduction to spatial audio
TRANSCRIPT
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 1/25
Introduction to Spatial Audio
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 2/25
Course goals
• Sound is an extremely important part of life for
humans – Communication
– Sense-making and sensing
– Geometric perception – Pleasure
– Survival
• As computing, sound sensors, loudspeakers
become ubiquitous, significant opportunities for
computer science/engineering/audio-folks
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 3/25
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 4/25
Digital Signal Processing
• Core course in EE departments, and
increasingly in other areas• Representation and manipulation of
signals using families of functions – Fourier series, wavelets, and others
– Fast algorithms
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 5/25
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 6/25
Spatial Audio• Sound is usually created by relatively compact
sources that occupy a small region
• It scatters off various objects in the world and is
perceived/recognized after human/computerauditory processing
• Humans draw various percepts from the
processed sound – Labels applied after processing• Direction, location, ambience, quality, near, far, …
• Computers can also process sound anddetermine – Direction, location, signal-to-noise ratios, …
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 7/25
Source Localization
• Obtain sound source location
• Obtain direction of the source
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 8/25
Course Administrative Details
• Prerequisites: The course will be somewhatmathematical, and you should be comfortable or willing
to work with some differential equations and do someMatlab programming.
• Exams: There will be a mid-term and a final exam. In
addition there will be homework that either involves shortproblem sets, or require reading of research papers.
• Credit: For computer science students, the class willcount as a PhD qualifying course, and a MS qualifying
course in Visual and Geometric Computing. The MScomp grades will be based on the mid-term and finalexam.
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 9/25
Outline A survey of the field and applications
Some basic principles of physical acoustics;
Notions of frequency, the Fourier transform, and representations for analysis;
An introduction to the human auditory system
Review of basic signal processing
Source localization and beamforming with arrays of microphones.Human spatial hearing: The physical and psychoacoustical basis of sound
localization and space perception.
Room acoustics: sound propagation in rooms. Modeling. The influence of shortand long term reverberation.
Head related transfer functionsModeling Room impulse responses and head related impulse responses.
An introduction to commercial systems for surround sound and spatial audio.
Emerging Spatial Audio Playback systems: Wave Field Synthesis. Ambisonics.
Research systems being developed at the University of Maryland.Selected Research topics
A i l
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 10/25
Array signal
processing• Use an array of
microphones/hydrophones tolocalize and recognize sound
sources – Sonar arrays during the cold
war
– Hunt for Red October (Tom
Clancy)
• Array signal processingimportant in many areas,
including for sound captureby PCs and speakerphones,antenna design for cell-phones, bluetooth etc.
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 11/25
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 12/25
PONG N G
SIGGRAPH 2000
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 13/25
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 14/25
Sniper detection
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 15/25
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 16/25
Speaker Arrays• Design
speakers tocreate virtualreality
• Designspeakers tocreate sounddirected at
points• 5.1, 7.1. 42.x
systems
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 17/25
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 18/25
Games and Virtual Reality
• Create spatially
distributed sounds• Artificial
environments• Capture spatial
sound for analysis
and later playback
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 19/25
Human Computer Interaction
• Sounds to signify events
• Spatial presentation of data
• Hands free speech capture
• Speaker localization
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 20/25
Auditory Prostheses
• Hearing aids
• Cochlear implants
• Auditory Displays
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 21/25
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 22/25
Biological tools
• Many problems appear insoluble …
however humans easily solve them andprovide a proof of existence of solutions
• Understand how human system does
things• Cochlear signal processing, head related
transfer functions,• Psychophysics: Capability of the human
system, Just noticeable differences, …
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 23/25
Head Related Transfer Functions
• Scattering of sound off humans
• Research area I am involved in, and willbe somewhat over-represented in this
course.
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 24/25
How do we perceive sound location?• Initial idea: Measure attributes of
received sound at the two ears• Compare sound received at two ears
– Interaural Level Differences (ILD)
– Interaural Time Differences (ITD)• Surfaces of constant Time Delay:
| x-x L| -|x-x R | = c δ t – hyperboloids of revolution
– Delays same for on cone-of-confusion
• Level differences vanishingly small
• Other mechanisms necessary to explain
– Scattering of sound• Off our bodies
• Off the environment
– Purposive Motion
HEAD
Source
Left ear Right ear
8/13/2019 Introduction to Spatial Audio
http://slidepdf.com/reader/full/introduction-to-spatial-audio 25/25