audios dk

7/25/2019 Audios Dk

1/65

Audio SDK Documentation

Version 1.0.0-Beta

7/25/2019 Audios Dk

2/65

Introduction | Audio

|

Copyrights and Trademarks

2015 Oculus VR, LLC. All Rights Reserved.

OCULUS VR, OCULUS, and RIFT are trademarks of Oculus VR, LLC. (C) Oculus VR, LLC. All rights reserved.BLUETOOTH is a registered trademark of Bluetooth SIG, Inc. All other trademarks are the property of theirrespective owners. Certain materials included in this publication are reprinted with the permission of the

copyright holder.

7/25/2019 Audios Dk

3/65

Audio | Conte

Contents

Introduction to Virtual Reality Audio................................................................... 5Overview............................................................................................................................................................ 5Localization and the Human Auditory System.................................................................................................. 5

Directional Localization................................................................................................................................ 6Distance Localization.................................................................................................................................... 8

3D Audio Spatialization..................................................................................................................................... 9Directional Spatialization with Head-Related Transfer Functions (HRTFs)................................................. 10Distance Modeling..................................................................................................................................... 11

Listening Devices............................................................................................................................................. 11Environmental Modeling.................................................................................................................................. 15Sound Design for Spatialization...................................................................................................................... 17Mixing Scenes for Virtual Reality..................................................................................................................... 18VR Audio Glossary........................................................................................................................................... 20

Oculus Audio SDK Guide.................................................................................. 22SDK Contents and Features............................................................................................................................ 22Requirements................................................................................................................................................... 22Features............................................................................................................................................................ 23

Supported Features.................................................................................................................................... 23Unsupported Features................................................................................................................................ 23

Sound Transport Time..................................................................................................................................... 24Pitfalls and Workarounds................................................................................................................................. 24Platform Notes................................................................................................................................................. 25Middleware Support........................................................................................................................................ 25Oculus Hardware Capabilities......................................................................................................................... 25

Oculus Native Spatializer for Unity....................................................................27Overview.......................................................................................................................................................... 27Requirements and Setup................................................................................................................................. 27Exploring Oculus Native Spatializer with the Sample Scene.......................................................................... 29Applying Spatialization.................................................................................................................................... 30

Legacy Oculus Spatializer for Unity Integration Guide......................................35Overview.......................................................................................................................................................... 35Reducing Android Latency: OpenSL Low-Latency Driver............................................................................... 35Installing into Unity.......................................................................................................................................... 36How to Use the Oculus Spatializer in Unity.................................................................................................... 36OSPManager.................................................................................................................................................... 37OSPAudioSource.............................................................................................................................................. 38

OSP VersionMigration in Unity...................................................................................................................... 40

Oculus Spatializer for Wwise Integration Guide................................................41Overview.......................................................................................................................................................... 41Installing into Wwise Tool............................................................................................................................... 41Installing into the Wwise Unity Integration..................................................................................................... 42How to Use the Oculus Spatializer in Wwise.................................................................................................. 42Global Properties............................................................................................................................................. 44Sound Properties............................................................................................................................................. 45Integrating the Oculus Spatializer................................................................................................................... 47

7/25/2019 Audios Dk

4/65

Contents | Audio

OSP Version Migration in Wwise.................................................................................................................... 48

Oculus Spatializer for FMOD Integration Guide...............................................49Overview.......................................................................................................................................................... 49How to Use in FMOD Studio.......................................................................................................................... 49Notes and Best Practices................................................................................................................................ 50Integrating the Oculus Spatializer in the Application..................................................................................... 51

Installing with the FMOD Studio Unity Integration......................................................................................... 53OSP Version Migration in FMOD.................................................................................................................... 53

Oculus Spatializer VST Integration Guide......................................................... 54Overview.......................................................................................................................................................... 54Using the Plugin.............................................................................................................................................. 55Track Parameters............................................................................................................................................. 55DAW-Specific Notes........................................................................................................................................ 57Legal Notifications........................................................................................................................................... 57

Oculus Spatializer AAX Integration Guide........................................................ 59Overview.......................................................................................................................................................... 59Using the Plugin.............................................................................................................................................. 60Track Parameters............................................................................................................................................. 60

Release Notes.................................................................................................... 63Audio SDK 1.0 Release Notes........................................................................................................................ 63Audio SDK 0.11 Release Notes...................................................................................................................... 63Audio SDK 0.10 Release Notes...................................................................................................................... 64

7/25/2019 Audios Dk

5/65

Audio | Introduction to Virtual Reality Au

Introduction to Virtual Reality AudioWelcome to audio development for virtual reality!

OverviewThis document introduces fundamental concepts in audio development for virtual reality (VR) with an emphasison key factors that deserve development attention.

We hope to establish that audio is crucial for creating a persuasive VR experience. Because of the key role thataudio cues play in our cognitive perception of existing in space, any effort that development teams devote togetting it right will pay off in spades, as it will contribute powerfully to the user's sense of immersion. This is astrue for small- or mid-sized teams as it is for design houses perhaps even more so.

Audio has been a crucial part of the computer and video gaming experience since the advent of the first coin-op games, which filled arcades with bleeps, bloops, and digital explosions. Over time, the state of computeraudio has steadily improved, from simple wave generators (SID, 1983) to FM synthesis (AdLib, 1987), evolving

on to 8-bit mono samples (Amiga OCS, 1985; SoundBlaster, 1989) and 16-bit stereo samples (SoundBlasterPro), culminating in today's 5.1 surround sound systems on modern gaming consoles (XBox, 2001).

Since the development of 5.1 surround, little has changed. The fundamental technology of playing waveformsover speakers is the same, and the game playing environment is still primarily the living room or den with alarge television and speakers.

Virtual reality, however, is changing all this. Instead of a large environment with speakers, virtual reality bringsthe experience in close to the player via a head-mounted display (HMD) and headphones. The ability to trackthe user's head orientation and position significantly empowers audio technology.

Until now, the emphasis has typically been placed on the visual aspects of virtual reality (resolution, latency,tracking), but audio must now catch up in order to provide the greatest sense of presence possible.

This document discusses the challenges, opportunities, and solutions related to audio in VR, and how someof the techniques learned in traditional game development must be revisited and modified for VR. It is notintended to be a rigorous scientific study of the nature of acoustics, hearing and human auditory perception.Its intended audience includes anyone with an interest in audio and VR, including sound designers, artists, andprogrammers.

If you are interested in learning about these details in greater depth, we recommend searching the Web for thefollowing terms:

Head-Related Impulse Response

Head-Related Transfer Function

Sound Localization

Localization and the Human Auditory System

Consider the following: human beings have only two ears, but are able to locate sound sources within threedimensions. That shouldn't be possible if you were given a stereo recording and were asked to determine ifthe sound came from above or below the microphones, you would have no way to tell. If you can't do it from arecording, how can you do it in reality?

7/25/2019 Audios Dk

6/65

Introduction to Virtual Reality Audio | Audio

Humans rely on psychoacoustics and inference to localize sounds in three dimensions, attending to factors suchas timing, phase, level, and spectral modifications.

This section summarizes how humans localize sound. Later, we will apply that knowledge to solving thespatialization problem, and learn how developers can take a monophonic sound and transform its signal so thatit sounds like it comes from a specific point in space.

Directional LocalizationIn this section, we will look at the cues humans use to determine the direction to a sound source. The two keycomponents of localization are direction and distance.

Lateral

Laterally localizing a sound is the simplest type of localization, as one would expect. When a sound is closer tothe left, the left ear hears it before the right ear hears it, and it sounds louder. The closer to parity, the morecentered the sound, generally speaking.

There are, however, some interesting details. First, we may primarily localize a sound based on the delaybetween the sound's arrival in both ears, or interaural time difference(ITD); or, we may primarily localize asound based on the difference in the sound's volume level in both ears, or the interaural level difference(ILD).The localization technique we rely upon depends heavily on the frequency content of the signal.

Sounds below a certain frequency (anywhere from 500 to 800 Hz, depending on the source) are difficult todistinguish based on level differences. However, sounds in this frequency range have half wavelengths greaterthan the dimensions of a typical human head, allowing us to rely on timing information (or phase) between theears without confusion.

At the other extreme, sounds with frequencies above approximately 1500 Hz have half wavelengths smallerthan the typical head. Phase information is therefore no longer reliable for localizing the sound. At thesefrequencies, we rely on level differences caused by head shadowing, or the sound attenuation that results fromour heads obstructing the far ear (see figure below).

We also key on the difference in time of the signal's onset. When a sound is played, which ear hears it firstis a big part of determining its location. However, this only helps us localize short sounds with transients asopposed to continuous sounds.

There is a transitional zone between ~800 Hz and ~1500 Hz in which both level differences and timedifferences are used for localization.

7/25/2019 Audios Dk

7/65


Front/Back/Elevation

Front versus back localization is significantly more difficult than lateral localization. We cannot rely on timedifferences, since interaural time and/or level differences may be zero for a sound in front of or behind thelistener.

In the following figure we can see how sounds at locationsAand Bwould be indistinguishable from each othersince they are the same distance from both ears, giving identical level and time differences.

Humans rely on spectral modificationsof sounds caused by the head and body to resolve this ambiguity.These spectral modifications are filters and reflections of sound caused by the shape and size of the head,neck, shoulders, torso, and especially, by the outer ears (or pinnae). Because sounds originating from differentdirections interact with the geometry of our bodies differently, our brains use spectral modification to inferthe direction of origin. For example, sounds approaching from the front produce resonances created by theinterior of our pinnae, while sounds from the back are shadowed by our pinnae. Similarly, sounds from abovemay reflect off our shoulders, while sounds from below are shadowed by our torso and shoulders.

All of these reflections and shadowing effects combine to create a direction selective filter.

Head-Related Transfer Functions (HRTFs)

A direction selection filter can be encoded as a head-related transfer function(HRTF). The HRTF is thecornerstone for most modern 3D sound spatialization techniques. How we measure and create an HRTF isdescribed in more detail elsewhere in this document.

Head Motion

HRTFs by themselves may not be enough to localize a sound precisely, so we often rely on head motion toassist with localization. Simply turning our heads changes difficult front/back ambiguity problems into laterallocalization problems that we are better equipped to solve.

In the following figure sounds atAand Bare indistinguishable from each other based on level or timedifferences, since they are identical. By turning her head slightly, the listener alters the time and leveldifferences between ears, helping to disambiguate the location of the sound. D1is closer than D2, which is acue that the sound is to the left (and thus behind) the listener.

7/25/2019 Audios Dk

8/65

Introduction to Virtual Reality Audio | Audio

Likewise, cocking our heads can help disambiguate objects vertically. In the following figure, the listener cocksher head, which results in D1shortening and D2lengthening. This provides a cue that the object is above herhead instead of below it.

Distance Localization

ILD, ITD and HRTFs help us determine the direction to a sound source, but they give relatively sparse cues fordetermining the distance to a sound. To determine distance we use a combination of factors, including initialtime delay, ratio of direct sound to reverberant sound, and motion parallax.

Loudness

Loudness is the most obvious distance cue, but it can be misleading. If we lack a frame of reference, we can'tjudge how much the sound has diminished in volume from its source, and thus estimate a distance. Fortunately,we are familiar with many of the sound sources that we encounter daily, such as musical instruments, humanvoice, animals, vehicles, and so on, so we can predict these distances reasonably well.

For synthetic or unfamiliar sound sources, we have no such frame of reference, and we must rely on other cuesor relative volume changes to predict if a sound is approaching or receding.

7/25/2019 Audios Dk

9/65


Initial Time Delay

Initial time delaydescribes the interval between the direct sound and its first reflection. The longer this gap, thecloser we assume that we are to the sound source.

Anechoic (echoless) or open environments such as deserts may not generate appreciable reflections, whichmakes estimating distances more difficult.

Ratio of Direct Sound to Reverberation

In a reverberant environment there is a long, diffuse sound tail consisting of all the late echoes interacting witheach other, bouncing off surfaces, and slowly fading away. The more we hear of a direct sound in comparisonto the late reverberations, the closer we assume it is.

This property has been used by audio engineers for decades to move a musical instrument or vocalist to thefront or to the back of a song by adjusting the wet/dry mix of an artificial reverb.

Motion Parallax

Motion parallax (the apparent movement of a sound source through space) indicates distance, since nearbysounds typically exhibit a greater degree of parallax than far-away sounds. For example, a nearby insect cantraverse from the left to the right side of your head very quickly, but a distant airplane may take many secondsto do the same. As a consequence, if a sound source travels quickly relative to a stationary perspective, wetend to perceive that sound as coming from nearby.

High Frequency Attenuation

High frequencies attenuate faster than low frequencies, so over long distances we can infer a bit about distancebased on how attenuated those high frequencies are. This is often a little overstated in the literature, becausesounds must travel hundreds or thousands of feet before high frequencies are noticeably attenuated (i.e., wellabove 10 kHz). This is also affected by atmospheric conditions, such as temperature and humidity.

3D Audio Spatialization

The previous section discussed how humans localize the sources of sounds in three dimensions. We now invertthat and ask, Can we apply that information to fool people into thinking that a sound is coming from a specificpoint in space?

7/25/2019 Audios Dk

10/65

| Introduction to Virtual Reality Audio | Audio

The answer, thankfully, is yes, otherwise this would be a pretty short document. A big part of VR audio isspatialization: the ability to play a sound as if it is positioned at a specific point in three-dimensional space.Spatialization is a key aspect of presence because it provides powerful cues suggesting the user is in an actual3D environment, which contributes strongly to a sense of immersion.

As with localization, there are two key components to spatialization: direction and distance.

Directional Spatialization with Head-Related Transfer Functions(HRTFs)We know that sounds are transformed by our body and ear geometry differently depending on the incomingdirection. These different effects form the basis of HRTFs, which we use to localize a sound.

Capturing HRTFs

The most accurate method of HRTF capture is to take an individual, put a couple microphones in their ears(right outside the ear canal), place them in an anechoic chamber (i.e., an echoless environment), play sounds inthe chamber from every direction we care about, and record those sounds from the mics. We can then comparethe original sound with the captured sound and compute the HRTF that takes you from one to the other.

We have to do this for both ears, and we have to capture sounds from a sufficient number of discrete directionsto build a usable sample set.

But wait we have only captured HRTFs for a specific person. If our brains are conditioned to interpret theHRTFs of our own bodies, why would that work? Don't we have to go to a lab and capture a personalized HRTFset?

In a perfect world, yes, we'd all have custom HRTFs measured that match our own body and ear geometryprecisely, but inreality this isn't practical. While our HRTFs are personal, they are similar enough to each otherthat a generic reference set is adequate for most situations, especially when combined with head tracking.

Most HRTF-based spatialization implementations use one of a few publicly available data sets, captured eitherfrom a range of human test subjects or from a synthetic head model such as the KEMAR.

IRCAM Listen Database MIT KEMAR

CIPIC HRTF Database

ARI (Acoustics Research Institute) HRTF Database

Most HRTF databases do not have HRTFs in all directions. For example, there is often a large gap representingthe area beneath the subject's head, as it is difficult, if no impossible, to place a speaker one meter directlybelow an individual's head. Some HRTF databases are sparsely sampled, including HRTFs only every 5 or 15degrees.

Most implementations either snap to the nearest acquired HRTF (which exhibits audible discontinuities) or usesome method of HRTF interpolation. This is an ongoing area of research, but for VR applications on desktops, itis often adequate to find and use a sufficiently-dense data set.

Applying HRTFs

Given an HRTF set, if we know the direction we want a sound to appear to come from, we can selectan appropriate HRTF and apply it to the sound. This is usually done either in the form of a time-domainconvolution or an FFT/IFFT pair.

If you don't know what these are, don't worry - those details are only relevant if you are implementing the HRTFsystem yourself. Our discussion glosses over a lot of the implementation details (e.g., how we store an HRTF,
https://www.kfs.oeaw.ac.at/index.php?option=com_content&view=article&id=608:ari-hrtf-database&catid=158:resources-items&Itemid=606&lang=enhttp://recherche.ircam.fr/equipes/salles/listen/https://www.kfs.oeaw.ac.at/index.php?option=com_content&view=article&id=608:ari-hrtf-database&catid=158:resources-items&Itemid=606&lang=enhttp://interface.cipic.ucdavis.edu/sound/hrtf.htmlhttp://sound.media.mit.edu/resources/KEMAR.htmlhttp://recherche.ircam.fr/equipes/salles/listen/

7/25/2019 Audios Dk

11/65

Audio | Introduction to Virtual Reality Aud

how we use it when processing a sound). For our purposes, what matters is the high-level concept: we aresimply filtering an audio signal to make it sound like it's coming from a specific direction.

Since HRTFs take the listener's head geometry into account, it is important to use headphones whenperforming spatialization. Without headphones, you are effectively applying two HRTFs: the simulated one, andthe actual HRTF caused by the geometry of your body.

Head Tracking

Listeners instinctively use head motion to disambiguate and fix sound in space. If we take this ability away, ourcapacity to locate sounds in space is diminished, particularly with respect to elevation and front/back. Evenignoring localization, if we are unable to compensate for head motion, then sound reproduction is tenuous atbest. When a listener turns their head 45 degrees to the side, we must be able to reflect that in their auditoryenvironment, or the soundscape will ring false.

VR headsets such as the Rift provide the ability to track a listener's head orientation (and, sometimes, position).By providing this information to a sound package, we can project a sound in the listener's space, regardless oftheir head position.

This assumes that the listener is wearing headphones. It is possible to mimic this with a speaker array, but it issignificantly less reliable, more cumbersome, and more difficult to implement, and thus impractical for most VR

applications.

Distance Modeling

HRTFs help us identify a sound's direction, but they do not model our localization of distance. Humans useseveral factors to infer the distance to a sound source. These can be simulated with varying degrees of accuracyand cost in software:

Loudness, our most reliable cue, is trivial to model with simple attenuation based on distance between thesource and the listener.

Initial Time Delayis significantly harder to model, as it requires computing the early reflections for a givenset of geometry, along with that geometry's characteristics. This is both computationally expensive and

awkward to implement architecturally (specifically, sending world geometry to a lower level API is oftencomplex). Even so, several packages have made attempts at this, ranging from simple shoebox models toelaborate full scene geometric modeling.

Direct vs. Reverberant Sound(or, in audio production, the wet/dry mix) is a natural byproduct of anysystem that attempts to accurately model reflections and late reverberations. Unfortunately, such systemstend to be very expensive computationally. With ad hoc models based on artificial reverberators, the mixsetting can be adjusted in software, but these are strictly empirical models.

Motion Parallaxwe get for free, because it is a byproduct of the velocity of a sound source.

High Frequency Attenuationdue to air absorption is a minor effect, but it is also reasonably easy to modelby applying a simple low-pass filter, and by adjusting cutoff frequency and slope. In practice, HF attenuationis not very significant in comparison to the other distance cues.

Listening Devices

Traditionally, high quality audio reproduction has been the domain of multi-speaker systems, oftenaccompanied by one or more subwoofers. However, with the rise of online gaming and voice chat, manyplayers have transitioned to headsets (headphones with integrated microphones).

7/25/2019 Audios Dk

12/65


For modern VR, especially with head tracking and user movement, speaker arrays are an evolutionary deadend. Headphone audio will be the standard for VR into the future, as it provides better isolation, privacy,portability, and spatialization.

Headphones

Headphones offer several significant advantages over free-field speaker systems for virtual reality audio:

Acoustic isolationfrom the listener's environment enhance realism and immersion. Head trackingis greatly simplified.

HRTFsare more accurate since they don't suffer from the doubling down of HRTF effects (soundsmodified from the simulated HRTF, and again by the listener's actual body geometry).

Access to controlswhile wearing an HMD is far simpler when those controls are physically attached to thelistener.

Microphonesare ideally placed and subject to much less echo/feedback.

Headphones are available in a variety of types with various trade-offs:

Closed Back Headphones

As a general rule of thumb, closed back headphones offer the most isolation and bass response. However,the closed construction may lead to discomfort (due to heat and weight), and they tend to offer less accuratereproduction due to internal resonance. Also, if placed on or over the ear, they cause the pinnae to impactsound reproduction slightly.

7/25/2019 Audios Dk

13/65


While acoustic isolation can help with immersion, it cuts listeners off from their environment so they may beunable to hear others entering the room, cell phone ringing, doorbell, et cetera. Whether that is a good thingor not is up to the individual.

Open Back Headphones

Open back headphones are generally more accurate and comfortable than closed-back headphones, but theydo not isolate listeners from the exterior environment, and broadcast to the surrounding environment as well.These are suitable for quiet areas devoted to a VR experience, possibly in conjunction with a subwoofer.

As with closed back headphones, when placed on or over the ear, open back headphones allow the pinnae toimpact sound reproduction slightly.

Earbuds

Earbuds (such as those that ship with cell phones or portable music players) are cheap, lightweight, andvery portable, though they typically lack bass. Some models, such as Apple EarPods, have surprisingly good

frequency response, albeit with a steady roll off of bass frequencies. These are mostly ignored for spatialization.Most earbuds are poor at isolation.

7/25/2019 Audios Dk

14/65


In-Ear Monitors

In-ear monitors offer superior isolation from your environment, are very lightweight, and have excellentfrequency response over the entire range. They remove the effects of the listener's pinnae from sound (unlikeon-ear headphones). They have the downside of requiring insertion into the ear canal, which eliminates theeffects of the ear canal from sound reproduction entirely (since most HRTFs are captured with microphonesright outside the ear canal).

Impulse Responses

Headphones, like all transducers, impart their own characteristics on signals, and since HRTFs are frequencysensitive, removing the headphone character from the signal will usually be beneficial. This can beaccomplished by deconvolving the output signal with the headphone's impulse response.

External Speaker Systems

Until recently, the most common way to provide sound immersion was to surround the listener with speakers,

such as a Dolby 5.1 or 7.1 speaker configuration. While partially effective for a fixed and narrow sitting position,speaker array systems suffer from key drawbacks:

Imprecise imagingdue to panning over large portions of the listening area.

No elevation cues, sounds only appear in a 360 degree circle around the listener.

Assumption of immobile listener; in particular, no head tracking.

Room effectssuch as reverberation and reflections impact the reproduced sound.

Poor isolationmeans that outside sounds can intrude on the VR experience.

It is doubtful that multi-speaker configurations will be common or effective for home VR applications, thoughthey may be viable for dedicated commercial installations.

Bluetooth

Bluetooth has become a popular communication method of wireless audio broadcast. Unfortunately, modernBluetooth implementations often incur significant latency, sometimes as high as 500 milliseconds. As a result,Bluetooth technology is not recommended for audio output.

7/25/2019 Audios Dk

15/65


Environmental Modeling

HRTFs in conjunction with attenuation provide an anechoic model of three dimensional sound, which exhibitsstrong directional cues but tends to sound dry and artificial due to lacking room ambiance. To compensate forthis, we can add environmental modeling to mimic the acoustic effects of nearby geometry.

Reverberation and Reflections

As sounds travel through space, they reflect off of surfaces, creating a series of echoes. The initial distinctechoes (early reflections) help us determine the direction and distance to a sound. As these echoes propagate,diminish, and interact they create a late reverberation tail, which contributes to our sense of space.

We can model reverberation and reflection using several different methods.

Shoebox Model

Some 3D positional implementations layer simple shoebox room modeling on top of their HRTFimplementation. These consist of specifying the distance and reflectivity of six parallel walls (i.e., theshoebox) and sometimes the listener's position and orientation within that room as well. With that basicmodel, you can simulate early reflections from walls and late reverberation characteristics.

While far from perfect, it's much better than artificial or no reverberation.

Artificial Reverberations

7/25/2019 Audios Dk

16/65


Since modeling physical walls and late reverberations can quickly become computationally expensive,reverberation is often introduced via artificial, ad hoc methods such as those used in digital reverb units ofthe 80s and 90s. While less computationally intensive than physical models, they may also sound unrealistic,depending on the algorithm and implementation especially since they are unable to take the listener'sorientation into account.

Sampled Impulse Response Reverberation

Convolution reverbs sample the impulse responsefrom a specific real-world location such as a recordingstudio, stadium, or lecture hall. It can then be applied to a signal later, resulting in a signal that sounds as if itwere played back in that location. This can produce some phenomenally lifelike sounds, but there are somedrawbacks. Sampled impulse responses rarely match in-game synthetic environments; they represent a fixedlistener position and orientation; they are monophonic; they are difficult to transition between different areas.

Even with these limitations, they still provide high-quality results in many situations.

World Geometry and Acoustics

The shoebox model attempts to provide a simplified representation of an environment's geometry. Itassumes no occlusion, equal frequency absorption on all surfaces, and six parallel walls at a fixed distancefrom the listener's head. Needless to say, this is a heavy simplification for the sake of performance, and as VR

environments become more complex and dynamic, it may not scale properlySome solutions exist today to simulate diffraction and complex environmental geometry, but support is notwidespread and performance implications are still significant.

Environmental Transitions

Modeling a specific area is complex, but still relatively straightforward. Irrespective of choice of model,however, there is a problem of audible discontinuities or artifacts when transitioning between areas. Somesystems require flushing and restarting the entire reverberator, and other systems introduce artifacts asparameters are changed in real-time.

Presence and Immersion

By creating audio that is on par with high quality VR visuals, developers immerse the user in a true virtual world,giving them a sense of presence.

Audio immersion is maximized when the listener is located inside the scene, as opposed to viewing it fromafar. For example, a 3D chess game in which the player looks down at a virtual board offers less compellingspatialization opportunities than a game in which the player stands on the play field. By the same token, anaudioscape in which moving elements whiz past the listener's head with auditory verisimilitude is far morecompelling than one in which audio cues cut the listener off from the action by communicating that they'reoutside of the field of activity.

7/25/2019 Audios Dk

17/65


Note: It should be noted that while the pursuit of realism is laudable, it is also optional, as we want

developers and sound designers to maintain creative control over the output.

Sound Design for SpatializationNow that we've established how humans place sounds in the world and, more importantly, how we can foolpeople into thinking that a sound is coming from a particular point in space, we need to examine how we mustchange our approach to sound design to support spatialization.

Mono

Most spatialization techniques model sound sources as infinitely small point sources; that is, sound is treatedas if it were coming from a single point in space as opposed to a large area, or a pair of discrete speakers. As aresult, sounds should be authored as monophonic (single channel) sources.

Avoid Sine Waves

Pure tones such as sine waves lack harmonics or overtones, which presents several issues:

Pure tones do not commonly occur in the real world, so they often sound unnatural. This does not mean youshould avoid them entirely, since many VR experiences are abstract, but it is worth keeping in mind.

7/25/2019 Audios Dk

18/65


HRTFs work by filtering frequency content, and since pure tones lack that content, they are difficult tospatialize with HRTFs

Any glitches or discontinuities in the HRTF process will be more audible since there is no additionalfrequency content to mask the artifacts. A moving sine wave will often bring out the worst in a spatializationimplementation.

Use Wide Spectrum Sources

For the same reasons that pure tones are poor for spatialization, broad spectrum sounds work well by providinglots of frequencies for the HRTF to work with. They also help mask audible glitches that result from dynamicchanges to HRTFs, pan, and attenuation. In addition to a broad spectrum of frequencies, ensure that there issignificant frequency content above 1500 Hz, since this is used heavily by humans for sound localization.

Low frequency sounds are difficult for humans to locate - this is why home theater systems use a monophonicsubwoofer channel. If a sound is predominantly low frequency (rumbles, drones, shakes, et cetera), then youcan avoid the overhead of spatialization and use pan/attenuation instead.

Avoid Real-time Format Conversions

Converting from one audio format to another can be costly and introduce latency, so sounds should bedelivered in the same output format (sampling rate and bit depth) as the target device. For most PCs, this willbe 16-bit, 44.1 kHz PCM, but some platforms may have different output formats (e.g. 16-bit, 48 kHz on GearVR).

Spatialized sounds are monophonic and should thus be authored as a single channel to avoid stereo-to-monomerging at run-time (which can introduce phase and volume artifacts).

If your title ships with non-native format audio assets, consider converting to native format at installation or loadtime to avoid a hit at run-time.

Mixing Scenes for Virtual RealityAs with sound design, mixing a scene for VR is an art as well as a science, and the following recommendationsmay include caveats.

Creative Control

Realism is not necessarily the end goal! Keep this in mind at all times. As with lighting in computerenvironments, what is consistent and/or correct may not be aesthetically desirable. Audio teams must becareful not to back themselves into a corner by enforcing rigid notions of correctness on a VR experience.

This is especially true when considering issues such as dynamic range, attenuation curves, and direct time ofarrival.

Accurate 3D Positioning of Sources

Sounds must now be placed carefully in the 3D sound field. In the past a general approximation of locationwas often sufficient since positioning was accomplished strictly through panning and attenuation. The defaultlocation for an object might be its hips or where its feet met the ground plane, and if a sound is played fromthose locations it will be jarring with spatialization, e.g. crotch steps or foot voices.

7/25/2019 Audios Dk

19/65


Directional Sources

The Oculus Audio SDK does not support directional sound sources (speakers, human voice, car horns, etcetera). However, higher level SDKs often model these using angle-based attenuation that controls thetightness of the direction. This directional attenuation should occur before the spatialization effect.

Area Sources

The Oculus Audio SDK does not support area sound sources such as waterfalls, rivers, crowds, and so on.

Doppler Effect

The Doppler effect is the apparent change of a sound's pitch as its source approaches or recedes. VRexperiences can emulate this by altering the playback based on the relative speed of a sound source and thelistener, however it is very easy to introduce artifacts inadvertently in the process.

The Oculus Audio SDK does not have native support for the Doppler effect, though some high-level SDKs do.

Sound Transport Time

In the real world, sound takes time to travel, so there is often a noticeable delay between seeing and hearing

something. For example, you would see the muzzle flash from a rifle fired at you 100 meters away roughly330 ms before you would hear it. Modeling propagation time incurs some additional complexity and mayparadoxically make things seem less realistic, as we are conditioned by popular media to believe that louddistance actions are immediately audible.

The Oculus Audio SDK supports time-of-arrival.

Non-Spatialized Audio

Not all sounds need to be spatialized. Plenty of sounds are static or head relative, such as:

User interface elements, such as button clicks, bleeps, transitions, and other cues

Background music

Narration Body sounds, such as breathing or heart beats

Such sounds should be segregated during authoring as they will probably be stereo, and during mixing so theyare not inadvertently pushed through the 3D positional audio pipeline.

Performance

Spatialization incurs a performance hit for each additional sound that must be placed in the 3D sound field. Thiscost varies, depending on the platform. For example, on a high end PC, it may be reasonable to spatialize 50+sounds, while you may only be able to spatialize one or two sounds on a mobile device.

Some sounds may not benefit from spatialization even if placed in 3D in the world. For example, very low

rumbles or drones offer poor directionality and could be played as standard stereo sounds with some panningand attenuation.

Ambiance

Aural immersion with traditional non-VR games was often impossible since many gamers or PC users relied onlow-quality desktop speakers, home theaters with poor environmental isolation, or gaming headsets optimizedfor voice chat.

With headphones, positional tracking, and full visual immersion, it is now more important than ever that sounddesigners focus on the user's audio experience.

7/25/2019 Audios Dk

20/65


This means:

Properly spatialized sound sources

Appropriate soundscapes that are neither too dense nor too sparse

Avoidance of user fatigue

Suitable volume levels comfortable for long-term listening

Room and environmental effects

Audible Artifacts

As a 3D sound moves through space, different HRTFs and attenuation functions may become active, potentiallyintroducing discontinuities at audio buffer boundaries. These discontinuities will often manifest as clicks, popsor ripples. They may be masked to some extent by reducing the speed of traveling sounds and by ensuring thatyour sounds have broad spectral content.

Latency

While latency affects all aspects of VR, it is often viewed as a graphical issue. However, audio latency can bedisruptive and immersion-breaking as well. Depending on the speed of the host system and the underlyingaudio layer, the latency from buffer submission to audible output may be as short as 2 ms in high performance

PCs using high end, low-latency audio interfaces, or, in the worst case, as long as hundreds of milliseconds.

High system latency becomes an issue as the relative speed between an audio source and the listener's headincreases. In a relatively static scene with a slow moving viewer, audio latency is harder to detect.

Effects

Effects such as filtering, equalization, distortion, flanging, and so on can be an important part of the virtualreality experience. For example, a low pass filter can emulate the sound of swimming underwater, where highfrequencies lose energy much more quickly than in air, or distortion may be used to simulate disorientation.

VR Audio GlossaryDefinitions of technical terms VR audio terms.

Term Definition

Anechoic Producing no echoes; very low or no reverberation.

Attenuation A loss of energy; in acoustics, typically a reduction in volume.

Direct Sound Sound that has traveled directly to the listener without reflecting (versus reverberantsound).

Early Reflections Reflected sounds that arrive relatively soon at a listener's location (i.e., before LateReflections).

Head-RelatedImpulse Response(HRIR)

A formal characterization of the effect of sound interacting with the geometry of aparticular human body. Used to create head-related transfer functions.

7/25/2019 Audios Dk

21/65


Term Definition

Head-RelatedTransfer Function(HRTF)

A transformation of an acoustic signal using a head-related impulse response. Used tosimulate the effects of interaction of a sound originating from a specific direction withthe geometry of a particular human body.

Head Shadowing The attenuationof sound caused by the head lying between an ear and the sound

source.

Initial Time Delay The interval between the arrival of a direct soundand its first reflection.

Interaural LevelDifference (ILD)

The difference in a sound's level or volume between the two ears.

Interaural TimeDifference (ITD)

The length of the interval between when a sound arrives at the first ear and when itarrives as the second ear.

Late Reflections Reflected sounds that arrive relatively late at a listener's location (i.e., after early

reflections).

Motion Parallax When moving objects are father from a perceiver, their apparent speed of traveldecreases; for example, a moving airplane on the horizon appears to be travelling moreslowly than a nearby car. The apparent rate of travel of an object can therefore be usedas a distance cue.

Pinnae The visible portion of the ear that lies outside the head.

Reverberant Sound Sound that has reflected or reverberatedbefore arriving at a listener's location (versusdirect sound).

Reverberation The reflection of sound off of a surface, or the temporary persistence of sound in aspace caused by reverberation.

Sound Localization 1. The process of determining the location of a sound's origin; or 2. the suggestion ofan object's location based on the manipulation of auditory cues.

Sound Spatialization The representation of a sound within three-dimensional space.

7/25/2019 Audios Dk

22/65

| Oculus Audio SDK Guide | Audio

Oculus Audio SDK GuideThis document describes how to install, configure, and use the Oculus Audio SDK.

The Audio SDK consists of documentation, samples, plugins, source code, libraries, and tools to helpdevelopers implement immersive audio in their VR apps.

Thus far, much of the discussion about virtual reality has centered on visual aspects such as pixel density,latency, frame rate, and visual artifacts. However, audio characteristics reinforce visual immersion by providingcompelling auditory cues which indicate the positions of objects around the viewer, including those that areoutside their field of view.

We strongly recommend beginning with Introduction to Virtual Reality Audiofor a technical discussion of thekey concepts and terminology.

SDK Contents and Features

The Oculus Audio SDK provides tools to help VR developers incorporate high-quality audio into their apps andexperiences by providing:

OculusHQsound rendering path, supporting a large number of sound sources, providing strong directionalcues, early reflections and late reverberations

Plugins for game enginessuch as Unity3D

Plugins for popular audio middlewaresuch as FMOD and Wwise

AAX Pluginfor use with Avid's Pro Tools

VST pluginfor Windows and OS X

Tutorials and white papers

A low-level C/C++ SDKthat provides spatialization and (optional) reverberation is also available by request.Please submit requests to the Oculus Audio Development Forumif you are interested.

Requirements

Head Tracking

By tracking the listener's head position and orientation, we can achieve accurate 3D sound spatialization. Asthe listener moves or rotates their head, they perceive the sound as remaining at a fixed location in the virtualworld.

Developers may pass Oculus PC SDK ovrPosefstructures to the Oculus Audio SDK for head tracking support.Alternatively, they can pass listener-space sound positions and no pose information for the same effect.

Headphones

The Oculus Audio SDK assumes that the end user is wearing headphones, which provide better isolation,privacy, portability, and spatialization than free-field speaker systems. When combined with head tracking andspatialization technology, headphones deliver an immersive sense of presence. For more on the advantagesand disadvantages of headphones for virtual reality, please refer to Listening Devicesin Introduction to Audiofor Virtual Reality.
https://forums.oculus.com/viewforum.php?f=76

7/25/2019 Audios Dk

23/65

Audio | Oculus Audio SDK Guid

FeaturesThis section describes the features supported by the Oculus Audio SDK.

Supported FeaturesThis section describes supported features.

Spatialization

Spatialization is the process of transforming monophonic sound sources to make them sound as though theyoriginate from a specific desired direction. The Oculus Audio SDK uses head-related transfer functions(HRTFs)to provide audio spatialization through the C/C++ SDK and plugins.

Note: It is important that all sounds passed to Oculus Audio Spatializers are monophonic - stereophonicrendering will be handled by our spatialization, and must not be applied over additional stereophonicrendering provided by your game engine or library.

Environmental Modeling

HRTFs provide strong directional cues, but without room effects, they often sound dry and lifeless. Someenvironmental cues (e.g., early reflections and late reverberation) are also important in providing strong cuesabout the distance to a sound source.

The Audio SDK supports early reflections and late reverberations using a simple 'shoebox model,' consistingof a virtual room centered around the listener's head, with four parallel walls, a floor, and a ceiling at varyingdistances, each with its own distinct reflection coefficient.

Unsupported FeaturesThere are other aspects of a high quality audio experience, however these are often more appropriatelyimplemented by the application programmer or a higher level engine.

Occlusion

Sounds interact with a user's environment in many ways. Objects and walls may obstruct, reflect, or propagatea sound through the virtual world. The Oculus SDK only supports direct reflections and does not factor in thevirtual world geometry. This problem needs to be solved at a higher level than the Oculus Audio SDK due tothe requirements of scanning and referencing world geometry.

Doppler Effect

The Doppler effectis the perceived change in pitch that occurs when a sound source is moving at a rapid ratetowards or away from a listener, such as the pitch change that is perceived when a car whooshes by. It is often

emulated in middleware with a simple change in playback rate by the sound engine.

Creative Effects

Effects such as equalization, distortion, flanging, and so on can be used to great effect in a virtual realityexperience. For example, a low pass filter can emulate the sound of swimming underwater, where highfrequencies lose energy much faster than in air, distortion may be used to simulate disorientation, a narrowbandpass equalizer can give a 'radio' effect on sound sources, and so on.

The Oculus Audio SDK does not provide these effects. Typically, these would be applied by a higher levelmiddleware package either before or after the Audio SDK, depending on the desired outcome. For example, a

7/25/2019 Audios Dk

24/65


low-pass filter might be applied to the master stereo buffers to simulate swimming underwater, but a distortioneffect may be applied pre-spatialization for a broken radio effect in a game.

Area and directional Sound Sources

The Oculus Audio SDK supports monophonic point sources. When a sound is specified, it is assumed that thewaveform data represents the sound as audible to the listener. It is up to the caller to attenuate the incomingsound to reflect speaker directional attenuation (e.g. someone speaking while facing away from the listener)and area sources such as waterfalls or rivers.

Sound Transport Time

Sound takes time to travel, which can result in a noticeable delay between seeing something occur and hearingit (e.g., you would see the muzzle flash from a rifle fired 100 m away roughly 330 ms before you would hear it).

We do not model sound propagation time, as it would incur additional complexity, and it can have theparadoxical effect of making things seem less realistic, as most people are conditioned by popular mediato believe that loud distance actions are immediately audible. Developers may delay a sound's onset

appropriately to simulate that effect, if desired.

Pitfalls and WorkaroundsSpatialization and room effects greatly contribute to realism, but they come at a cost.

Performance

Spatialization incurs a performance hit for each additional sound that must be placed in the 3D sound field. Thiscost varies, depending on the platform. For example, on a high end PC, it may be reasonable to spatialize 50+sounds, while you may only be able to spatialize one or two sounds on a mobile device.

Some sounds may not benefit from spatialization even if placed in 3D in the world. For example, very lowrumbles or drones offer poor directionality and could be played as standard stereo sounds with some panningand attenuation.

Audible Artifacts

As a 3D sound moves through space, different HRTFs and attenuation functions may become active, potentiallyintroducing discontinuities at audio buffer boundaries. These discontinuities will often manifest as clicks, popsor ripples. They may be masked to some extent by reducing the speed of traveling sounds and by ensuring thatyour sounds have broad spectral content.

LatencyWhile latency affects all aspects of VR, it is often viewed as a graphical issue. However, audio latency can bedisruptive and immersion-breaking as well. Depending on the speed of the host system and the underlyingaudio layer, the latency from buffer submission to audible output may be as short as 2 ms in high performancePCs using high end, low-latency audio interfaces, or, in the worst case, as long as hundreds of milliseconds.

High system latency becomes an issue as the relative speed between an audio source and the listener's headincreases. In a relatively static scene with a slow moving viewer, audio latency is harder to detect.

7/25/2019 Audios Dk

25/65

Audio | Oculus Audio SDK Guid

Compatibility between VR and Non-VR Games

Few developers have the luxury of targeting VR headsets exclusively, and must support traditional, non-VR games using external speakers and without the benefit of positional tracking. Weighing quality versuscompatibility can be difficult and incur development time.

Platform NotesThe Oculus Audio SDK currently supports Windows 7+, Android (Gear VR), Mac OS X, and Linux. This sectioncovers issues that may arise with different versions.

Component Windows Gear VR /Android

Mac Linux

C/C++ SDK yes yes yes yes

Wwise plugin yes TBD TBD no

Unity plugin yes yes yes TBD

FMOD plugin yes yes yes no

VST plugin yes no yes no

AAX plugin soon! no soon! no

Windows: The Oculus Audio SDK supports Windows 7 and later, both 32 and 64-bit.

Mac: The Oculus Audio SDK supports Mac OS X 10.9 and later, 32 and 64-bit.

Linux: The Oculus Audio SDK has been ported to Linux Ubuntu 14 (64-bit).

Android/Gear VR: Oculus Audio SDK supports Android phones on the Gear VR platform.

Middleware SupportVery few Oculus developers will use the Oculus Audio C/C++ SDK directly. Most developers use a middlewareframework, such as Audiokinetic Wwise or FMOD, and/or an engine such as Unity or Epic's Unreal. For thisreason, we support middleware packages and engines commonly used by developers.

Audiokinetic Wwise: Oculus provides a Wwise compatible plugin for Windows. More information about thisplugin can be found in the Oculus Spatializer for Wwise Integration Guide.

FMOD: The Oculus Audio SDK supports FMOD on the Windows, Mac and Android platforms. Moreinformation about this plugin can be found in the Oculus Spatializer for FMOD Integration Guide.

Unity3D: The Oculus Audio SDK supports Unity 4.6 on Android, Mac OS X and Windows. More informationabout this plugin can be found in the Oculus Spatializer for Unity Integration Guide.

Unreal Engine: Epic's Unreal Engine 4 supports numerous different audio subsystems. The Wwiseintegration (available directly from Audiokinetic) has been tested with our Wwise Spatializer plugin (seeabove).

Oculus Hardware Capabilities

Each Oculus hardware platform has different audio capabilities.

7/25/2019 Audios Dk

26/65


Note: All Oculus VR headsets requires headphones.

DK1does not provide any audio output and thus relies on the end user's system for audio output.

DK2also relies on the end user's system for audio output, but adds positional tracking.

The Oculus CV1HMD has a built-in DAC (audio over USB connected to the headset) and ADC (for

microphone). The audio subsystem is class compliant and supports Windows, Mac and Linux. The HMD hasintegrated headphones as well and a microphone.

The Samsung Gear VRmobile product supports 16-bit, 48 kHz sound. Like DK1, it supports head orientationbut not positional tracking. Headphone choice is dictated by the end user.

Mobile performance is significantly lower than desktop PC performance, so try to minimize the number of audiosources.

7/25/2019 Audios Dk

27/65

Audio | Oculus Native Spatializer for Un

Oculus Native Spatializer for UnityWelcome to this guide to using the Oculus Native Spatializer plugin in Unity!

OverviewThis guide describes how to install and use the Oculus Native Spatializer plugin in Unity 5.2+ and in end-userapplications.

The Oculus Native Spatializer Plugin (ONSP) is an add-on plugin for Unity that allows monophonic soundsources to be spatialized in 3D relative to the user's head location.

The Native Oculus Spatializer is built on Unitys Native Audio Plugin, which removes redundant spatializationlogic and provides a first-party HRTF. It supersedes our previous Oculus Spatializer Plugin for Unity (availableprior to Audio SDK 1.0 ), which used scripts to set settings on an Audio Source.

Our ability to localize audio sources in three-dimensional spaceis a fundamental part of how we experiencesound. Spatializationis the process of modifying sounds to make them localizable, so they seem to originatefrom distinct locations relative to the listener. It is a key part of creating a sense of presence in virtual realitygames and applications.

For a detailed discussion of audio spatialization and virtual reality audio, we recommend reviewing ourIntroduction to Virtual Reality Audioguide before using the Oculus Native Spatializer. If youre unfamiliar withUnitys audio handling, be sure to review the Unity Audio guide.

Note: Our previous OSP for Unity is now available as the Legacy Oculus Spatializer, and is intendedprimarily for users of Unity 4.

Requirements and Setup

Requirements

Windows 7/8

Unity 5.2 Professional or Personal, or later. See Unity Compatibility and Requirementsfor details on ourrecommended versions.

Note: The ONSP is not compatible with Unity 4. Projects build with Unity 4 should continue use theLegacy OSP for Unity.

Download and Setup

Note: We recommend removing any previously-imported versions of the OSP or ONSP beforeimporting a new plugin. See Updating to Oculus Native Spatializer for Unity from previous OSP for UnityVersionsbelow for instructions.

To download the ONSP and import it into a Unity project:

1. Download the Oculus Audio SDK Plugins package from the Oculus Downloadspage.

2. Extract the zip.
https://developer.oculus.com/documentation/game-engines/latest/concepts/unity-req/https://developer.oculus.com/downloads/https://developer.oculus.com/documentation/game-engines/latest/concepts/unity-req/http://docs.unity3d.com/Manual/Audio.htmlhttp://docs.unity3d.com/Manual/AudioMixerNativeAudioPlugin.html

7/25/2019 Audios Dk

28/65

| Oculus Native Spatializer for Unity | Audio

3. Open your project in the Unity Editor, or create a new project.

4. Select Assets> Import Package> Custom Package.

5. Select OculusNativeSpatializer.unitypackage and import.

6. When the Importing Packagedialog opens, leave all assets selected and click Import.

To turn on the Native Spatializer:

1. Go to Edit > Project Settings> Audioin Unity Editor2. Select Spatializer Pluginin the OculusSpatializerdrop-down setting in the AudioManager Inspector panel asshown below.

We recommend setting DSP Buffer Sizeto Best latencyto set up the minimum buffer size for the platform thatis supported, reducing overall audio latency.

Updating to Oculus Native Spatializer for Unity from previous OSP for Unity Versions

1. Note the settings used in OSPManager in your project.

2. Replace OSPAudioSource.cs (from previous OSP) on AudioSources with OculusSpatializerUserParams.cs in/Assets/OSP.

3. Set the appropriate values previously used in OSPManager in the plugin effect found on the mixer channel.Note that the native plugin adds functionality, so you will need to adjust to this new set of parameters.

4. Remove OSPManager from the project by deleting OSPManager*.* from /Assets/OSP exceptyour newly-added OculusSpatializerUserParams.cs.

5. Verify that OculusSpatializer is set in the Audio Manager and that Spatialization is enabled for that voice.

All functions such as Play, Stop, et cetera, that used to be on the previous OSP no longer exist. Instead, oneuses the functions on AudioSource to start, stop and modify sounds as required.

7/25/2019 Audios Dk

29/65


Exploring Oculus Native Spatializer with the SampleSceneTo get started, we recommend opening the supplied demonstration scene RedBallGreenBall, which provides asimple introduction to OSNP resources and examples of how what the spatializer sounds like.

This simple scene includes a red ball and a green ball, which illustrate different spatializer settings. A loopingelectronic music track is attached to the red ball, and a short human voice sequence is attached to the greenball.

Launch the scene in the Unity Game View, navigate with the arrow keys, and control the camera orientationwith your mouse to quickly hear the spatialization effects.

To import and open RedBallGreenBall :

1. Create a new Unity project.

2. Import the OculusNativeSpatializer.unitypackage.

3. When the Importing Packagedialog opens, leave all assets selected and click Import.

4. Enable the Spatializer as described in Download and Setup

5. Open RedBallGreenBall in /Assets/scenes.

To preview the scene with a Rift:

1. Import and launch RedBallGreenBall as described above.2. In Build Settings, verify that the PC, Mac and Linux Standaloneoption is selected under Platform.

3. In Player Settings, select Virtual Reality Supported.

4. Preview the scene normally in the Unity Game View.

To preview the scene in Gear VR (requires gamepad):

1. Be sure you are able to build and run projects on your Samsung phone (Debug Mode enabled, adbinstalled, etc.) See the Mobile SDK Setup Guidefor more information.

2. Follow the setup steps at the top of this section.
https://developer.oculus.com/documentation/mobilesdk/latest/concepts/book-dev-intro/

7/25/2019 Audios Dk

30/65


3. In BuildSettings

a. Select Androidunder Platform.

b. Select Add Currentto Scenes in Build.

4. In Player Settings, select Virtual Reality Supported.

5. Copy your osig to /Assets/Plugins/Android/assets.

6. Build and run your project.

7. Navigate the scene with a compatible gamepad.

Applying Spatialization

Attach the helper script ONSPAudioSource.cs, found in Assets/OSPNative/scripts, to an AudioSource. Thisscript accesses the extended parameters required by the Oculus Native Spatializer. Note that all parametersnative to an AudioSource are still available, though some values may not be used if spatialization is enabled onthe audio source.

In this example, we look at the script attached to the green sphere in our sample RedBallGreenBall:

7/25/2019 Audios Dk

31/65


7/25/2019 Audios Dk

32/65


OculusSpatializerUserParams Properties

Enable Spatialization Effectively, a bypass button for spatialization. If disabled, the attached Audio Sourcewill act as a native audio source without spatialization. This setting is linked to thecorresponding parameter in the Audio Source expandable pane (collapsed in the abovecapture).

Disable Reflections Select to disable early reflections and reverb for the spatialized audio source.To use early reflections and reverb, you must deselect this value and add anOculusSpatializerReflection plugin to the channel where you send the AudioSource inthe Audio Mixer. See Audio Mixer Setupbelow for more details.

Gain Adds up to 24 dB gain to audio source volume (in db), with 0 equal to unity gain.

Inverse Square Attenuation Settings

Enable If selected, the audio source will use an internal attenuation falloff curve controlled bythe Near and Far parameters. If deselected, the attenuation falloff will be controlled by

the authored Unity Volume curve within the Audio Source Inspector panel.Note: We strongly recommend enabling internal attenuation falloff for a more accuraterendering of spatialization. The internal curves match both the way the direct audiofalloff as well as how the early reflections and reverbs are modelled.

Falloff Near Sets the point at which the audio source starts attenuating, in meters. It also influencesthe reflection/reverb system, whether or not internal inverse square attenuation isenabled. Larger values will result in less noticeable attenuation when the listener is nearthe sound source.

Falloff Far Sets the point at which the audio source reaches full volume attenuation, in meters.It also influences the reflection/reverb system, whether or not internal inverse square

attenuation is enabled. Larger values allow for loud sounds that can be heard from adistance.

Audio Mixer Setup

Note: In Unitys terminology, groupis roughly the same as channel. In this section, we use the termsinterchangeably.

Unity 5 includes a flexible mixer architecture for routing audio sources. A mixer allows the audio engineerto add multiple channels, each with their own volume and processing chain, and set an audio source to thatchannel. For detailed information, see Unitys Mixer documentation.

Shared Reflection/ReverbTo allow for a shared reflection/reverb system within our Oculus Spatializer, you must create a mixer channeland add the OculusSpatializerReflection effect to that channel.

1. Select the Audio Mixertab in your Project View.

2. Select Add Effectin the Inspector window.

3. Select OculusSpatializerReflection.

4. Set the Outputof your attached Audio Source to Master (SpatializerMixer).

5. Set reflection/reverberation settings to globally affect spatialized voices.
http://docs.unity3d.com/Manual/AudioMixer.html

7/25/2019 Audios Dk

33/65


Table 1: Mixer Settings

E.Rflt On Set to any non-zero value to enable the early reflection system.

E.Rflt Rev On Set to any non-zero value to enable global reverb (requires early reflection (E.Rflt On) tobe enabled).

Room X/Y/Z Sets the dimensions of the theoretical room used to calculate reflections, in meters. Thegreater the dimensions, the further apart the reflections. Range: 0 - 200m.

Left, Right, Up, Down,Behind, Front

Sets the percentage of sound reflected by each respective wall. At 0, the reflection isfully absorbed. At 1.0, the reflection bounces from the wall without any absorption.Caps at 0.97 to avoid feedback.

RedBallGreenBall Example

To see how this works in RedBallGreenBall, access the Audio Mixer by selecting the Audio Mixertab in yourProject View. Then select Masterunder Groupsas shown below.

Select the green sphere in your Scene View. Note that the Outputof the attached Audio Source vocal1 is set toour Master (SpatializerMixer):

7/25/2019 Audios Dk

34/65


You can now set reflection/reverberation settings to globally affect spatialized voices:

7/25/2019 Audios Dk

35/65

Audio | Legacy Oculus Spatializer for Unity Integration Guid

Legacy Oculus Spatializer for UnityIntegration GuideWelcome to this guide to using the Legacy Oculus Spatializer plugin in Unity!

OverviewThe Legacy Oculus Spatializer Plugin (OSP) is an add-on plugin for the Unity tool set that allows monophonicsound sources to be spatialized in 3D relative tothe user's head location. This integration guide outlines how toinstall and use OSP in the Unity editor and end-user application.

Note: Developers using Unity 5 should use the Oculus Native SpatializerPlugin. The Legacy OculusSpatializer Plugin is provided for Unity 4 development.

Version Compatibility

This integration has been tested with Unity 5.1onWindows 7 / 8(32 and 64-bit), as well as on Mac OS X10.8.5 and Android 4.4.4 (KitKat). It is also known to work with Unity 4.5and later.

Both Unity Professional and Free are supported.

General OSP Limitations

1. CPU usage increases when early reflections are turned on, and increases proportionately as roomdimensions become larger.

Reducing Android Latency: OpenSL Low-LatencyDriver

Android audio latency is known to be exceedingly high, generally hitting 100-200 ms between the time asound event is triggered to when the sound is actually heard. Gear VR minimizes audio latency within Unityapplications by switching to an optimized OpenSL driver.

Your Android application must be built with the proper AndroidManifest.xml file. This is accomplished by usingthe official Gear VR Unity integration and following the relevant manifest instructions.

Use of the OpenSL driver is currently available in Unity v4.6.1p4 and higher. To find it:

1. Navigate to https://unity3d.com.

2. Click Download.

3. Click Looking for a patch releases?

4. Select Unity v4.6.1p4 or higher.

It is also publicly available in Unity 5.

Note: Use of OpenSL through Unity on a VR-enabled device is still a work in progress. As such, pleaseknow that it will be easier to overload the audio CPU usage with OpenSL over the higher-latency
https://unity3d.com/https://unity3d.com/

7/25/2019 Audios Dk

36/65

| Legacy Oculus Spatializer for Unity Integration Guide | Audio

Android drivers; you may hear more buffer over-run issues depending on the complexity of your audioscene. We recommend treating OpenSL as a preview of low-latency audio on mobile VR.

For up-to-date information about OpenSL support in Unity, please refer to the Oculus Unity Integration Guide.

Installing into UnityUnity OSP contents are found in the Unity distribution package OculusSpatializer.unitypackage. This packagecontains all of the components required to install the OculusSpatializer into your Unity project.

To import the package into Unity:

1. Open the project in which you would like to install the plugin, or create a new one.

2. Click on OculusSpatializer.unitypackage file from your operating system to install all files into the project.

Note: OculusSpatializer.unitypackage contains a test scene called OSPTest, which uses the Unitystandard asset FirstPersonController. If you install the package into a project which already includes thisasset, you will have a resource conflict. To avoid this conflict, install everything from the unitypackageexcept for the OSPTestScenedirectory.

Once installed, you will see the following folders in your project:

OSP

OSPTestScene

Plugins: includes the OculusSpatializerPlugins for various platforms.

How to Use the Oculus Spatializer in Unity

To get started, we recommend that you create a new Unity project and import OculusSpatializer.unitypackage,and then have a look at the test scene OSPTest found in the folder OSPTestScene. This is a good way tobecome familiar with the OSP resources and to hear what the spatializer sounds like.

7/25/2019 Audios Dk

37/65


To add spatialization to audio in Unity, start with the prefabs OSPManagerand OSPAudioSourcein /OSP/Prefabs.

OSPManager

Add OSPManager to a scene to initialize the spatialization engine. OSPManager contains global properties thatthe application can use to change the characteristics of the spatializer.

OSPManager Properties

Bypass Disables spatialization. All sounds routed through OSPAudioSource will get Unity native2D panning.

Global Scale The scale of positional values fed into the spatializer must be set to meters. Someapplications have different scale values assigned to a unit. For such applications, usethis field to adjust the scale for the spatializer. Unity defaults to 1 meter per unit.

Example: for an application with unit set to equal 1 cm, set the Global Scale value to0.01 (1 cm = 100 m).

Gain Sets a global gain to all spatialized sounds. Because the spatializer attenuates theoverall volume, it is important to adjust this value so spatialized sounds play at the samevolume level as non-spatialized (or native panning) sounds.

7/25/2019 Audios Dk

38/65


Enable (Early Reflections) Allows early reflections to be turned on. Early reflections enhance the spatializationeffect, but incurs a CPU hit.

Reverb On Adds a fixed reverb tail to the output when enabled. This reverb is calculated based onthe room parameters that are set up to calculate the reflections. While its use does notnecessarily incur a CPU hit, changing the room parameters may cause CPU hitches tooccur.

Room Dimensions Sets the dimensions of the theoretical room used to calculate reflections. The greaterthe dimensions, the further apart the reflections. Room size currently caps at the rangeshown in the descriptor.

Reflection Values Sets the percentage of sound reflected by each wall (Left/Right, Forward/Backward, UpDown). At 0, the reflection is fully absorbed. At 1.0, the reflection bounces from the walwithout any absorption. Caps below 1.0 to avoid feedback.

Reflection Gain This value will adjust the gain on the reflection signal (both early reflections and reverb)It is based in meters, which represents how far the reflection signal would travel before

attenuating to zero. The greater the number, the louder the reflection signal will be inthe final output.

Notes and Best practices

Up to 64 sounds may be spatialized. Additional sounds use native Unity 2D panning until a spatialized soundslot becomes free (e.g., a sound being spatialized stops).

Set your audio output configuration is set to a 2.1 or 2 Stereo Channel Configuration (Speakers orHeadphones). The spatializer will not work on higher channel configurations.

Note that the theoretical room used to calculate reflections follows the listener's position and rotates with thelistener's orientation. Future implementation of early reflections will allow for the listener to freely walk around a

static room.

When using early reflections, be sure to set non-symmetrical room dimensions. A perfectly cubical room maycreate re-enforcing echoes that can cause sounds to be poorly spatialized. The room size should roughly matchthe size of the room in the game so the audio reinforces the visuals. The shoebox model works best whensimulating rooms. For large spaces and outdoor areas, it should be complimented with a separate reverb.

IMPORTANT: When early reflections are enabled, you must ensure that the room size is set to be large enoughto contain the sound. If a sound goes outside the room size relative to the listeners position, early reflectionswill not be heard.

OSPAudioSourceUse OSPAudioSource to create sounds that will be spatialized. OSPAudioSource consists of a native UnityAudio Source component and an OSPAudioSource script component that interacts with the spatializer.

7/25/2019 Audios Dk

39/65


OSPAudioSource Properties

OSP works as an add-on to the native Unity Audio Source component. It derives much of its functionality fromthe Audio Source component, including the distance attenuation curves.

Note: When triggering the sounds manually, you mustcall the OSPAudioSource component Play and

Stopfunctions. Please look at the OSPAudioSource.cs script for a list of the public functions available tocontrol a spatialized sound source.

A few caveats are detailed in Notes and Best Practices below.

Bypass Use this toggle if you want the audio to skip 3D spatialization and play via native 2DUnity panning. This feature can be toggled for convenience during run-time, and maymake integration of OSP Audio Sources into existing audio mangers easier and moreflexible.

Play On Awake We recommend that you use this toggle in the OSPAudioSource instead of the onefound in the native Unity AudioSource component. Enabling Play On Awakein the

native AudioSource component will work, but you may hear a hiccup in the sound asthe spatializer tries to obtain spatialization resources.

Disable Reflections If set to true, this sound instance will not calculate reflections/reverberation. Use thisfor sounds that do not need to have reflections enabled. Reflections take up extra CPU,so this is a good way to reduce the overall audio CPU cost. Note that if reflections aredisabled globally within OSP manager, this setting will have no effect.

Notes and Best Practices

3D Sound Settings within the Unity native Audio Source component are available to be used by the sounddesigner. However, Pan Leveland Spreadare overridden by the OSPAudioSource component while audio is

being spatialized:

7/25/2019 Audios Dk

40/65


It is important not to modify these values while sound is running through the spatializer, or it will dramaticallyaffect the output of the spatialization.

Please ensure that you check the 3D Soundcheck box in audio asset import settings (Unity 4 only; Unity 5 doesnot include this option).

Currently, only monophonicsounds will spatialize. A stereo sound will not be collapsed down to a monophonicsound (even if it is set by the user in Unity as a 3D sound). As such, a spatialized sound may not use stereospread to make a sound encompass the listener as it gets closer - this a common practice for currentspatialization techniques.

OSP Version Migration in Unity

To migrate to a new version of the Oculus Spatializer Plugin for Unity, just open an existing Unity project andimport the new version of OculusSpatializer.unitypackage included in the Audio SDK.

7/25/2019 Audios Dk

41/65

Audio | Oculus Spatializer for Wwise Integration Guid

Oculus Spatializer for Wwise IntegrationGuideWelcome to this guide to using the Oculus spatializer plugin in Wwise!

OverviewThe Oculus Spatializer Plugin (OSP) is an add-on plugin for the Audiokinetic Wwise tool set that allowsmonophonic sound sources to be spatialized in 3D relative to the user's head location. This integration guidedescribes how to install and use OSP in both the Wwise application and the end-user application.

Version Compatibility

Two distributions of the Wwise plugin are provided. Please choose the appropriate one, based on the version

of Wwise used by your application:AudioSDK\Plugins\Wwise This version is compatible with all Wwise 2014.x

versions and has been tested against Wwise 2014.1on Windows (32-bit and 64-bit).

AudioSDK\Plugins\Wwise2015 This version is compatible with all Wwise 2015.xversions and has been tested against the Wwise2015.1 beta on Windows (32-bit and 64-bit).

General OSP Limitations

1. CPU usage increases when early reflections are turned on, and increases proportionately as roomdimensions become larger.

Limitations Specific to Wwise Integration

1. The plugin may only be added to one bus. If you add the plugin to a second bus, you may experience someunwanted noise on the audio output.

Installing into Wwise Tool

Directory Structure

The following directories may be found within this distribution file:

include: Contains the header file OculusSpatializer.h, which is used to integrate Wwise into anapplication. It contains important registration values that an application must use to properly register OSPwithin Wwise. The header file also includes (commented out) source code that shows how to properlyregister the plugin with the Wwise run-time.

Win32andx64: Contains the DLL used for both the Wwise tool and the user application.

bin\plugins: Contains OculusSpatializer.dlland OculusSpatializer.xml. Located in \Win32 or\x64.

7/25/2019 Audios Dk

42/65

| Oculus Spatializer for Wwise Integration Guide | Audio

Installation

Copy the files found in the folder \bin\plugins to the folder Wwise\Authoring\\Release\bin\plugins.

Note: "platform" = "\Win32" or "\x64".

Installing into the Wwise Unity IntegrationThis guide describes installation of the Oculus Spatializer Plugin for Wwise with the Wwise Unity Integration.

Audiokinetic provides a Wwise plugin called the Wwise Unity Integration, which allows the use of Wwise inUnity games (see Audiokinetics documentation here).

Requirements

You must have the Wwise SDK installed, and the corresponding version of the Unity integration (check thatWWISEROOT and WWISESDK environment variables are correct).

Python 2.7 or 3.x is required, and your PATH environment variable must include the installation location.

If youre building the Wwise integration source in VS 2013, you may need to modify the Library Directoriespaths in the AkSoundEngine project for each build config from _vc100to _vc120.

Installation

1. Download the Wwise Unity Integration and source code for the relevant Wwise version (two separatedownloads) from the Audiokinetic website.

2. In the Unity Editor, select Assets> Import> Custom Packageand select the Wwise Integration to import itinto your Unity project.

3. Save the project and close Unity.

4. Open the Wwise Unity Integration help doc in \Assets\Wwise\Documentation.

a. Follow the instructions in the section Build the Integration code.

b. Follow the instructions in the section Adding Licensed Plugins.

5. Add the contents OculusSpatializer.h from the Oculus Audio SDK following the call toAK::SoundEngine::RegisterAllBuiltInPlugins() .

6. Recompile the lib and deploy to Unity Assets\Plugins as described in Adding Licensed Plugins.

How to Use the Oculus Spatializer in Wwise

Run the Wwise.exefrom the proper Authoring\\bin folder.

To add OSP to a bus, create a new audio bus and place it under one of the master buses.
https://www.audiokinetic.com/https://www.audiokinetic.com/library/2015.1_5418/?source=Unity&id=main.html

7/25/2019 Audios Dk

43/65

Audio | Oculus Spatializer for Wwise Integration Guid

Note: Mixer bus plugins may not be added to master buses.

Click on the newly-created bus and then click on the Mixer Plu

audios dk

Documents