0272-1716/02/$17.00 © 2002 IEEE IEEE Computer Graphics and Applications 49

Sound rendering1 is analogous to graphicsrendering when creating virtual auditory

environments. In graphics, we can create images by cal-culating the distribution of light within a modeled envi-ronment. Illumination methods such as ray tracing andradiosity are based on the physics of light propagationand reflection. Similarly, sound rendering is based onphysical laws of sound propagation and reflection.

In this article, we aim to clarifyreal-time sound rendering tech-niques by comparing them to visualimage rendering. We also describehow to perform sound rendering,based on the knowledge of soundsource(s) and listener locations,radiation characteristics of soundsources, geometry of 3D models,and material absorption data—inother words, the congruent dataused for graphics rendering. In sev-eral instances, we use the DigitalInteractive Virtual Acoustics (DIVA)auralization system,2 which we’vebeen developing since 1994 at theHelsinki University of Technology,as a practical example to illustrate a

concept. (The sidebar “Practical Applications of theDIVA System” [next page] briefly describes two appli-cations of our system.)

In the context of sound rendering, the term auraliza-tion3—making audible—corresponds to visualization.Applications of sound rendering vary from film effects,computer games, and other multimedia content toenhancing experiences in virtual reality (VR).

Approaches to sound renderingThe sound rendering methods we present in this arti-

cle enable dynamic rendering, in which the position ofthe listener or sound sources can change. Equally, thevirtual world’s geometry can change during the ren-dering process. Although a static setting would be tech-nically much simpler, it has only limited applicability

(virtual concert recordings, for example). In many VRapplications, user interaction is an essential feature, andthus we must perform dynamic rendering in real time.

In sound rendering, we can add spatial cues such assound source location or reverberation to the sound sig-nal. These cues can be based on either human percep-tion or the physics of sound. In many applications, theoriginal sound signal is the essential content, and ren-dering only adds spatial features as effects. Then, wecan obtain accurate enough results by applying percep-tual sound rendering. In this approach, a detailed 3Dmodel isn’t necessary, but we can use descriptive para-meters (such as reverberance or brilliance) to control asignal-processing algorithm to produce a desired audi-tory sensation. The perceptual approach is well suitedfor postprocessing music performances to create addi-tional room acoustic effects.4

The physics-based approach is more appropriate whenwe’re interested in the acoustic environment rather thanthe sound itself. For navigating in a virtual environment(VE), locations of objects and sound sources are neces-sary. For an architect or acoustics designer, the essentialelements are the audible effects caused by a constructedenvironment’s objects and properties.

Interactive audio–visual virtualenvironments

Figure 1 (next page) illustrates the basic architectureof a virtual auditory environment. Typically, the observ-er can fly around in a 3D model where light and soundsources are enclosed. Graphics rendering is based on the3D geometry, light sources, and materials. Sound ren-dering shares the same (although often simplified)geometry and requires audio signal sources located inthe 3D space, as well as acoustic properties of materials.

Due to differences in physics and sense organs ofsound and light, the rendering system requires differentsignal resolutions and update rates. Human visionprocesses many light signals from different directions(high spatial resolution) in parallel, whereas these sig-nals are integrated over time such that a frame rate ofabout 30 Hz usually suffices. Sound arriving from alldirections to the ear is integrated into one signal, where

We survey sound rendering

techniques by comparing

them to visual image

rendering and describe

several approaches for

performing sound rendering

in virtual auditory

environments.

Tapio Lokki, Lauri Savioja, and Riitta VäänänenHelsinki University of Technology, Finland

Jyri HuopaniemiNokia Research Center, Finland

Tapio TakalaNokia Ventures Organization, Finland

Virtual Worlds, Real Sounds

CreatingInteractive VirtualAuditoryEnvironments

Virtual Worlds, Real Sounds

50

The DIVA virtual orchestra1 is a showcasedesigned for the Siggraph 97 Electric Garden. Themain goal was to demonstrate several importantaudio–visual simulation themes (computergraphics and human animation, sound synthesis,auralization, and user interaction) in oneinteractive virtual concert experience.

The virtual orchestra consists of four modules:

� tracking of tempo and other musical characteris-tics from a live human conductor;

� visualizing animated virtual musicians and virtualspaces, as Figure A depicts;

� generating sound based on MIDI-encoded scores,especially applying physical modeling of musicalinstruments; and

� auralizing sound sources in acoustical spaces usingbinaural or multichannel reproduction techniques.

During one week at Siggraph 97, more than700 people conducted the virtual orchestra. Sincethen, it has been a popular demonstration andalso used in artistic installations. In one concert,the virtual players were accompanied by a realstring quintet. The virtual orchestra was also part

of the Second International Sibelius Conductors’Competition, where the audience was offered achance to conduct the competition pieces in theconcert hall’s lobby.

We’ve also applied the DIVA system in the fieldof room acoustic design. The Marienkirche film,2

shown in Siggraph 98 Electronic Theater, is anexample of the first high-quality audio–visualdemonstration where both sound and imageswere automatically rendered, based on roomgeometry and surface data. Figure B shows oneexample frame.

References1. T. Lokki et al., “Real-time Audiovisual Rendering and

Contemporary Audiovisual Art,” Organised Sound, vol.3, no. 3, 1998, pp. 219-233.

2. T. Takala et al., “Marienkirche - A Visual and Aural Demon-stration Film,” Electronic Art and Animation Catalog (Proc.Siggraph 98), ACM Press, New York, 1998, p. 149.

Practical Applications of the DIVA System

A The cartoonish animated players of the virtual orchestra, playing at theElectric Garden subway station during Siggraph 97.

B A snapshot of the Marienkirche film, presented inthe Electronic Theater in Siggraph 98.

Rendering Display

User interaction

Model of 3D geometry

Observer's positionand orientation 30 Hz

Audio signal

Auralization

Auralizationparameters 30 Hz

Samples 44,100 Hz

Samples 44,100 Hz

Images 30 Hz

Auralizationparameterscalculation

Visualrendering

1 Processes ofgraphics andsound render-ing. Using the3D model anduser interac-tions as input,they produce anaudio–visualscene with arendered imageand spatializedsound.

© 1

998

Erkk

i Rou

sku

spectral and temporal features give information aboutthe sound paths through the environment. Although asample rate of more than 40 kHz is necessary for the sig-nal to cover the whole audible spectrum, most dynamicperceptional cues are blurred over a sensory fusion inter-val of about 50 ms. Thus, a similar update rate (as withthe visual display) is sufficient for simulating changes inan auditory environment. However, to avoid abruptlocalization changes (which might cause audible clicks),we smooth these changes using interpolation.

Virtual auditory environmentsThe basic defining components of a visual VE are light

sources, 3D geometry, and the light transmittance prop-erties of surfaces and materials. Respectively, for virtualacoustics, we need to model sound sources, roomacoustics, and in some cases (depending on the soundreproduction method), human spatial hearing properties.

Sound-source modelingLight sources emit light at different colors, usually

modeled with the three-component RGB spectrum.Lights can have different radiation characteristics, suchas omnidirectional or spot-like, even with color textures.With sound sources, the emitted signal can be variableover the whole audio spectrum, and the radiation char-acteristics are usually frequency dependent.

Sound-source modeling deals with methods to pro-duce sound in an auditory scene. The most straightfor-ward way is to take prerecorded digital audio as a sourcesignal. In such a case, however, we have little opportu-nity to change it interactively. Another choice is to usegenerative techniques such as speech synthesis, musi-cal instrument models, or algorithms producing every-day sounds. This article doesn’t concentrate on soundsynthesis methods, but they are an important part of vir-tual auditory environments.

In most auralization systems, sound sources have tra-ditionally been treated as omnidirectional point sources.This approximation is valid for many cases. For exam-ple, most musical instruments have frequency-dependent radiation patterns. Typical sound sources,such as the human voice, loudspeakers, or many musi-cal instruments radiate more energy to the frontal hemi-sphere, whereas sound radiation gets attenuated andlow-pass filtered when the angular distance from theon-axis direction increases.

Modeling room acousticsBecause sound and light are both waves—one elec-

tromagnetic and the other mechanical motion—manysimilarities appear in their propagation. In most cases,we can handle both as rays emanating from the sources,traveling through the air, bouncing from surfaces, andfinally ending at a receiver.

With respect to traveling speed and wavelength, how-ever, light and sound differ. In human scales, light istransmitted instantaneously, whereas the speed of sound(340 meters per second in air) causes noticeable propa-gation delays perceived as echoes and reverberation. Thewavelength of light (400 to 740 nanometers) is negligi-bly small and must be considered only in special cases.

The range of audible wavelengths covers three orders ofmagnitude (0.017 to 17 m) comparable in size to objectsin human environments. Therefore, diffraction is a phe-nomenon we must consider as well. Occlusion becomesfrequency dependent such that while higher frequenciesget attenuated by an occluding object, lower frequenciescan pass it without noticeable effect. For example, aspeaking person can be heard out-of-sight around a cor-ner, although the sound is muffled.

Another interesting phenomenon is diffuse reflec-tions. In graphics, the topic is a popular research area,and current radiosity algorithms solve it nicely. Withreal-time physics-based acoustic simulations, this stillis an open issue. The radiosity method solves an ener-gy equilibrium, which we can apply to stationarynoise, but it isn’t suitable for auralization of time-varying sounds.

Despite the lack of diffraction modeling, the most com-monly used methods in room acoustic simulation are raybased. It’s interesting to note that the history of ray trac-ing dates back to the late 1960s in both graphics5 andsound rendering.6 Other ray-based techniques are theimage-source method7 and beam tracing. In addition,researchers have applied wave-based algorithms, suchas finite-element (FEM), boundary element (BEM), andfinite-difference-time-domain (FDTD) methods. Due totheir computational complexity, these methods aren’tpractical through the whole audible frequency range, andwe can’t currently use them in real-time applications.

Ray-based methods typically neglect diffraction, butwe can roughly model diffusion by dividing reflectionsinto specular and diffusive components, as in graphics.Despite these simplifications, we can do quite reliableroom acoustic simulations, and today’s concert halls andother acoustically interesting spaces are often designedwith the aid of computers.

Auditory displays and modeling of spatialhearing

To experience VR, we need both visual and auditorydisplays with 3D capability. Rendering high-qualityimages for both eyes independently gives a stereoscop-ic view. In interactive situations, we can achieve the cor-rect projection with a head-tracking device combinedwith either a head-mounted display (HMD) or shutterglasses within a spatially-immersive display (SID) suchas the Cave Automatic Virtual Environment.

Three-dimensional sound reproduction means thatusers can perceive the direction of an incoming sound.As in graphics, techniques fall into two categories: sur-rounding loudspeakers and headphone reproduction.A straightforward approach would be to place a loud-speaker in the direction of each virtual sound source.However, this is impractical for arbitrarily moving

IEEE Computer Graphics and Applications 51

To experience VR, we need both visual and

auditory displays with 3D capability.

sources. With a limited set of loudspeakers, we musttune each speaker so that the resulting sound field inthe listener’s ear canals is approximately correct. Bothamplitude panning techniques (such as vector baseamplitude panning [VBAP]8) and decoded 3D peri-phonic systems (such as Ambisonics9) require manyloudspeakers and several cubic meters of echoless space.

Headphones are more flexible than multichannelloudspeaker setups, but they make sound processingmore cumbersome. A head-tracking device and mod-eling of human spatial hearing are necessary for thisapproach. The main cues for lateral directional hear-ing are the interaural level and time differences (ILDand ITD, respectively). These alone don’t resolve ele-vation or the front-back confusion,10 so a more complexhead-related transfer function (HRTF) is necessary. Itmodels the reflections and filtering by the listener’shead, shoulders, and pinnae (part of the external ear),which provide more elaborate directional cues. We canmeasure the HRTFs on human subjects or dummyheads or derive them from mathematical models. Bin-aural reproduction techniques using digital filtersderived from HRTFs are widely applied today, and theyprovide quite natural 3D sound, despite the fact thathead and pinnae dimensions vary between individuals.In interactive situations, head movements also helpresolve the perceived sound’s direction.

Besides sound direction, the distance of a virtualsound source should also be audible. However, render-ing this cue isn’t straightforward because the sensationof distance results from many environmental cues simul-

taneously, including reverberation, attenuation of highfrequencies caused by air, and the relative movementbetween the source and the listener.

Interactive sound renderingWe capture the essential information of sound prop-

agation from a source to a listening point in an impulseresponse that contains all the information about thesound source’s radiation and a room’s reverberation.The most straightforward way to auralize this responseis to convolve it with the stimulus signal, usually ane-choic sound (free from echoes and reverberation). Inthis way, we add the sound propagation and reflectioninformation to the sound—that is, render the soundthrough the modeled space.

A single impulse response doesn’t contain informa-tion about the sound’s incoming directions. In addition,an impulse response describes only sound propagationfrom one source point to a single listening point. Forexample, if the listener moves, the whole impulseresponse changes. For these reasons, the direct convo-lution approach is impractical for creating dynamic vir-tual auditory environments.

In dynamic sound rendering, we parameterize theimpulse response used in convolution for two reasons:to make dynamic rendering possible and save compu-tational load. In this article, we present a general three-level approach to the parameterization (see Figure 2).To give a practical example, we describe how we imple-mented it in the DIVA auralization system.2 The threelevels of parametrization are

Virtual Worlds, Real Sounds

52 July/August 2002

Auralization

Image-sourcemethod

Real-time processing Nonreal-time analysis

Direct sound andearly reflections

Room acoustic simulation

Simulations Measurements

Acoustical attributesof the room

Artifical latereverberation

10.5

0−0.5

−10 50 100 150

Room geometrymaterial data

2 The threeparametrizationlevels andhybrid methodfor real-timesound render-ing. We derivethe roomimpulseresponse (lowerright) fromimage sources(red) and statis-tical reverbera-tion (blue).

� defining the scene,� calculating sound propagation and reflections in the

space, and� auralization and audio signal processing.

Defining a sceneSimilar to graphics, a scene’s acoustic model contains

3D geometry. Typically, polygonal modeling is sufficientand the number of surfaces is much less than in graph-ics rendering. Each polygon is associated with a mater-ial described by absorption and diffusion coefficients.These factors depend on frequency and direction, butin practice, we only give them direction independentlyin octave bands. This established custom in acoustics iscaused by the impractical and laborious measurementof direction-dependent coefficients. In addition, themodel contains location and orientation of soundsources and the listener. Both the sources and listenerhave directivity characteristics, but there’s no commonpractice for describing them.

The sidebar “Audio Scene Description APIs” brieflyexplains one possible way to use 3D audio scene descrip-tion languages to hierarchically organize the necessary(acoustical and architectural) data to build up a scene’stextual representation.

Real-time room acoustic simulationSeveral wave-based and ray-based methods calculate

sound propagation and reflections in a space. For real-time simulation, ray-based methods are widely used.The image-source method7 is especially suitable for cal-culating early reflections, but we must model the latereverberation separately due to the exponential growthof the computational load when tracing higher-orderimage sources. A common assumption in acoustics isthat the late reverberation is diffuse (direction inde-pendent) and exponentially decaying. This lets us usecomputationally efficient recursive filter structures. Wecan control these with statistical late reverberation para-meters such as reverberation time and energy.

The image-source method is a ray-based method whereeach reflection path is replaced by an image source. Wefind the image sources by reflecting the original sourceagainst all surfaces. The reflecting process can be recur-sively continued to find higher-order image sources. In atypical room geometry, most of them are invalid becausethe reflection path isn’t realizable. In addition, some ofthe image sources are invisible to the listening pointbecause of occlusion. Thus, a visibility check similar toculling in computer graphics is needed. (We use ray cast-ing in our DIVA implementation.) For each image source,the reflection path from the source to the receiver throughall the reflecting surfaces is reconstructed inside the 3Dmodel. Then possible occlusions (intersections with othersurfaces) are calculated and checked that all reflectingpoints actually lie inside the reflecting surface’s bound-aries. To reduce the number of required intersection cal-culations, we use a spatial directory with adaptive spatialsubdivision and hash-based addressing.

The input data for the image-source calculation is theroom geometry and the material data. Based on thelocation and orientation of the sound sources and the

listener, the calculation provides information of audi-ble specular reflections. The parameters for each visibleimage source are

� order of reflection,� orientation (azimuth and elevation angles) of sound

source,

IEEE Computer Graphics and Applications 53

Audio Scene Description APIsResearchers have developed several application programming

interfaces (APIs) to help define 3D sound scenes, named audioscene description APIs. Here, we briefly describe commonprinciples in different 3D sound APIs and concentrate on the soundscene description interface of the MPEG-4 standard.1

Defining a room acoustic model includes writing the room’sgeometry in terms of polygon surfaces, associated with soundreflecting properties that depend on the materials. The modeldefinition also contains positions of the sound sources and possiblytheir directivity properties. The 3D sound APIs are designed tofacilitate the process of writing this data in a hierarchical andobject-oriented way so that information that intuitively belongstogether is grouped under the same sound scene object.

The MPEG-4 standard1 enhances modeling of 3D sound scenes.Unlike its predecessors, the MPEG-4 standard defines the codingand specifies the presentation of multimedia objects to the enduser. In this context, the standard includes a set of controlsenabling interactive and dynamic audio–visual 3D rendering. Thesespecifications are in the Systems part of MPEG-4, called BIFS(Binary Format for Scenes).

BIFS is an extension of VRML, with various nodes added to makethe specification more compatible with the goals of MPEG-4 (suchas playing streamed audio and visual content in virtual scenes andpresenting advanced 2D content). AudioBIFS is a set of BIFS nodesfor mixing and preprocessing sound streams before their playbackat the MPEG-4 decoder.2 The Sound node (familiar from VRML)adds sound sources in 3D scenes in defined positions and withsimple directivity patterns. For modeling sound propagation in 3Dspaces, the MPEG-4 standard adds a new set of nodes (calledAdvanced AudioBIFS) that describe the effect of the room, airabsorption, obstruction of sound caused by walls, and the Dopplereffect, in a manner described in this article.

MPEG-4 adopts both physical and perceptual approaches to 3Dsound environment modeling. By physical approach, we mean theroom acoustic modeling we present in this article. Acousticproperties are associated with the geometry objects. Theperceptual approach, on the other hand, relies on a set ofparameters that defines the perceived quality of a room acousticeffect. In this approach, these qualitative parameters characterize aroom impulse response that is rendered at an MPEG-4 decoder.The parameters are associated with each sound source separately,and they don’t depend on other objects in the scene.

References1. ISO/IEC 14496-1:1999, Coding of Audiovisual Objects, Int’l Organization

for Standardization, Geneva, Dec. 1999.2. E.D. Scheirer, R. Väänänen, and J. Huopaniemi, “AudioBIFS: Describing

Audio Scenes with the MPEG-4 Multimedia Standard,” IEEE Trans. Multi-media, vol. 1, no. 3, Sept. 1999, pp. 237-250.

� distance from the listener,� identification of all the materials involved in the

reflection path, and� incoming direction of the sound (azimuth and eleva-

tion angle in relation to the listener).

We calculate the parameters of late reverberationoffline, as Figure 2 illustrates. This lets us tune the rever-beration time and other reverberation features accord-ing to the modeled space.

In our hybrid model, the actual impulse response isn’tconstructed for the convolution (auralization) process.Instead, we use a signal-processing structure that mod-ifies the input signal as it would be modified in convo-lution. With the image-source method, we calculatethese parameters, with which we define the signal-processing parameters for the auralization module.

Auralization and signal processingImage-source calculation provides the auralization

parameters that are finally converted to signal-processing parameters. We use this two-level processbecause auralization parameters don’t need to be updat-ed for every audio sample. However, we must recalcu-late the signal-processing parameters on a sample bysample basis. For efficiency, we look them up from pre-calculated tables or create them by interpolating theauralization parameters. The practical update rate (forexample, 30 Hz) of auralization parameters depends onthe available computational power and maximum tol-erable latency.

We implement the final auralization process as a sig-nal-processing structure (see Figure 3). The appliedparameters are filter coefficients or the lengths of delays.Most of them are calculated beforehand and stored tosimple data structures that are easily accessed with theauralization parameters. For example, we calculatematerial absorption as follows. First, the absorptioncoefficients in octave bands are translated to reflectioncoefficients. Then, a digital filter, which covers the wholeaudible frequency range, is fitted to this data and usedin signal processing. For each material and materialcombination, one filter is calculated offline. In real-timeprocessing, the only information needed per one imagesource is the material’s identification, and the correctfilter is picked from a table.

The signal-processing structurein Figure 3 contains a long delayline, which we feed with anechoicsound to process it. The imagesource’s distance from the listenerdefines the pick-up point to the filterblocks T0 ... N(z) where N is the imagesource’s ID (N = 0 corresponds todirect sound). Blocks T0 ... N(z) mod-ify sound signal with the soundsource directivity filters, distancedependent gains, air absorption fil-ters, and material filters (not fordirect sound). The sound’s incom-ing direction is defined with blocksF0 ... N(z) containing directional fil-

tering or panning depending on the reproductionmethod. The superimposed outputs of the filters F0 ... N(z) are finally summed with the outputs of the latereverberation unit R, which is a complex recursive algo-rithm. Note that the filter blocks F0 ... N(z) produce bin-aural output. For other reproduction methods, we needdifferent filters.

The whole signal-processing structure containshundreds of filter coefficients and delay line lengths,so a detailed description isn’t possible in this article.Find more information in a previous article2 in whichwe also discuss signal-processing issues of dynamicauralization.

Assessing auralization qualityAcousticians traditionally judge the objective quality

of room acoustics with criteria calculated from impulseresponses. Almost all these attributes—for example,reverberation time and clarity—are based on energydecay times at certain frequency bands. Although thesecriteria aren’t very accurate, we can use them to predictthe room acoustics and design of concert halls.

The more relevant evaluation of auralization is basedon subjective judgments. Ultimately, the only thing thatmatters in auralization is that the produced outputsounds are plausible for the application under study.Acoustics is also a matter of taste and thus an overalloptimum quality might be difficult to define. In dynam-ic situations, the quality measurement is even harder todefine because generally accepted objective criteriadon’t exist. However, we can obtain subjective qualityjudgments with listening tests.

For example, we evaluated the quality of our DIVAauralization system by comparing measured and ren-dered responses and sounds. We chose an ordinarylecture room (dimensions 12 m × 7.3 m × 2.5 m, seeFigure 4), for which we performed both objective andsubjective evaluation. Figure 5 depicts an examplebinaural impulse response pair. The early parts of theresponses slightly differ, but they are close to eachother. To find out the audible differences, we also con-ducted listening tests.

The results were surprisingly good, and we noticedno perceived differences with sustained tonal sound sig-nals. However, transient-like signals, such as snare drumhits, were slightly different.

Virtual Worlds, Real Sounds

54 July/August 2002

Latereverberation

unit, RAnechoicsound

T1(z)

F1(z)

T0(z)

F0(z)

TN(z)

FN(z)

Out (left)

Out (right)

Delay line3 The DIVAauralizationsignal process-ing structure.

Future trends in sound renderingAlthough current auralization systems produce

natural sounding spatial audio and are quite plausi-ble, these systems apply simplified modeling meth-ods. One major defect in current real-timephysics-based auralization systems is the lack of dif-fraction. Recently, a few articles have reported aug-menting ray-based methods with diffraction. Forexample, Tsingos et al.11 presented one solution basedon Uniform Theory of Diffraction that’s well suited forcomplex geometries. Svensson et al.12 presentedanother method based on the exact Biot–Tolstoy solu-tion. However, these diffraction models aren’t yetapplicable in real time.

Wave-based methods aim at solving the wave equa-tion in a space. This approach seems promisingbecause diffraction and diffusion are automaticallyincluded in the simulation. From the auralizationpoint of view, the most promising wave-based methodis the 3D digital waveguide mesh.13 Mathematically,it’s a finite difference method, and the computation isdone in the time domain. The computational loadgrows in O(f4) with frequency (3D mesh plus sam-pling frequency), and therefore, with current com-putational capacity, the waveguide mesh simulationscan be done only at low frequencies. As an example,we’ve simulated a large hall with a stage having a totalvolume of approximately 13,000 cubic meters up to

IEEE Computer Graphics and Applications 55

4 (a) The photograph and (b) 3Dmodel of the lecture room studiedin evaluation of the DIVA auraliza-tion system. Note that for acousticrendering a much simpler model issufficient than for photorealisticvisual rendering.

(a) (b)

1.0

0.5

Am

plit

ude

0

−0.5

−1.0

Left ear canal signal

Measured, point r1

Right ear canal signal

Measured, point r1

1.0

0.5

Am

plit

ude

0

−0.5

−1.00 50 100

Time (ms)

Auralized reverb, point r1Auralized reflections, point r1

Auralized reverb, point r1Auralized reflections, point r1

0 50 100

Time (ms)

50 5000

0

0

100 100

1.0

0.5

−0.5

−1.0

1.0

0.5

−0.5

−1.0

Time (ms) Time (ms)

Am

plit

ude

Am

plit

ude

5 The measured (top) and rendered (bottom) binaural impulse responses of the lecture room in Figure 4.

500 Hz. Figure 6 illustrates the sound field in the hallat 63 ms after an excitation impulse. The surface pre-sents the sound pressure at a height of 1.5 m abovethe floor while the excitation was on the stage.

There has also been much progress in the roomacoustic modeling and auralization during the pastdecade. Commercial room acoustic modeling systemsexist, and computers are in everyday use in the designof acoustically challenging spaces such as concert hallsand music studios. However, a real challenge is model-ing dynamic environments in which we can interactivelychange everything, including the geometry and mate-rials. Such a tool could be practical for architects andother designers. Although accurate sound renderingmethods exist already, they are still computationally toolaborious. As computers get faster and algorithms moreefficient, we gradually approach the ultimate goal ofreal-time simulation of sound and light behavior. �

AcknowledgmentsThis work has been partly financed by the Academy

of Finland through the Helsinki Graduate School inComputer Science, the Pythagoras Graduate School,and project funding.

References1. T. Takala and J. Hahn, “Sound Rendering,” Computer

Graphics (Proc. Siggraph 92), vol. 26, no. 2, ACM Press,New York, 1992, pp. 211-220.

2. L. Savioja et al., “Creating Interactive Virtual Acoustic Envi-ronments,” J. Audio Eng. Soc., vol. 47, no. 9, Sept. 1999, pp. 675-705.

3. M. Kleiner, B.-I. Dalenbäck, and P. Svensson, “Auraliza-

tion: An Overview,” J. Audio Eng. Soc., vol. 41, no. 11, Nov.1993, pp. 861-875.

4. J.-M. Jot, “Real-Time Spatial Processing of Sounds forMusic, Multimedia and Interactive Human–ComputerInterfaces,” Multimedia Systems, vol. 7, no. 1, Jan./Feb.1999, pp. 55-69.

5. A. Appel, “Some Techniques for Shading Machine Ren-derings of Solids,” AFIPS 1968 Spring Joint ComputerConf., Thompson Books, Washington, D.C., vol. 32, 1968,pp. 37-45.

6. A. Krokstad, S. Strom, and S. Sorsdal, “Calculating theAcoustical Room Response by the Use of a Ray TracingTechnique,” J. Sound and Vibration, vol. 8, no. 1, 1968, pp.118-125.

7. J.B. Allen and D.A. Berkley, “Image Method for EfficientlySimulating Small-Room Acoustics,” J. Acoustical Soc. ofAmerica, vol. 65, no. 4, Apr. 1979, pp. 943-950.

8. V. Pulkki, “Virtual Sound Source Positioning Using VectorBase Amplitude Panning,” J. Audio Eng. Soc., vol. 45, no.6, Jun. 1997, pp. 456-466.

9. M. Gerzon, “Periphony: With-Height Sound Reproduc-tion,” J. Audio Eng. Soc., vol. 21, no. 1-2, Jan./Feb. 1973,pp. 2-10.

10. J. Blauert, Spatial Hearing: The Psychophysics of HumanSound Localization, 2nd ed., MIT Press, Cambridge, Mass.,1997.

11. N. Tsingos et al., “Modeling Acoustics in Virtual Environ-ments Using the Uniform Theory of Diffraction,” Comput-er Graphics (Proc. Siggraph 2001), ACM Press, New York,2001, pp. 545-552.

12. U.P. Svensson, R.I. Fred, and J. Vanderkooy, “Analytic Sec-ondary Source Model of Edge Diffraction Impulse Respons-es,” J. Acoustical Soc. of America, vol. 106, no. 5, Nov. 1999,pp. 2331-2344.

13. L. Savioja and V. Välimäki, “Interpolated 3-D Digital Wave-guide Mesh with Frequency Warping,” Proc. 2001 IEEE Int’lConf. Acoustics, Speech, and Signal Processing (ICASSP 01),IEEE Press, Piscataway, N.J., vol. 5, 2001, pp. 3345-3348.

Tapio Lokki is a PhD student in theLaboratory of TelecommunicationsSoftware and Multimedia at theHelsinki University of Technology,Finland. He has a MSc in electricalengineering from the Helsinki Uni-versity of Technology. His research

interests include room acoustic modeling, auralization,and virtual reality.

Lauri Savioja is an acting profes-sor in the Laboratory of Telecommu-nications Software and Multimediaat the Helsinki University of Technol-ogy, Finland. He has a PhD in com-puter science from the HelsinkiUniversity of Technology. His

research interests include virtual reality, room acoustics,and human–computer interaction.

Virtual Worlds, Real Sounds

56 July/August 2002

6 A visualization of a 3D waveguide mesh simulation in a simplified con-cert hall model.

Jyri Huopaniemi is a researchmanager at Nokia Research Center’sSpeech and Audio Systems Labora-tory in Helsinki, Finland. He has aPhD in electrical engineering fromthe Helsinki University of Technolo-gy. His professional interests include

3D sound and interactive audio, digital audio signal pro-cessing, virtual audio–visual environments, middleware,and multimedia.

Riitta Väänänen is a PhD studentin the Laboratory of Acoustics andAudio Signal Processing at theHelsinki University of Technology.She has a MSc in electrical engineer-ing from the Helsinki University ofTechnology, Finland. Her research

interests include room reverberation modeling and param-etrization of 3D sound in virtual reality systems. She par-ticipated in the MPEG-4 standardization process from1998 to 2001 and spent 2001 at IRCAM (Institute forResearch and Coordination of Acoustic and Music), Paris,as a visiting researcher.

Tapio Takala is a professor ofinteractive digital media in the Lab-oratory of Telecommunications Soft-ware and Multimedia at the HelsinkiUniversity of Technology, Finland. Heis currently on a leave of absence,working for interaction development

at Nokia Ventures Organization. He has a PhD in comput-er science from the Helsinki University of Technology. Hisinterests range from modeling and design theory to inter-active multimedia performance, including artificial lifesimulations. His research group specializes in sound for VR.

Readers may contact Tapio Lokki at the Helsinki Uni-versity of Technology, Telecommunications Software andMultimedia Laboratory, PO Box 5400, FIN-02015 HUT,Finland, email Tapio.Lokki@hut.fi.

For further information on this or any other computingtopic, please visit our Digital Library at http://computer.org/publications/dlib.

IEEE Computer Graphics and Applications 57

January–March: Advances in MultimediaMultimedia means different things to differentcommunities. For researchers, it might bedatabases, search engines, or indexing tools,whereas content providers might be moreconcerned with streaming audio and video,compression techniques, and contentdistribution methods. This issue will offervarious viewpoints, practices, evolvingstandards, and innovative projects inmultimedia.

April–June: Content-Based MultimediaIndexing and Retrieval

Important research areas in multimediaindexing include audio, video, image, textual,and information retrieval. This special issue willcover the state of the art in multimediaindexing, especially image indexing, videoindexing, user access and annotation,description of semantic content, andapplications.

July–September: Multimedia R&DMultimedia systems and applications involve abroad range of topics, including hardware andsoftware for media compression, mediastorage/transport, data modeling, andabstractions to embedded multimedia inapplication programs. Even with this widecoverage, multimedia is still spreading itsinfluence to nontraditional professional sections.

October–December: Multimedia TrendsLearn more about the latest trends inmultimedia and explore what researchers aredeveloping for the next generation ofmultimedia applications. Find out whatpractitioners have learned in the field and whatthey plan to do next to improve the form andfunction of tomorrow’s multimedia.

2002 Editorial Calendar