Thesis Proposal

Song Yang

There is a growing interest in utilizing digital technologies to create new forms of musicalexpression. One field of development is multi-channel sound diffusion, which allows one to placevirtual sound sources within a spherical listening space, creating an effect known as periphonic(three-dimensional) sound. Through this technique, one can create the illusion of moving soundsources [1].

Background

Recent research has been directed at providing performers with real-time control of sounddiffusion [2]. Current control methods involve capturing the performer’s physical gestures viaprosthetic devices [3,4,5] or motion-tracking cameras [6] and translating these gestures intosound trajectories. Pressure sensors installed below concert hall floors have been used tomeasure shifts in the performer’s balance, which are subsequently used to direct sounds [7].

Motivation

Despite the control offered by these solutions, numerous drawbacks exist. Prosthetic devicesphysically encumber the performer, reducing the degree of virtuosity they can exhibit.Motion-tracking cameras must be customized due to the different physical movements requiredfor each instrument, and modifications of concert environments are costly and time-consuming.Therefore, existing control systems employing visual/tactile cues are inflexible, and moreadaptable methods are required.

Objective

This thesis proposes to design and implement a control system for sound diffusion based solelyon auditory cues from the performer. The completed project will be a computer programcomprising of three sub-systems: 1. a machine listening interface which records the performerand identifies, in real-time, sound parameters such as dynamic envelope, entry delay, pitch anddensity; 2. an algorithm which maps these sound parameters to spatial parameters, such asorigin, trajectory and velocity; and 3. a multi-channel sound diffusion program.

Methodology

Due to the limited project timeframe, the thesis will focus on the second and third subsystems.The program will be implemented using SuperCollider, a C-family language for audio synthesis,audio control and algorithmic composition [8]. It will take as input a sound element and itsparameters. The mapping algorithm will then generate an appropriate spatial trajectory forquadrophonic (four-channel) output. Ambisonics, a signal encoding/decoding technique, will beused to achieve three-dimensional sound diffusion, due to its relatively low hardwarerequirements and comparatively high degree of localization detail [9,10].

[1] J. Chowning, ”The Simulation of Moving Sound Sources”, Computer Music Journal, vol.1, no. 3, pp. 48-52, 1977.

[2] D. Salvati, S. Canazza and A. Roda, ”Sound Spatialization Control by Means of AcousticSource Localization System”, in Proceedings of the IEEE International Conference onSystems, Man, and Cybernetics, Anchorage, AK, USA, 2011.

[3] M. Marshall, J. Malloch and M. Wanderley, ”Gesture Control of Sound Spatialization forLive Musical Performance”, Lecture Notes in Computer Science, vol. 5085, pp. 227-238,2011.

[4] J. Schacher, ”Gesture Control of Sounds in 3D Space”, in Proceedings of the InternationalConference on New Interfaces for Musical Expression, New York, NY, USA 2007.

[5] M. Wanderley and M. Battier, ”Trends in Gestural Control of Music”, IRCAM, Paris,France, 2000.

[6] M. Marshall, J. Malloch and M. Wanderley, ”Non-Conscious Gesture Control of SoundSpatialization”, in Proceedings of the ENACTIVE Conference, Grenoble, France, 2007.

[7] L. McElligott, E. Dixon and M. Dillon, ”Processing the Effort Generated by Levels ofExpressive Gesturing in Music”, in Proceedings of the International Conference on NewInterfaces for Musical Expression, Singapore, Singapore, 2002.

[8] S. Wilson, D. Cottle and N. Collins, The SuperCollider Book, Cambridge, MA, USA: MITPress, 2011.

[9] F. Hollerweger, ”Periphonic Sound Spatialization in Multi-User Environments”, Universityof McGill, Montreal, Canada, 2006.

[10] D. Malham and A. Myatt, ”3-D Sound Spatialization using Ambisonic Techniques”,Computer Music Journal, vol. 19, no. 4, pp. 58-70, 1995.