7/30/2019 INTERACTIVESONIFICATION10.1007-s12193-012-0095-7

1/2

J Multimodal User Interfaces (2012) 5:8586

DOI 10.1007/s12193-012-0095-7

E D I T O R I A L

Interactive sonification

Roberto Bresin

Thomas Hermann

Andy Hunt

Published online: 20 April 2012

OpenInterface Association 2012

In October 2010, Roberto Bresin, Thomas Hermann and

Andy Hunt launched a call for papers for a special issue onInteractive Sonification of the Journal on Multimodal User

Interfaces (JMUI). The call was published in eight major

mailing lists in the field of Sound and Music Computing

and on related websites. Twenty manuscripts were submit-

ted for review, and eleven of them have been accepted for

publication after further improvements. Three of the papers

are further developments of works presented at ISon 2010

Interactive Sonification workshop. Most of the papers went

through a three-stage review process.

The papers give an interesting overview of the field of

Interactive Sonification as it is today. Their topics include

the sonification of data exploration and of motion, a newsound synthesis model suitable for interactive sonification

applications, a study on perception in the everyday periphery

of attention, and the proposal of a conceptual framework for

interactive sonification.

R. Bresin (

)KTH Royal Institute of Technology, Stockholm, Sweden

e-mail: roberto@kth.se

T. Hermann

Ambient Intelligence Group, CITEC, Bielefeld University,

Bielefeld, Germany

e-mail: thermann@techfak.uni-bielefeld.de

A. Hunt

Department Electronics, University of York Heslington,

York YO10 5DD, UK

e-mail: andy.hunt@york.ac.uk

1 Data exploration

This is the traditional topic of Interactive Sonification, and

is reflected by the number of papers in this category, five out

of eleven. In an interesting study Grond and Hermann apply

voice synthesis (vowel sounds) for the sonification of mathe-

matical functions. In another work, Ferguson and co-authors

discuss the interaction design for multi-touch computing for

exploring and interacting with representations of time-series

data simultaneously in both the visual and auditory modali-

ties. Metatla and co-workers focus on data exploration, and

present an approach to designing hierarchy-based sonifica-

tion for supporting non-visual interaction with relational di-

agrams. In another study Ness and colleagues propose the

use of a multimodal tangible interface that allows the users

to explore data in both time and space dimensions while re-

ceiving immediate sonic feedback of their actions. This in-

terface is applied to phenology, the study of periodic biolog-

ical processes, and can be used to explore the effects of cli-

mate change. Sonification of the environment for delivering

location-based information to mobile users is the challenge

faced by El-Shimy and collaborators in their research work.

They propose a system that allows for increased awareness

of the environment for users with limited vision capabili-

ties or whose visual attention is otherwise occupied in other

tasks.

2 Motion

One of the papers which presents a follow-up study of work

presented at ISon 2010 is the work by Dubus in which he

presents an evaluation of four sound models for the sonifi-

cation of elite rowing. The sonified data were those of the

movement (speed and acceleration) of a single scull rowing

mailto:roberto@kth.semailto:thermann@techfak.uni-bielefeld.demailto:andy.hunt@york.ac.ukmailto:andy.hunt@york.ac.ukmailto:thermann@techfak.uni-bielefeld.demailto:roberto@kth.se

7/30/2019 INTERACTIVESONIFICATION10.1007-s12193-012-0095-7

2/2

86 J Multimodal User Interfaces (2012) 5:8586

boat. Results show a good ability of athletes to efficiently ex-

tract basic characteristics of the sonified data, and highlight

the important issue of aesthetics in interactive sonification

design. In another paper dedicated to motion, Varni and col-

leagues present three interactive sonification models of the

synchronisation of gestures between two people each shak-

ing a mobile phone. Interactive sonification of their hand

movements helped users to keep synchronised with eachother. In a study on the sonification of everyday actions,

e.g., pressing a button on an ATM machine, Susini and co-

workers found that the level of usability (low vs. high) of the

user interface affects the choice of sounds that best deliver a

sense of naturalness of the interaction.

3 Sound synthesis

The representation of continuous processes in interaction

and interface design often uses liquid metaphors, such as

dripping or streaming of fluids. In an original work Dri-

oli and Rocchesso present a physics-based sound synthesismodel of liquid phenomena suitable for interactive sonifica-

tion of this class of processes.

4 Perception

In a follow-up study of their work presented at ISon 2010

Bakker and colleagues present a qualitative study on the ev-

eryday periphery of attention. They found that sound plays

a major role, supporting their approach to use interactive

sonification as an interaction style for peripheral interac-

tion.

5 Conceptual framework

In a further development of their study presented at ISon

2010, Diniz and co-authors present in more detail their the-

oretical foundations combining gestalt-based electroacous-

tic composition techniques, user body-centred spatial explo-

ration and mediation technology for the definition of a con-

ceptual framework for interactive sonification.

6 Concluding comments

The diversity of topics brought together in this special is-

sue demonstrates the richness and interdisciplinary of re-search in Interactive Sonification. The guest editors hope

that the articles will provide inspiration and trigger new

cross-discipline ideas.