icad-01, espoo, finland, july 29 - august 1, 2001 sonic browsing: an auditory tool for multimedia...

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Sonic Browsing: An Auditory Tool For Multimedia Asset Management

Mikael Fernström & Eoin Brazil

Interaction Design Centre

University of Limerick, IRELAND

www.idc.ul.ie

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Outline

• Introduction, Multimedia Asset Management

• Applications

• Sonic Browser

• Study results

• Conclusion

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Introduction

• how sound can be used to assist users working

with complex data sets.

• Searching and Browsing datasets of everyday sounds

• Visual & Aural representation

• Direct Sonification

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Intro: Multimedia Asset Management

• Multimedia Designers

• Large sets of everyday sounds used as datasets

• Idea -> use “cocktail party effect” to exploit ability to switch between sounds in auditory scene

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Applications

• Applications

• Collection of everyday sounds, multimedia authoring

• Copyright Issues

• Use by Multimedia Designers

• ...

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Sonic Browser - Concepts

• Screenscape and Soundscape• Visual, star-field type display

– shape, colour, size, and location

• Audio, direct music representation– drivers that support multiple stream audio

• Tight Coupling– Objects under the Aura are high-lighted and

played simultaneously

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Sonic Browser - Visual

Visual Representation:

• Star-field representation of auditory objects

• Arbitrary mapping, controlled by the user– shape, colour, size, horizontal and vertical

location

• Tunes playing are high-lighted.

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Sonic Browser - Audio

Audio: Direct representation of tunes (exploting the cocktailparty effect)

• Sounds are panned out in a stereo field controlled by the visual location of the tunes nearest to the cursor.

• The volume of the tunes playing concurrently is proportional to the visual distance between the objects and the cursor

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Sonic Browser – Screenshot

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Sonic Browser – Usage

• Cursor on object - the object plays, max volume, centered.

• Click - show dialogue box with details of the properties of the objects

• Double-click - access the object, e.g. show the details in the database, edit, etc.

• Single-click & Drag – move the object in display

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Study - Overview

• Design of interface elements and components of an interface when working with sound resources

• Examined differences in– Microsoft Windows™ Explorer– Sonic Browser

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Windows Explorer – Screenshot

ICAD-01, Espoo, Finland, July 29 - August 1, 20018

Users and Data set

• Postgraduates in the CSIS at UL

• Two commerical datasets– first set contained 54 sounds, used for

familiarisation. – second set contained 244 sounds, used in the

test phase

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Tasks

• Familiarisation – practice tasks

• Testing– similar way with a different sound data set and

tasks

• Post testing debrief & questionnaire

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Overview of Sound Areas

• The various categories of sound are shown below

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Sound Datasets

• Everyday Sound

• Musical Sound

Area of Interest

Start of Song

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Study Findings

• Hierarchical Layout helped navigation

• Long file names for files versus using cryptic short file names were more useful

• Aura allowed panning across files quickly

• Windows Explorer only has the ability to play one sound at a time

• Categorization differs between users

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Conclusion

• Straight hierarchy view does improve performance but can lead to confusion

• Use long descriptive file names

• Categorization problems can be avoid by use of an aura

• Areas / Regions of interest may not always be at the start of a sound, use of Cue points

ICAD-01, Espoo, Finland, July 29 - August 1, 2001

Future Work & Reference

• Development of a zoom facility• Addition of display by category• Inclusion of “tagging” or “shading” for files

of interest • Use of cue points to play sound

areas/regions of interest• Use of three property sliders for filtering• Website: http://www.idc.ul.ie