mcadams 1993 - intro to auditory cognition

Stephen McAdams, Emmanuel Bigand: Introduction to Auditory Cog... http://mediatheque.ircam.fr/articles/textes/McAdams93b/index-e.html

1 of 8 9/30/08 1:06 PM

Serveur IRCAM - CENTRE POMPIDOU 1996-2005.Tous droits rservs pour tous pays. All rights reserved.

Introduction to Auditory Cognition Stephen McAdams, Emmanuel Bigand

Thinking in Sound: The Cognitive Psychology of Human Audition, Oxford University Press, Oxford 1993

Copyright Oxford University Press 1993

(Version franaise)

The Mind sees and the Mind hears. The Rest is Blind and DeafEpicharmus 45O B.C.E.

[cited in Coren & Ward (1989)]

Preface:Auditory perception reveals a paradox similar to that encountered in all realms of perception: nothingseems more simple than to perceive the sound environment around us, and yet it is a phenomenon thatappears to rebel against scientific analysis. What difficulty exists, for example, in recognizing one'sname in a sentence pronounced by another person, in differentiating the noise of a car from that of anairplane, in perceiving the captivating rock rhythms of Bill Haley, in recognizing the voice of one's ownchild or the steps of a familiar person? All that seems to be needed is simply to `open one's ears'.Imagine for just an instant, however, the array of information and the set of procedures with which itwould be necessary to endow a computer in order for it to be able to distinguish a violin from a flute in amusical polyphony, to extract an alarm signal from the sonorous background within which it isembedded, to grasp the connection between a theme and a musical variation, to analyze by simplylistening to the noise of a car a problem with the motor, to detect an inhabitual noise in the beating of apatient's heart, and so on. In spite of the high level of technological sophistication available today, it isquite likely that these kinds of problems could only be resolved after many years of research in artificialintelligence and signal processing, since the amount of necessary information is so great and the ways itmust be combined are so complex.This difficulty in analyzing the process of listening underscores the great richness of information presentin the world of sound around us. The main objective of this book is to evaluate the current state of whatwe know about cognitive aspects of human audition. The book is primarily oriented toward students inthe cognitive sciences and scientists specializing in fields other than auditory psychology. Most of thematerial presented presumes at least an elementary knoweldge of acoustic terminology and of theconcepts and methods of experimental psychology. But before we move on to describe the contents ofthe volume, let us consider what there is of a cognitive nature in audition.

What is cognitive in audition?Etymologically speaking, the term `cognition' refers to the notion of knowledge. It has been used in amore specific sense to designate the conditions that allow humans to develop knowledge of the world. It


2 of 8 9/30/08 1:06 PM

almost goes without saying that no knowledge can be acquired in the absence of perceiving: in otherwords, no theory of knowledge is complete without a theory of its acquisition, and thus of perception.To emphasize the cognitive aspects of audition is thus primarily to remind us that auditory informationparticipates in a fundamental way in the development of knowledge.Of course the processes that are part and parcel of the perception of acoustic information can seem quitedifferent from those that intervene in more abstract intellectual activities (thinking, logic, reasoning,decision making, imagination, etc.). Two remarks need to be made on this point, however. First of all,the originality of the cognitive project is the desire to present an integrated picture of the ensemble ofintellectual processes, in making evident the continuity that exists between more elementary aspects ofthese activities (sensory information processing) and more abstract aspects (symbolic informationprocessing). The cognitive project therefore goes beyond the traditional division into independentintellectual functions: perception, memory, learning, language, intellection, etc. To emphasize thecognitive character of human audition is also to manifest the will to situate the processes of auditoryperception within this continuity.The second remark more directly concerns the mental processes implied in auditory perception. Theemphasis placed on the term `cognitive' in this volume highlights the fact that beyond the elementaryphases of processing, higher-level processes that bring into play mental representations, decisionmaking, inference, and interpretations by the perceptual system are necessary to elaborate a coherentrepresentation of the sound world.The postulate of a cognitive approach to perception is that the sensory information must be interpreted inorder to give rise to a coherent perception. Interpretation is necessary since the information contained inthe stimuli that reach the sensory organs is not always sufficient to form a coherent image of thesurrounding sound environment. In these cases, the perceptual system must represent and then compareauditory information that is not directly present at the sensory level. One of the main reasons for this isthat sound events are events that succeed one another in time: to perceive their structure requires theelaboration of a mental representation in order to be able subsequently to establish relations amongevents that are separated from one another by several minutes or even hours. Music offers a typicalexample here: how can we perceive the unity of a sound structure that develops over a very longtime-span (one and a half hours in the case of Beethoven's 9th Symphony) without elaboratingrepresentations of the sub-structures (thematic ones, for example) that are developed in the work?When the sensory data that are immediately available are found to be insufficient, the perceptual systemanalyzes the situation by taking into consideration knowledge that it has acquired of the surroundingsound world. Information from the environment does not, with the exception of newborns, stimulate acompletely nave organism. Acquired knowledge interacts with the current sensory data to interpret theauditory stimulation. Imagine for an instant that we are being guided through an Amazonian rain forest:we would hear exactly the same noise as the native of the region that accompanies us but we would beincapable, due to a lack of knowledge of the environment, not only to extract from the soundbackground events corresponding to the cries of iguanas and Macaques, the songs of Wistiti monkeys,or the rustling of the leaves of tropical trees, but we would also be unable to assign meanings to theentire sound structure that may in the long run be important for survival. In the same way, the GoldenEars of the French Navy have been trained in sonar detection to listen to what most of us would hear asa noisy underwater sound field and to perceptually segregate and identify a multitude of underwatersources such as clicking shrimp, whales, porpoises, schools of fish, and ocean-going vessels. Theyeven succeed in classifying such vessels as commercial or military, as surface or submarine, as diesel ornuclear, as Russian, American, or French. In particularly dramatic cases, an inability to perceptuallysegregate sound sources and recognize their nature could fail to lead a listener to deduce from thesesignals the presence of imminent danger: the appearance of a jaguar, or a clandestin submarine, forexample.In the case of sound structures like music, whose organization is highly determined by cultural rules, asimple observation of the information recorded at the level of the sensory organs does not suffice to


3 of 8 9/30/08 1:06 PM

explain the difference in our perception of a Mozart quartet or one by Beethoven. In many othersituations, the information available at the level of the sense organs may be too ambiguous to give rise toa unique perception of the current situation. Imagine, for instance, that we are in an old country house atnight: are the noises we perceive the result of the steps of an unwanted intruder seeking to enter thehouse, or of the creeking of old woodwork strained by the wind? The identification of the sources ofsound can be different according to the kinds of information to which we pay most attention: regularitiesin the noise or the timbral qualities of the materials being set into vibration. Faced with ambiguousstimuli of this kind, the perceptal system normally makes a subconscious decision in order to organizethe sound figure. The knowledge possessed by the listener certainly plays an important role at this level:in the preceding example, a child, an adult or the owner of the house would arrive at differentinterpretations of the same situation since they do not have the same auditory knowledge base.Auditory illusions are another situation that emphasizes the interpretive work performed by theperceptual system: nevertheless, in this case, the inference realized on the basis of the availableinformation is incorrect and results in the perception of an unreal sound object. Composers have knownfor centuries how to exploit these perceptual characteristics in order to create seductive sound figures.Bach's sonatas and partitas for violin are a remarkable example of auditory illusion: the listener perceivesquite clearly the presence of two violins playing in different pitch registers although there is actuallyonly one violinist rapidly alternating between registers. Or, by appropriately orchestrating severalinstruments playing the same melody in parallel at different pitches, Ravel in his piece Bolero succeeds in tricking the ear into hearing not the original set of familiar instruments, but a single, new `virtual'sound source with a marvelous, previously unheard timbral quality.

The current state of auditory cognition researchOur perception of the sound world thus greatly surpasses the quality of the sensory informationavailable at each instant: it results from mental processing. The study of cognitive aspects of informationprocessing by the different sensory modalities is currently making great progress in the cognitivesciences. Interest in higher-level processing is marked in the realm of vision by the publication of a largenumber of articles and books. The works of Marr (1982), Pinker (1984) and Humphreys & Bruce(1989) present important syntheses of theoretical principles and basic experimental data from visionresearch. In the realm of audition, the study of cognitive processes has been dealt with in publicationsthat are primarily limited to the understanding of spoken language. The range of subjects habituallycovered in such texts includes the following:

the perception of stimulus qualities or attributes and patterns (all senses),perceptual organization processes (usually confined to vision),perceptual categorization and perceptual constancy (confined to vision and speech),recognition and identification of objects, events, and patterns (confined to vision and speech),memory processes (vision and speech with occasional reference to audition),attentional processes (primarily visual although dichotic listening is often well-represented),development, learning, and skill acquisition (rarely touching upon audition),grammars of large-scale temporal structures (exclusively linguistic, almost never referring tomusic),problem solving and reasoning (never related to auditory problem solving as might be involved inmusical composition, for example).

Comparable volumes for non-verbal audition (e.g. Moore 1982; Warren 1982; Botte, Canvet, Demany& Sorin 1989) are limited to data on peripheral physiology and to psychoacoustic aspects of hearing;studies of cognitive processes are discussed only to a limited extent. The recent publications of Sloboda(1985), Dowling & Harwood (1986), Handel (1989), Dooling & Hulse (1989), and Bregman (1990),however, mark an extremely important turning point in addressing issues in audition that go beyond thesimple laboratory stimuli and listening situations that have been the primary domain of exploration inpsychoacoustics for over a century now. To our knowledge, no volume exists either in English or


4 of 8 9/30/08 1:06 PM

French that covers in a didactic fashion the whole range of subjects in the domain of auditory cognition.In order to rectify this deficit, a Tutorial Workshop was organized by the Hearing Group of the FrenchAcoustical Society. Specialists in various domains of auditory perception and cognition were invited toparticipate and to write chapters on their area of expertise.It is particularly important to note that the realm where much truly auditory (nonverbal) cognitiveresearch has taken place is music psychology. In the last 20 to 30 years this area has seen a substantialexpansion in the problems posed and the methodologies developed for exploring the musical mind.Most human listeners possess and deploy a high level of perceptual and cognitive sophistication forunderstanding, appreciating, and participating in musical activity. The primary interest of this area for thecognitive sciences is that musical systems are ubiquitous in the cultures of the world, they have attaineda degree of structural (i.e. grammatical) complexity that rivals that of language, and they can be learnedwithout explicit training at a very early age as is also the case with language. The importance of musicpsychology for auditory cognition in general is evidenced by the strong place it occupies in most of thechapters of this volume.One of the areas that has not received much attention, with the exception of some studies in youngchildren, is the understanding of the role of auditory cognitive processes and their integration with thoseof other sensory and more general cognitive systems in everyday activity, e.g. using acoustic cues fornavigating safely through a city environment, for manipulating and monitoring the function ofsophisticated machinery, or for evaluating the identity and significance of events in the environment thatare not within one's current field of vision. Research programs addressing some of these issues are sureto demonstrate a more important role for audition in everyday activity than has been granted to thissensory system by our more visually and linguistically oriented colleagues.

Organization of the bookThere are many ways to subdivide a field of research. The division adopted in this book comprises the following (overlapping) domains of auditory research--perceptual organization (Bregman), globalpattern perception (Warren), attentional processes (Jones & Yee), memory processes (Crowder),recognition (McAdams), neuropsychology (Peretz), music perception (Bigand), and developmentalpsychology (Trehub & Trainor). The main types of auditory processing dealt with in this book, and theirinteractions, are schematically summarized in Figure 1. Sound vibrations enter the inner ears where theyare analyzed and transduced into nerve impulses that are sent by way of the auditory nerve to the brain. (The process of transduction of vibrations into neural impulses is only briefly discussed in the chapterby McAdams.)


5 of 8 9/30/08 1:06 PM

[Figure 1]Auditory grouping processes effect the fusion and segregation of concurrent sound elements intoauditory events, as well as the temporal integration and segregation of successive sound events intoauditory streams. These processes are brought into play daily in many different ways. For example,sitting in front of the television you may suddenly hear the honking horn of a car in the street outside thewindow. Without the slightest hesitation this newly arrived noise will be heard as separate from andsuperimposed upon that of the television and not as forming a single sound object with whateverhappens to be sounding on the television at the moment. Such auditory grouping processes are the main


6 of 8 9/30/08 1:06 PM

focus of Bregman's chapter, but are also discussed in the chapters by Warren, Jones & Yee, Bigand, andTrehub & Trainor. According to the theory of auditory scene analysis, auditory grouping generally precedes the extraction or computation of perceptual properties or attributes; that is, attributes are the derived perceptual properties of the elements that have been grouped together. In Rimsky-Korsakov's The Flight of theBumblebee, for example, the notes follow in extremely rapid succession in such a way as to form asingle sound stream, but come too fast to be heard separately. What imposes itself on a listener'sperception is the overall form of the melodic line, anchored in time by the contour peaks and valleys aswell as the accented notes played by the trumpet player. The global perception of sequential soundpatterns and their temporal organization are discussed in the chapters by Warren and Jones & Yee.Once the perceptual qualities are represented in the auditory system, they can be interpreted with respectto evoked abstract knowledge structures. The result is the recognition and identification of, as well as the assignment of meaning or significance to events and sequences of events with respect to the local stimulus context and the previous experience of the listener. If you are cooking in the kitchen and suddenly hear a loud `kablam' in the dining room, auditory analysis of the heard noises allows you to identify a plate breaking on the floor, forks and knives tumbling around, the muffled sound of the dish of vegetables crashing, and a cat's surprised meow. From this analysis you succeed in attaching meaningto the whole scene, the cat was playing with the corner of the table cloth and pulled everything with itoff the table onto the floor. The psychological and neuropsychological processes underlying suchauditory recognition of sound sources, sound events, and sound sequences are reviewed in the chaptersby Crowder, McAdams, and Peretz. A more general review of specifically auditory aspects implicatedin memory processes is developed by Crowder. In adults, the perception of musical relations is in partconditioned by abstract knowledge structures that have been acquired in listening to the music of a givenculture. The form of these knowledge structures for Western tonal/metric music is delineated in thechapter by Bigand. In addition, the existence of precocious abilities that underly their development inchildren is reviewed by Trehub & Trainor, while the impairment of access to such structures inbrain-damaged patients is discussed by Peretz. The perceived relations among sound events can evoke a whole interpretive framework that affects theperception of subsequent events. This framework strongly influences the establishment of larger-scalestructural relations (of a hierarchical or associative nature). The processes that are responsible for thislevel of perception and comprehension are labeled event structure processing in Figure 1. Seated in a concert hall you await the arrival of the conductor: the musicians are warming up producing a greatsound mass. Listening attentively, you don't hear any relation between what each one is playing. Youdecide therefore that each player is individually rehearsing different parts of the score. Then the concertstarts. This time the events seem to follow one another with ease and you clearly perceive that the musicprogresses and develops through time. The exposition of the themes finishes and you then hear chordsthat are quite different from those in the preceding section. These changes in key announce the section inwhich the themes are developed. To grasp this progession supposes that each of the new sound eventscan be connected to the preceding events. In cases where the incoming sound material is completelyunknown to the listener, or does not correspond to known systems of structural relations (as would befound in the music of a given culture, for example), more primitive structuring procedures, such assegmentation into groups of events, may still operate directly on the relations among perceived surfaceattributes. The end result of event structure processing is the gradual elaboration of a mental representation of the structure of the sound environment currently being experienced by the listener.Studying the elaborationof a mental representation is important for cognitive psychology, since it is quite crucial for any sensorysystem whose input is entirely transitory, i.e. there are no persisting objects in audition, and informationabout acoustic events must be accumulated through time, attended to dynamically, and processed withrespect to previous experience while new information is being received. You may be following thevarious paths of musical development in a symphony, but you have established a mental representationof what happened at the beginning, which allows you to realize that at this moment the music is


7 of 8 9/30/08 1:06 PM

progressively moving toward the reexposition of the principal theme, and you understand that themovement is about to come to a close. The chapter by Bigand reviews the processes implicated in eventstructure processing and the building up of a mental representation of the structure of the sound environment. These processes are shown to influence the degree to which listeners are capable ofestablishing larger-scale relations and thus to appreciate musical form and the development of musicalideas. One of the important implications of event structure perception is that these structures serve as aframework having implications for the structure of acoustic information that has not yet arrived. By wayof the interpretive framework evoked from abstract knowledge structures, anticipations and expectationsmay be set up that orient the listener's attention to forthcoming events of a particular kind or at particulartimes.These attentional processes may also be simply affected by perceived regularities in the stimulusstructure. It is possible that attending itself may influence the activity of lower-level organizationalprocesses (and even the process of sensory transduction). This area of attending to auditory events andits implication for perception and action are developed in the chapter by Jones & Yee.AcknowledgementsThe Tutorial Workshop that resulted in the collective writing of this book was the fourth in a seriesorganized by the Hearing Group of the French Acoustical Society (S.F.A.). The first three Workshopscovered cochlear physiology (Aran, Dancer, Dolmazon, Pujol & Tran Ba Huy 1988), psychoacousticsand auditory perception (Botte, Canvet, Demany & Sorin 1989), and the central auditory nervoussystem (Roman, in press). The organization of the workshop and the preparation of the book were madepossible by generous financial or infrastructural support from the following organizations:

the Hearing Group of the S.F.A.,the program `Sciences de la Cognition' of the French Ministries of Research and Technology andof National Education,the Direction de Recherches, Etudes et Techniques of the French Ministry of Defense, andthe Institut de Recherche et Coordination Acoustique/Musique (IRCAM).

The Tutorial Workshop series was conceived and vigorously animated for several years by Marie-ClaireBotte (then President of the Hearing Group, and Vice President of the S.F.A.). Primarily due to theunflagging efforts of Dr. Botte, auditory research in France has seen a particularly remarkable change instatus over the last decade. It is thus with a profound gratitude, deep respect, and a great deal of affectionthat we dedicate this book to her.The chapters of this book were submitted to ad hoc peer review, each being reviewed by from two tofour referees in addition to the reviews made by the volume editors. Each co-editor ensured the reviewprocess for the other. We would like first of all to thank the authors for cooperating with us in this ratherarduous task. Secondly we would like to thank the reviewers who include Alain Lieury, Brian Moore,Bruno Repp, Charles Watson, Christopher Darwin, Earl Schubert, Eric Clarke, Francis Eustache, JayDowling, Jean-Franois Camus, Jose Morais, Josiane Bertoncini, Karen Yankelovich, Ken Robinson,Rachel Clifton, Robert Zatorre, Stanislas Dehaene, and Stephen Handel. Three reviewers chose not tohave their names listed. We would also like to thank Ccile Marin for preparing the subject and nameindexes and Carolyn Drake and the authors for their helpful comments on the glossary.This book is being published in both English and French. The French edition will appear as Les aspects cognitifs de l'audition humaine, published by Presses Universitaires de France.S.M. & E.B.Paris, 8 February 1992


8 of 8 9/30/08 1:06 PM

ReferencesAran, J.-M., Dancer, A., Dolmazon, J.-M., Pujol, R. & Tran Ba Huy, P. (1988). Physiologie de lacochle. Editions INSERM/E.M.I., Paris.Botte, M.-C., Canvet, G., Demany, L. & Sorin, C. (1989). Psychoacoustique et perception auditive. Editions INSERM/E.M.I./S.F.A., Paris.Bregman, A. S. (1990). Auditory scene analysis: The perceptual organization of sound. MIT Press, Cambridge, Mass.Coren, S. & Ward, L. M. (1989). Sensation and perception, 3rd ed. Harcourt, Brace, Jovanovich, SanDiego.Dooling, R. J. & Hulse, S. H. (1989). The comparative psychology of audition: Perceiving complexsounds. L. Erlbaum Assoc., Hillsdale, N.J.Dowling, W. J. & Harwood, D. L. (1986). Music cognition. Academic Press, Orlando, Florida.Handel, S. (1989). Listening: An introduction to the perception of auditory events. MIT Press, Cambridge, Mass.Humphreys, G. W. & Bruce, V. (1989). Visual cognition: Computational, experimental, andneuropsychological perspectives. L. Erlbaum Assoc., Hillsdale, N.J.Marr, D. (1982). Vision: A computational investigation into the human representation and processingof visual information. W. H. Freeman, San Francisco.Moore, B. C. J. (1982). Introduction to the psychology of hearing, 3d. ed. Academic Press, London.Pinker, S. (ed.) (1984). Visual cognition. MIT Press, Cambridge, Mass. [reprinted from Cognition, vol. 18].Roman, R. (in press). Le systme auditif central: Anatomie et physiologie. Editions INSERM/E.M.I., Paris.Sloboda, J. A. (1985). The musical mind: The cognitive psychology of music. Clarendon Press, Oxford.Warren, R. M. (1982). Auditory perception: A new synthesis. Pergamon Press, New York.____________________________Server IRCAM-CGP, 1996-2008 - file updated on 06/20/1997 at 11h14m08s.

mcadams 1993 - intro to auditory cognition

Documents