the impact of cultural context on the perception of sound and musical language (and its relevance...
DESCRIPTION
Keynote talk at the Lisbon Consortium event (http://lisbonsummerschool.wordpress.com)TRANSCRIPT
The impact of Cultural Context on the Perception of Sound and Musical Language
(and its relevance for the development of Computational Models of Audition)
The Lisbon Consortium
Luís Gustavo Martins [email protected]
Lisbon, June 26th 2013
CATÓLICA'PORTO+ARTES+
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
CATÓLICA'PORTO+ARTES+
Outline • Human listening comprises many layers of information analysis and processing.
• From the transduction of low-level auditory stimuli in the ears… • … to their organization in the brain,
• higher-level factors take an important role in the auditory process: • Memory • Attention • Experience • Culture • Context • …
• As a result, listening in general is influenced by a large number of variables interacting, cooperating and competing with each other in a rather complex network.
• Recent research in the field of computational sound and music analysis is now more aware of the impact of the cultural background in the perception of sound and music.
CATÓLICA'PORTO+ARTES+
Can Culture and Context affect our Perception? Sensation
• neurological process by which we become aware of our environment (Jandt, 2012).
• You are not directly aware of what is in the physical world… • … but, rather, of your own internal sensations.
• Is variation in human sensation attributable to culture? Source: h"p://www.sagepub.com/upm1data/45975_Chapter_3.pdf=
CATÓLICA'PORTO+ARTES+
Can Culture and Context affect our Perception?
CATÓLICA'PORTO+ARTES+
Can Culture and Context affect our Perception? “Kids are encouraged to have fun outdoors, playing in the nature.”
CATÓLICA'PORTO+ARTES+
Can Culture and Context affect our Perception? “Kids are encouraged to have fun outdoors, playing in the nature.”
CATÓLICA'PORTO+ARTES+
Can Culture and Context affect our Perception? “Unattended kids may become extremely violent when playing outdoors!”
CATÓLICA'PORTO+ARTES+
Can Culture and Context affect our Perception? “Unattended kids may become extremely violent when playing outdoors!”
CATÓLICA'PORTO+ARTES+
Can Culture and Context affect our Perception?
Dogs in the Western Culture
Source: http://www.sagepub.com/upm-data/45975_Chapter_3.pdf
PART 1 CULTURE AS CONTEXT FOR COMMUNICATION66
InterpretationThe third step in the perception process is interpretation. This refers to attaching meaning to sense data and is synonymous with decoding. The same situation can be interpreted quite differently by diverse people. A police officer arriving at a crime scene can be experienced by the victim as calming and relief giving but by the criminal as fearsome and threatening.
Here, too, the effect of culture is great. As you encounter people of your own culture, you constantly make judgments as to age, social status, educational background, and the like. The cues you use to make these decisions are so subtle that it’s often difficult to explain how and why you reach a particular conclusion. Do people in the United States, for example, perceive tall men as more credible? Perhaps.
Applying these same cues to someone from another culture may not work. People in the United States, for example, frequently err in guessing the age of Japanese individuals, such as judging a Japanese college student in her mid-20s to be only 14 or 15.
Dogs as Pets or as Food
The meanings you attach to your perceptions are greatly determined by your cultural back-ground. Think of how speakers of English categorize life. Most probably use the categories of human life and animal life. Now think of how you typically categorize animal life—probably into wild animals and domesticated animals. Now think of how you typically categorize domesticated animal life—probably into animals used for food, animals used for sport and recreation, and pets. Look at the picture of the puppy and capture your feelings.
Most of us see this puppy in the category of pet, for which we have learned to relate warm, loving feelings. Puppies are cute, cuddly, warm, loving creatures. Now look at the next picture
of a man holding up a dog, read the caption, and capture your feelings. Most of us who love dogs find this picture uncomfortable and disgusting. How can people eat dogs? They are pets, not food! It all depends on where you categorize them. Dogs are pets in some cultures and food in others. In the Arab world, dogs are acceptable as watchdogs and as hunting dogs but are not kept in the home as pets because they are seen as unclean and a low form of life. To call someone a dog is an insult among Arabs. People in most cultures have strong ideas about which foods are acceptable for human consumption and which are not. People in some countries think the custom in the United States of eating corn on the cob is disgusting because that food is fit only for pigs. Some Ukrainians like to eat salo, raw pig fat with black bread and vodka, which might cause nausea in some, as would knowing that horse meat from California is served in restaurants in Belgium, France, and Japan.
Many consider dogs as pets. (The author’s first dog, Smokey.)
CATÓLICA'PORTO+ARTES+
Can Culture and Context affect our Perception?
Dogs in the Eastern Culture
Source: http://www.sagepub.com/upm-data/45975_Chapter_3.pdf
CHAPTER 3 Culture’s Influence on Perception 67
Your reaction of disgust to the picture is a culturally learned interpretation—and that inter-pretation can be quite strong. In 1989, California made it a misdemeanor for any person to sell, buy, or accept any animal traditionally kept as a pet with the intent of killing the animal for food. More recently, animal rights groups have protested the sale of live animals, such as turtles, frogs, lobsters, crabs, fish, and chicken, for food at Asian-American markets. Asian tradition is that fresh meat is tastier and more healthful, that the best meat “enters your house still breathing.” Animal rights activists contend that the animals are treated inhumanely in the shops and are killed in ways that cause them unnecessary pain. Asian-American groups argue that eating dogs and cats is an extreme rarity among Southeast Asian immigrants and call the law and the animal rights activists racist.
In some cultures, parts of some animals are categorized as medicine. In other cultures, certain animals are considered sacred and certainly would not be eaten. The Hindu elephant-headed God Ganesh is accompanied by a rat whenever he travels. Rats, like cows, are deified in India. No Hindu worship is complete without an offering to Ganesh and his companion, the rat. Rats are fed and rarely killed in India.
Weather Vane as Christian Cross
The examples so far have been of practices that could offend some English speakers. Let’s turn that around with an example of what speakers of English do that could be offensive to others. Johnston Pump Company, a U.S. company now based in Brookshire, Texas, has been doing business with Saudi Arabia for more than 70 years. By the 1930s, Johnston Pump was well
Can you explain your feelings about this photograph? As China’s economy boomed and affluence spread, attitudes toward dogs changed. Traditional Chinese may have eaten dog meat because it was thought to improve blood circulation. Urban Chinese today are more likely to have dogs as pampered companions.
CATÓLICA'PORTO+ARTES+
Can Culture and Context affect our Perception? What do you (can you!) “see” here?
CATÓLICA'PORTO+ARTES+
Can Culture and Context affect our Perception? Humans (independent of their cultural context) develop, from early ages, a “model” for “the human face”... So maybe, you see a face...
CATÓLICA'PORTO+ARTES+
Can Culture and Context affect our Perception?
Or, maybe you know what a saxophone looks like, and how it is (usually) played...
CATÓLICA'PORTO+ARTES+
Can Culture and Context affect our Perception? What if you never left your native tribe, deep in the African savana? Can you still perceive something from the figure?
CATÓLICA'PORTO+ARTES+
Can Culture and Context affect our Perception? Or maybe you are a kid with strong “cultural” references from cartoons? (and a large imagination)
CATÓLICA'PORTO+ARTES+
Can Culture and Context affect our Perception?
• Nisbett (2003) has demonstrated that:
• humans sense and perceive the world in ways unique to their environments • by contrasting Eastern and Western cultures.
• Perception and thought are not independent of the cultural environment
• therefore, our brains are both shaped by the external world and shape our perception of the external world.
• Sensation is the neurological process of becoming aware of our environment and is affected by our cultures (Jandt, 2012).
Source: h"p://www.sagepub.com/upm1data/45975_Chapter_3.pdf=
CATÓLICA'PORTO+ARTES+
Sensing and Perceiving the World around us
A World of Sounds
CATÓLICA'PORTO+ARTES+
A World of Sounds Sound production and propagation
CATÓLICA'PORTO+ARTES+
A World of Sounds Sound propagation – waves of energy
CATÓLICA'PORTO+ARTES+
A World of Sounds Sound sensors: the Human Ear THE HUMAN AUDITORY SYSTEM
Figure 1: A sketch of the peripheral region of the human auditory system (adaptedfrom [Beranek, 1990]).
a pre-processing of the acoustical signal, the most notable aspect being frequency analysis.
The structure of the peripheral region is represented in Figure 1. Broadly, the periphery
can be divided into three areas: the outer, the middle and the inner ear. The outer ear
(pinna) collects sound energy from the surrounding environment and transmits it through
the ear canal (meatus) to the eardrum (tympanic membrane), giving rise to an oscillatory
motion. The pinnae play a role in spatial hearing, by imposing spectral characteristics on
sound that depend on its direction of incidence with respect to the head. In the middle
hear, the vibrations in the eardrum are transmitted to the fluid-filled cochlea by means
of three tiny bones (the ossicles: the malleus, the incus and the stapes). They form a
mechanical lever system whose main purpose is to match the impedance of air to that of
the cochlear fluids and to protect the ear against aggressive sound intensities by modifying
the transmission gain. The middle ear is filled with air whose pressure is equalized with
that of the outside environment through the Eustachian tube. The inner ear corresponds
to the cochlea, the organ of hearing. It is a tube shaped like a snail shell, filled with fluid,
that is divided along its length by two membranes: Reissner’s membrane and the basilar
membrane. The basilar membrane varies in mass and sti↵ness along its length so that
di↵erent regions of the membrane vibrate at di↵erent resonant frequencies. As a result,
the basilar membrane exhibits a complex pattern of motion in response to sound-induced
13
CATÓLICA'PORTO+ARTES+
A world of Sounds Sound sensors: the Human Ear
CATÓLICA'PORTO+ARTES+
A World of Sounds Beyond sensing: higher order Auditory processes and their interactions
MUSIC LISTENING
ABSTRACT
KNOWLEDGE
STRUCTURES
EVENT
STRUCTURE
PROCESSING
EXTRACTION
OF
ATTRIBUTES
AUDITORY
GROUPING
PROCESSES
MENTAL
REPRESENTATION OF
SOUND
ENVIRONMENT
TR
AN
SD
UC
TIO
N
TR
AN
SD
UC
TIO
N
ATTENTIONAL
PROCESSES
Figure 2: The main types of auditory processing and their interactions (adaptedfrom [McAdams and Bigand, 1993]).
possible to extract perceptual attributes which provide a representation of each element in
the auditory system.
These attributes can now be interpreted with respect to evoked abstracted knowledge
structures that allow the recognition and identification of events and sequences of events,
as well as the assignment of meaning and significance. All this is performed at the light
of the local stimulus context and the past experience of the listener (see the discussion of
schema-based grouping in section 2.3.1).
These perceived relations among sound events can influence the perception of sub-
sequent events (and even revise past interpretations – e.g. the McAdams-Reynolds
oboe example [McAdams, 1984] or the “old-plus-new” organization principle described
in [Bregman, 1990, pp.261]). This event structure processing influences the establishment
of larger-scale structural relations.
The resulting elements from the event structure processing give rise to the progressive
creation of a mental representation of the sound environment involving the listener. Given
the transitory nature of the acoustic stimulus, information about acoustic events must
be somehow accumulated through time. This accumulated mental representation takes
a crucial part on the ability to dynamically focus attention to new incoming acoustic
16
CATÓLICA'PORTO+ARTES+
A World of Sounds Auditory Scene Analysis (ASA) - The lake analogy…
Source: http://www.cns.nyu.edu/~david/courses/perception/lecturenotes/localization/localization.html
CATÓLICA'PORTO+ARTES+
A World of Sounds Auditory Scene Analysis – The Cocktail Party Effect
Source: http://online.wsj.com/article/SB10001424052702303459004577361850069498164.html
CATÓLICA'PORTO+ARTES+
A World of Sounds Auditory Scene Analysis - Polyphonic, multitimbral, complex musical compositions
• Listeners can still identify:
• Melodic lines • Instruments (or groups of instruments playing) • Harmony • Beat • Rhythm • …
CATÓLICA'PORTO+ARTES+
Perception Principles
The Gestalt Theory of Perception
CATÓLICA'PORTO+ARTES+
Perception Principles
The Gestalt Theory of Perception
CATÓLICA'PORTO+ARTES+
Perception Principles
Auditory Scene Analysis (ASA) - the Perceptual Organization of Sound
http://webpages.mcgill.ca/staff/Group2/abregm1/web/
CATÓLICA'PORTO+ARTES+
Perception Principles Segregation of a melody from interfering tones
• Bottom-up VS Top-Down approaches
Source: http://webpages.mcgill.ca/staff/Group2/abregm1/web/downloadstoc.htm
“At the intermediate separations, familiarity with the melody can sometimes allow listeners to hear it out, but there is a still a tendency to hear runs of tones that group melodic and distractor notes.”
CATÓLICA'PORTO+ARTES+
Music Perception
How is Similarity defined, for Music? (And what impact has cultural context and background on its definition?)
Query:
Similar Music?
CATÓLICA'PORTO+ARTES+
Music Perception
How is Similarity defined, for Music? (And what impact has cultural context and background on its definition?)
“- This all sounds like Fado to my hears…” “- Not to me… I’m a Fado singer…I can tell a difference!”
=
CATÓLICA'PORTO+ARTES+
Music Perception
How is Similarity defined, for Music? (And what impact has cultural context and background on its definition?)
Saudades de Coimbra (Fado) - Verdes Anos
Amália Rodrigues - Fado Português
João Braga - Fado de Lisboa
“- This all sounds like Fado to my hears…” “- Not to me… I’m a Fado singer…I can tell a difference!”
=
CATÓLICA'PORTO+ARTES+
Music Perception How is Similarity defined, for Music? (And what impact has cultural context and background on its definition?)
“- This all sounds like Techno to my hears…” “ – No to me… I’m a DJ… I can tell a difference!”
CATÓLICA'PORTO+ARTES+
Music Perception How is Similarity defined, for Music? (And what impact has cultural context and background on its definition?)
“- This all sounds like Techno to my hears…” “ – No to me… I’m a DJ… I can tell a difference!”
The Big Fake Traffic Signs (ELECTRO)
Je T'aime (Digital Dog mix) Armand Van Helden (DEEP HOUSE)
Exorcise (A1) Umek (TRANCE)
Memories David Guetta (COMMERCIAL HOUSE)
CATÓLICA'PORTO+ARTES+
Music Perception “Music is different things and does different things in different cultures; the bundles of elements and functions which are music for any given culture may overlap minimally with those of another culture, even for those cultures where "music" constitutes a discrete and identifiable category of human activity in its own right.” (Cross, 2001)
http://techno.org/electronic-music-guide/
- Expert listeners in a genre, or in a given culture can perceive subtle variances in a music genre, and identify sub-genres, artist styles, etc.
- They make part of a “culture” around that musical genre / style
- “Music", like speech, is a product of both our biologies and our social interactions (Cross, 2001)
CATÓLICA'PORTO+ARTES+
Music Perception How is Similarity defined, for Music? (And what impact has cultural context and background on its definition?)
• It is an ill–posed question…
• It is multidimensional, may use different and simultaneous criteria, such as: • Pitch • Harmony • Timbre • Rhythm • Genre • Texture • …
• It may depend on the context of the query • Difficult to infer…
• It will certainly be culturally biased • Difficult to match expectations
CATÓLICA'PORTO+ARTES+
Effects of Culture on Music Perception • Perception of Pitch (Wong et al., 2012)
• Studies on Chinese and Canadian population with Amusia • Amusia: a neurogenetic disorder affecting the music (pitch and rhythm) processing • Contonese is a tone language, Canadian French and English are not tone languages
• Cantonese speakers as a group tend to show enhanced pitch perception ability compared to speakers of Canadian French and English.
• This enhanced ability occurs in the absence of differences in rhythmic perception and persists even after relevant factors such as musical background and age were controlled.
• These findings not only provide critical evidence for a double association of music and
speech, but also argue for the reconceptualization of communicative disorders within a cultural framework.
• Along with recent studies documenting cultural differences in visual perception, our auditory evidence challenges the common assumption of universality of basic mental processes and speaks to the domain generality of culture-to-perception influences.
Source: http://www.plosone.org/article/info:doi/10.1371/journal.pone.0033424
CATÓLICA'PORTO+ARTES+
Effects of Culture on Music Perception • Memory and Musical Expectation for Tones (Curtis and Bharucha, 2009)
• Explored how musical culture shapes one’s listening experience.
• Western participants heard a series of tones drawn from either: • the Western major mode (culturally familiar) • or the Indian thaat Bhairav (culturally unfamiliar)
• and then heard a test tone.
• They made a speeded judgment about whether the test tone was present in the prior series of tones. • False alarm rates were higher for test tones congruent with the major mode than for
test tones congruent with Bhairav. • In contrast, false alarm rates were lower for test tones incongruent with the major mode
than for test tones incongruent with Bhairav.
• These findings suggest that one’s internalized cultural knowledge may drive musical expectancies when listening to music of an unfamiliar modal system.
Source: http://openscholar.purchase.edu/meagan_curtis/files/curtisbharucha2009musicperception.pdf
CATÓLICA'PORTO+ARTES+
Effects of Culture on Music Perception • A cross cultural study on Music Mood Perception (Hu and Lee, 2012)
• People's cultural background is often assumed to be an important factor in music mood perception.
• This paper reports on a study comparing mood judgments on a set of 30 songs by American and Chinese people.
• Results show that mood judgments do indeed differ between American and Chinese respondents.
• Furthermore, respondents’ mood judgments tended to agree more with other respondents from the same culture than those from the other group.
Source: http://ismir2012.ismir.net/event/papers/535-ismir-2012.pdf
CATÓLICA'PORTO+ARTES+
Computational Approaches to Sound and Music Analysis
Typical Pipeline
Hearing Representation
Analysis Understanding
Acting Interaction
Signal Processing Feature Extraction
Machine Learning Analysis and Retrieval
Human Computer Interaction
CATÓLICA'PORTO+ARTES+
Computational Approaches to Sound and Music Analysis
Different perspectives involved
Source: h"p://mires.eecs.qmul.ac.uk/wiki/index.php/Roadmap=
TECHNOLOGICAL PERSPECTIVE
2.1 Musically
relevant data
2.2 Music
representationsModels
2.3 Data processing methdologies
2.4 Knowledge-driven
methdologies
2.5 Estimation of
elements related to musical concepts
Estimated musical
concepts
Other estimatedconcepts
2.6Evaluation
methodologies
SOCIO-CULTURAL PERSPECTIVE
4.1Social aspects
4.2Multiculturality
USER PERSPECTIVE
3.1User behaviour
3.2User interaction
EXPLOITATION PERSPECTIVE
5.1 Music distribution
applications
5.2Creative tools
5.3Other exploitation areas
CATÓLICA'PORTO+ARTES+
Computational Approaches to Sound and Music Analysis
Technological Perspective:
CATÓLICA'PORTO+ARTES+
Computational Approaches to Sound and Music Analysis
Challenge: how to include the Social-Cultural Perspective?
Source: h"p://mires.eecs.qmul.ac.uk/wiki/index.php/Roadmap=
TECHNOLOGICAL PERSPECTIVE
2.1 Musically
relevant data
2.2 Music
representationsModels
2.3 Data processing methdologies
2.4 Knowledge-driven
methdologies
2.5 Estimation of
elements related to musical concepts
Estimated musical
concepts
Other estimatedconcepts
2.6Evaluation
methodologies
SOCIO-CULTURAL PERSPECTIVE
4.1Social aspects
4.2Multiculturality
USER PERSPECTIVE
3.1User behaviour
3.2User interaction
EXPLOITATION PERSPECTIVE
5.1 Music distribution
applications
5.2Creative tools
5.3Other exploitation areas
CATÓLICA'PORTO+ARTES+
Computational Approaches to Sound and Music Analysis
Challenge: how to include the Social-Cultural Perspective? • Music Information Research (MIR) involves the understanding and modeling of
music-related data in its full contextual complexity.
• Music is a communication phenomenon that involves people and communities immersed in specific social and cultural context.
• MIR aims at processing musical data that captures the social and cultural context and at developing data processing methodologies with which to model the whole musical phenomenon.
Source: h"p://mires.eecs.qmul.ac.uk/wiki/index.php/Roadmap=
CATÓLICA'PORTO+ARTES+
Computational Approaches to Sound and Music Analysis
Challenge: how to include the Social-Cultural Perspective? • Music Computing research (IT in general) does not respond to the world's
multi-cultural reality.
• Data models, cognition models, user models, interaction models, ontologies, ... are culturally biased.
• Music information is more than CDs and metadata.
h"p://compmusic.upf.edu=;=h"p://mires.eecs.qmul.ac.uk/wiki/index.php/Roadmap= ==
CATÓLICA'PORTO+ARTES+
Computational Approaches to Sound and Music Analysis
Challenge: how to include the Social-Cultural Perspective? • Promote a multicultural approach to Music Computing research.
• Advance in the formalization and description of music, making it more accessible to computational approaches.
• Reduce the gap between audio signal descriptions and semantically/culturally meaningful music concepts.
• Develop information modelling techniques applicable to different music repertories.
• Develop computational models to represent culture specific music contexts.
h"p://compmusic.upf.edu=;=h"p://mires.eecs.qmul.ac.uk/wiki/index.php/Roadmap= ==
CATÓLICA'PORTO+ARTES+
compmusic
Music information modeling
Sound and Music Computing
Music Information Processing
Computational Musicology
Ontologies
Cognition models
Interaction models
Data models
User models
Computational Approaches to Sound and Music Analysis
Challenge: how to include the Social-Cultural Perspective?
h"p://compmusic.upf.edu==
CATÓLICA'PORTO+ARTES+
References • Richard Nisbett, “The Geography of Thought: How Asians and Westerners Think Differently ... and Why”, New York: Free
Press, 2003, ISBN 978-0-7432-1646-3 • Fred E. Jandt, “An Introduction to Intercultural Communication”, Chapter 3 - “Culture’s Influence on Perception”, SAGE
Publications, 2012, http://www.sagepub.com/upm-data/45975_Chapter_3.pdf
• Ian Cross, “Music, cognition, culture and evolution”, Annals of the New York Academy of Sciences Vol 930, 2001, pp 28-42.
• Wong PCM, Ciocca V, Chan AHD, Ha LYY, Tan L-H, et al. (2012) “Effects of Culture on Musical Pitch Perception.”, PLoS ONE 7(4): e33424. doi:10.1371/journal.pone.0033424
• Meagan E. Curtis, Jamshed J. Bharucha, “Memory and Musical Expectation for Tones in Cultural Context”, Music Perception VOLUME 26, ISSUE 4, PP. 365–375, ISSN 0730-7829, ELECTRONIC ISSN 1533-8312
• Xiao Hu, Jin Ha Lee, “A CROSS-CULTURAL STUDY OF MUSIC MOOD PERCEPTION BETWEEN AMERICAN AND CHINESE LISTENERS”, 13th International Conference on Music Information Retrieval (ISMIR), 2012, Porto, Portugal
• Bregman, A. (1990). Auditory Scene Analysis -- The Perceptual Organization of Sound.
• Bregman, A., & Ahad, P. (1996). Demonstrations of Auditory Scene Analysis: The Perceptual Organization of Sound.
• Wang, W. (Ed.). (2010). Machine Audition - Principles, Algorithms and Systems. IGI Global.
CATÓLICA'PORTO+ARTES+
References • Xavier Serra, Michela Magas, Emmanouil Benetos, Magdalena Chudy, Simon Dixon, Arthur Flexer, Emilia Gómez, Fabien
Gouyon, Perfecto Herrera, Sergi Jorda, Oscar Paytuvi, Geoffroy Peeters, Jan Schlüter, Hugues Vinet, Gerhard Widmer, “Roadmap for Music Information ReSearch”, Geoffroy Peeters (editor), 2013, Creative Commons BY-NC-ND 3.0 license, ISBN: 978-2-9540351-1-6 - http://mires.eecs.qmul.ac.uk
• Wang, D., & Brown, G. (2006). Computational Auditory Scene Analysis: Principles, Algorithms and Applications.
CATÓLICA'PORTO+ARTES+
THANK YOU!
Luís Gustavo Martins
[email protected] http://lgustavomartins.wordpress.com/about/
http://artes.ucp.pt
http://artes.ucp.pt/citar http://artes.ucp.pt/ccd