the brain and multimodal communication elisabeth ahlsén university of gothenburg sskkii...
Post on 21-Dec-2015
215 views
TRANSCRIPT
The Brain and Multimodal Communication
Elisabeth AhlsénUniversity of Gothenburg
SSKKII Interdisciplinary Center & Department of Linguistics
Nationella Forskarskolan i Kognitionsvetenskap ht 2008
Contents1. Communication on different levels of
awareness and control - corresponding brain functions
2. What models cover - view of cognition and communication
3. Multimodal communication - some controversies
4. Examples of current research trends5. Brain damage - cognition and
communication
1. Communication on different levels of awareness and control -
corresponding brain functions
Communicative Body Movements:1)Facial gestures (nose, eyebrows, cheek, forehead, chin etc)2)Head movements3)Gaze direction, mutual gaze4)Lip movement5)Hand and arm movements6)Leg and foot movements7)Body posture8)Distance btw communicators9) Spatial orientation10) Touch11) Non-linguistic sounds
Types of representation
3 ways of carrying, conveying and sharing information
• Index• Icon• Symbol
Variation in awareness and intentionality
Bodily communication often morespontaneous, unaware, unintended (index, icon) than words (symbols).
Dimensions of content in communication
Words BodyFactual information
Emotions and Attitudes
Physiological states
Identity
Social relations
Communication ManagementInteractive communication managementOwn communication management
Interactive view on embodied communicationTwo communicators form one dynamic system by
establishing stabilizing communication links
Max Feedback + Max OutlookFeedback
Feedback employs all of these levels, too.
Right hemisphere lesions
• Left neglect, spatial disorder
• Prosody disorders
• Lexical-semantic disorders
• Problems with emotion information
• Discourse disorder (complexity, ToM, metaphor, humor, irony, etc)
AmygdalaMediation and control of major affective activities like friendship, love and affection, on the expression of mood and, mainly, on fear, rage and aggression
The center for identification of danger. Fundamental for self
preservation. Humans with marked lesions of the amygdala, loose the affective meaning of the perception of an outside information, like the sight of a well known person. The subject knows, exactly, who the person is, but is not capable to decide whether he likes or dislikes him (or her).
Hippocampus
Formation of long-term memory.When both hippocampi (right and left) are destroyed, nothing can be retained in the memory. The subject quickly forgets any recently received message. The intact hippocampus allows the animal to compare the conditions of a present threat with similar past experiences, thus enabling it to choose the best option.
Thalamus
Lesion or stimulation of the medial dorsal and anterior nuclei of the
thalamus are associated with changes in emotional reactivity. The medial dorsal nucleus makes connections with cortical zones of the
pre-frontal area and with the hypothalamus. The anterior nuclei connect with the mamillary bodies, and through them, via fornix, with the hippocampus and the cingulate gyrus.
HypothalamusLesions of the hypothalamic nuclei interfere with several vegetative functions and some of the so-called motivated behaviors.Specifically, its lateral parts seem to be involved with pleasure and rage, while the median part is like to be involved with aversion, displeasure and a tendency to uncontrollable and loud laughing. However, in general terms, the hypothalamus has more to do with the expression (symptomatic manifestations) of emotions
Cingulate gyrus(gyrus cinguli)
Its frontal part coordinates smells and sights with pleasant memories of previous emotions. This region also participates in the emotional reaction to pain and in the regulation of aggressive behaviour
Brain stem
Responsible for the "emotional reactions", (indeed, they are just reflex answers) of inferior vertebratesRemains active, not only as alerting mechanisms, vital for survival, but in the maintenance of the sleep-awake cycle.
Prefrontal areaIts intense bi-directional connections with thalamus, amygdala and other subcortical structures, account for the important role it plays in the genesis and, specially, in the expression of affective states.
When the pre-frontal cortex suffers a lesion, the subject looses his sense of social responsibility as well as the capacity for concentration and abstraction. In some cases, although consciousness and some cognitive functions, like speech, remain intact, the subject can no longer solve problems, even the most elementary ones.
2. What models cover - view of cognition and communication
A. Localist-associationist ”classical” modelLanguage - cortical centers + subcortical white
fibres-Broca’s area - production-Wernicke’s area - perception-comprehension-Connections-Concept centerGeschwind
B. Dynamic localization of function
A. R. Luria
Functional systems of many subfunctions in different areas, not always language specific, (e.g. rapid alternations, inertia)
Dynamic systems, can be restored-changed
3 blocks: brain stem, posterior PTO, anterior
C. Hierarchical / evolution based models(”holistic”)
J.H. Jackson- J. Brown: Microgenesis”from deep to surface” in phylogenesis,
ontogenesis and microgenesis”Focus on role of subcortical, older structures,
emotion etc.Automatic vs. Controlled behavioronLevels of function for language, emotion,
perception
3. Multimodal communication - brain and cognition - some controversies
• Localism - Associationism - Holism• Modularity - Interaction• Symbol manipulation - Distributed activation
/representation• Linear models - Interactive Activation
Models (IAM,ANN)• Language/speech and gesture intertwined or
separate/compementary - related how?• Definition/delineation of language / communication
system• Different types of body movements - how are they related
and how do they relate to spoken language?
4. Examples of current research trends
- Embodiment - many aspects, gesture one of them, role of body for development of language another
- ”Growth point” (McNeill)- Cultural influence vs universality of gesture +
relation to speech- Building embodied artificial agents- Studying coordination-communication on lower
levels of awareness and control- Neuroimaging? - Action-Perception link - localization
5. Brain damage & multimodal communication
From language/words sentences to communication - multimodal
Widening of the concept of aphasia (cf. language) - controversy
From left hemisphere cortex to any language-communication disorder, including, RH, subcortical-cerebellar, distributed, e.g. Alzheimer etc., TBI
New approaches needed and substantial research
New models needed
Next episode…
On
evolution,
mirror neurons,
the role of imitation and pantomime
and possible revisions of theories about communication disorders…
Assignment
Choose one of the controversies mentioned above and write one page argumenting for each of the two sides or suggesting two different views and approaches to answering one of the questions.
Literature:
Arbib M (2005). From monkey-like action recognition to human language: an evolutionary framework for neurolinguistics. Brain and Behavioral Sciences, 28, 105–24.
Ahlsén, E. 2008. Neurological disorders of embodied communication. In.I. Wachsmuth, G. Knoblich & M. Lenzen (eds.) Embodied Communication in Humans and Machines. Oxford: Oxford University. Press.