Mapping a Multidimensional Emotion in Responseto Television Commercials

Jon D. Morris,1,2* Nelson J. Klahr,2,3 Feng Shen,1 Jorge Villegas,1

Paul Wright,2,3 Guojun He,2,3 and Yijun Liu2,3*

1College of Journalism and Communications, University of Florida 2078 Weimer Hall, Gainesville, Florida2Department of Psychiatry, University of Florida, McKnight Brain Institute,

100 Newell Drive L4-100, Gainesville, Florida3Department of Neuroscience, University of Florida, McKnight Brain Institute,

100 Newell Drive L4-100, Gainesville, Florida Neuroscience

Abstract: Unlike previous emotional studies using functional neuroimaging that have focused on eitherlocating discrete emotions in the brain or linking emotional response to an external behavior, this studyinvestigated brain regions in order to validate a three-dimensional construct namely pleasure,arousal, and dominance (PAD) of emotion induced by marketing communication. Emotional responsesto ve television commercials were measured with Advertisement Self-Assessment Manikins(AdSAM1) for PAD and with functional magnetic resonance imaging (fMRI) to identify correspondingpatterns of brain activation. We found signicant differences in the AdSAM scores on the pleasure andarousal rating scales among the stimuli. Using the AdSAM response as a model for the fMRI imageanalysis, we showed bilateral activations in the inferior frontal gyri and middle temporal gyri associ-ated with the difference on the pleasure dimension, and activations in the right superior temporalgyrus and right middle frontal gyrus associated with the difference on the arousal dimension. Thesendings suggest a dimensional approach of constructing emotional changes in the brain and provide abetter understanding of human behavior in response to advertising stimuli. Hum Brain Mapp 30:789796, 2009. VVC 2008 Wiley-Liss, Inc.

Key words: fMRI; AdSAM; neuromarketing

INTRODUCTION

Researchers have used a variety of self-report techniquesin analyzing the emotional responses to commercials.Some have used a discrete self-report approach thatfocused on specic emotions such as happiness and anger[Izard, 1977; Plutchik, 1984]. Other researchers have used amore robust three-dimensional self-report approach[Osgood et al., 1957; Russell and Mehrabian, 1977; Sundarand Kalyanaraman, 2004] including physiological mea-sures to assess the emotional response. A consolidationbetween self-report and physiological measurementswould provide convergent validity for both methods, butcurrently a link between the two measures of emotion inresponse to marketing communications has not been fully

Additional Supporting Information may be found in the onlineversion of this article.Contract grant sponsors: American Association of AdvertisingAgencies and the American Research Foundation, American Bev-erage Medical Research Foundation.

*Correspondence to: Jon D. Morris, College of Journalism, Univer-sity of Florida 2078 Weimer Hall, PO Box 118400, Gainesville, FL32611 USA. E-mail: jonmorris@u.edu or Yijun Liu, The Universityof Florida McKnight Brain Institute, Gainesville, FL 32610, USA.E-mail: yijunliu@u.edu

Received for publication 8 August 2007; Revised 30 November2007; Accepted 28 December 2007

DOI: 10.1002/hbm.20544Published online 19 February 2008 in Wiley InterScience (www.interscience.wiley.com).

VVC 2008 Wiley-Liss, Inc.

r Human Brain Mapping 30:789796 (2009) r

explored. In this study, we proposed that the three-dimen-sional self-report approach to emotion is an effective meth-odological tool that emulates key physiological functionsin the brain. Such ndings could provide a promisingnew perspective for investigating issues that have beenunexplored or vaguely dened in previous neuroimagingstudies.The search for physiological links to emotion reects an

approach that seeks fundamental or discrete emotions[Mandler, 1984]. Subcortical emotional responses havebeen recorded through classical conditioning of fear reac-tions to audio or visual stimuli [LeDoux et al., 1989]. Theresponses either interrupt the cognitive focus of currentattention or inuence the context for ongoing cognitiveprocesses [Simon, 1982]. Pleasant and unpleasant emo-tional responses were found related to increase in the neu-ral activity in the medial prefrontal cortex, thalamus, andhypothalamus, while unpleasant emotions were associatedwith the neural activity in the occipitotemporal cortex,parahippocampal gyrus, and amygdala [Lane et al., 1997].Additionally, facial expressions of disgust or anger werefound to increase the neural activity in the left inferiorfrontal gyrus [Sprengelmeyer et al., 1998] and anterior in-sular cortex [Phillips et al., 1997]. A meta-analysis of emo-tion activation studies in PET and fMRI [Phan et al., 2002]concluded that no single brain region is activated by allemotions, and no single brain region is activated by oneparticular emotion.The discrete approach assumes categorical judgment of

emotional stimuli. This requires connections between bothhemispheres [Bowers et al., 1991] and between the anteriorcingulated cortex (ACC) and the bilateral prefrontal corti-ces [Devinsky et al., 1995]. Hence, certain brain regionsmay become activated from the demand to categorize orlabel discrete emotions rather than becoming activatedfrom the natural emotional responses to given stimuli. Fur-thermore, most neuroimaging studies treat emotions astwo discrete categoriespleasant and unpleasantwhileignoring the nuances along the pleasure dimension andthe additional explanatory power of the arousal and domi-nance dimensions. For example, the intensity of fear hasbeen associated with brain activities in the left inferiorfrontal gyrus [Morris et al., 1998] while anger and disgusthave been associated with different degrees of intensity orarousal [Iidaka et al., 2001].The alternative three dimensional approach to emotion

attempts to simplify the representation of responses byidentifying a set of common dimensions that can be usedto distinguish specic emotions from one another. Thisapproach includes both verbal and nonverbal measures[Lang, 1980; Osgood et al., 1957; Russell and Mehrabian,1977], and has been largely ignored in previous research.One example of this approach is the pleasuredispleasure,arousalcalm, and dominancesubmissiveness (PAD)model [Russell and Mehrabian, 1977]. The three bi-polardimensions are independent of each other, and the var-iance of emotional responses can be identied with their

positions along these three dimensions. The dimensionalapproach helps differentiate emotions postulated by thediscrete approach [Shaver et al., 1987] by providing anumeric level of each dimension to describe the specicemotions. Each discrete emotion can be identied by spe-cic combinations of the dimensions. The meaning of thesespecic adjectives may differ by individual, culture, orother inuences; nevertheless, the method for identifyingthe response is universal.The current study was designed to assess the validity of

the dimensional approach for measuring emotionalresponses by comparing them to the brain responsesobtained through neuroimaging. The neuroimaging datawas derived from blood oxygenation level-dependent(BOLD) signals detected with fMRI. The neuroimaginganalysis targeted the amygdala, prefrontal cortex (PFC),and temporal cortex for several reasons. First, the PFC hasbeen found to have neural projections to the amygdala[McDonald et al., 1996], and the temporal cortex hasbeen found to send stimulus information to the PFC[Hasselmo et al., 1989]. Second, the temporal cortexhas been found to allow the brain to merge perceptualand semantic information, past memories and short-termmanipulation of the stimuli [Mitchell et al., 2003].Third, both the semantic information and the dynamicnature of video clips have been found to increase neuralactivity in the PFC [Adolphs, 2002; Decety and Chami-nade, 2003].The self-report data was derived from responses to the

Advertisement Self-Assessment Manikins (AdSAM1) scale[Morris et al., 2005]. This scale provides a nonverbal,cross-cultural, visual measure of emotional response thatmeasures the dimensions of pleasure, arousal, and domi-nance, and we therefore suggest is a better tool than averbal technique that requires respondents to cognitivelytranslate their reactions into words before reporting theirfeelings. Thus, we postulate that this methodology,grounded in psychological literature since the 1950s,should be the basis of emotional detection in the brain.

MATERIALS AND METHODS

Subjects

Twelve healthy, right-handed young adult participants(6 males/6 females, age range 2228, mean age 24.8)signed written, informed consent for the protocolapproved by the Institutional Review Board at theUniversity of Florida. At the time of the scan, none ofthe participants reported taking any psychiatric medica-tion or having any history of neurological disorders. Allof the participants stated having either normal vision orcorrected to normal vision, and three of the participantsused specialized, nonmagnetic corrective lenses insidethe scanner. All participants were nancially compen-sated $50.00 USD.

r Morris et al. r

r 790 r

Experimental Protocols

The participants viewed ve commercials inside of thescanner in a block design paradigm created with E-Prime(Psychology Software Tools, Pittsburgh, PA). The commer-cials were presented by back-projection using a 1700 LCDscreen with a resolution of 1024 pixels by 768 pixelsthrough the Integrated Functional Imaging System (IFIS,MRI Devices, Waukesha, WI). An MRI-compatible audi-tory system (Resonance Technology) with stereo earphonesand a microphone protected participants from scannernoise while permitting verbal communication with theoperator and transmitted the sound of the commercials.The rst two commercials lasted 30 s each and the otherthree commercials lasted 1 min each. Resting blocksconsisting of viewing a red cross on a black screen forduration of 30 s were interspersed between each com-mercial block.The functional scan consisted of six runs with each run

(except for the initial resting-state run) being separatedinto three blocks: (1) a resting period, (2) a task of viewinga commercial, and (3) the AdSAM task. The initial runstarted with a 30 s resting block and then included twosets of the AdSAM task in response to How do you nor-mally feel? and How do you feel right now? Followingthe commercial block, the participants performed theAdSAM task, which consisted of three trials of ratingpleasure, arousal, and dominance. The participants wereasked to convey their feelings in terms of pleasure (happyvs. sad), arousal (stimulated vs. bored), and dominance (incontrol vs. cared for) by selecting the most appropriateSelf-Assessment Manikin out of ve possible choices im-mediately after viewing a commercial. The responses andreaction times were recorded with a right-handed buttonresponse glove (IFIS, MRI Devices, Waukesha, WI). Theparticipants were instructed to indicate how they feltafter watching each commercial by rating the AdSAMscales without spending a lot of time thinking aboutthe questions. They took a training at the same daybefore the scanning and were explained how to bestinterpret the manikin gures. For the training, the par-ticipants practiced the task by answering questionsregarding their general feelings and their feeling towardsviewing a commercial for V8 vegetable juice outside of thescanner.The ve commercials presented were all broadcasted

more than 10 years ago to avoid the likelihood of the par-ticipants having previously seen them on television. Therst commercial was a public service announcement calledBe a Hero. It shows happy looking children answering aquestion Who is my hero? Some name famous peoplebut one boy names his teachers. The commercial urgesthose interested to get involved in teaching and make adifference. The second commercial was from the springwater company Evian. This commercial portrays beautifulmountains, blue sky, and the sun sparkling on snow in theFrench Alps. The third commercial was from the soft drink

Coke. It shows a young boy offering his Coke bottle outsidethe locker room after a football game to Mean Joe Greeneof the Pittsburgh Steelers. At rst Mean Joe Green politelydeclines but then changes his mind, accepts the Coke bot-tle, and passes his jersey to the young boy as a return gift.The fourth commercial was for the sports drink Gatorade. Itshowcases special effects of a 23-year-old Michael Jordan,in a Chicago Bulls uniform, playing the modern-dayMichael Jordan in a one-on-one grudge match. A 1987 ver-sion of Jordans head was digitally placed on the torsoof an actual performance double playing against the real39-year-old Jordan. The two engage in the one-on-one bas-ketball, which shows the older-but-wiser Jordan schoolinghis younger, more energetic self. The fth and nal commer-cial was and an Anti-Fur public service announcement. Thecommercial starts with scenes from a fashion show with sev-eral models showcasing fur coats. The spectators are clap-ping and admiring the fur coats. As one model turns aroundthere is blood dripping from her coat and it is splattered allover the spectators who are now terried and disgusted. Asthe model leaves, there is a bloody trail behind her. It closeswith and an announcement: It takes up to 40 dumb animalsto make a fur coat but only one to wear it.

Imaging Parameters

A 3T head-dedicated MRI scanner (Siemens Allegra;Munich, Germany) was used. Functional images wereacquired with a gradient-echo EPI pulse sequence sensitiveto the BOLD signal using the following parameters: TR 53.0 s, TE 5 30 ms, FA 5 908, Matrix size 5 64 3 64, FOV5 240 mm, 36 axial slices with a slice thickness of 3.8 mmwithout gaps. The rst two functional volumes were dis-carded because of their T1 saturation. For structural core-gistration with the functional images, T1-weighted 3D ana-tomical images were acquired with a MPRAGE sequencein the following parameters: TR 5 1.5 s, TE 5 4.38 ms, FA5 88, Matrix size 5 256 3 256; FOV 5 240 mm, 160 slices,slice thickness 5 1.1 mm.

Data Analysis

BrainVoyager v. 4.9.6 (Brain Innovations, Maastricht,Holland) was used to analyze all of the imaging data. Thefunctional images from each participant were rst coregis-tered with the 3D anatomic images and then normalizedinto Talairach space. The resulting 3D functional data thenunderwent motion correction, linear trend removal, spatialsmoothing (5.7-mm FWHM Gaussian lter). A GLM pro-duced voxel-wise statistical activation maps. The predic-tors were estimated hemodynamic responses to the tasks.Contrasts between the predictors were used to evaluatethe relative contribution of each condition to the variancein the BOLD signal. A statistical threshold was set to P