“Giant Toxic Lakes You Can See from Space”:

A Theory of Multimodal Messages and Emotion in Legitimacy

Work

“Producing tar sands oil releases three times the global warming pollution of conventional oil and creates giant toxic lakes you can see from space”

Scholars have long recognized that organizations need to appear legitimate to access

resources and avoid sanctions from stakeholders (Meyer & Rowan, 1977). It is therefore not

surprising that actors often carry out legitimacy work – defined as purposeful activity to shape

others’ evaluation of something as “desirable, proper or appropriate” (Suchman, 1995, p. 574;

Treviño et al., 2014) – to attack or support the legitimacy of organizations that they wish to

influence (Bitektine & Haack, 2015). The existing literature highlights that legitimacy work

involves attempts at persuasion in order to influence an individual’s beliefs, attitudes, or

behaviours (Voronov & Vince, 2012; Huy, 2012), but most prior discussions have focused

primarily on the words that actors produce in these persuasive efforts (e.g., Maguire & Hardy,

2009; Suddaby & Greenwood, 2005; Vaara, Tienari, & Laurila, 2006).

Recently, however, researchers have become increasingly interested in the persuasiveness

of images (Boxenbaum et al., 2018; Christiansen, 2018; Shortt & Warren, 2018; Höllerer et al.,

2013) and how, when combined in multimodal messages, words and images work differently

than when used separately (Höllerer, Jancsary & Grafström, 2018; Meyer et al., 2018; Lefsrud,

Graves & Phillips, 2016). In this paper, we build on this new stream of research on when

multimodal messages might more effectively persuade to theorize how combinations of images

and words promote certain emotions (Zietsma et al., 2018; Moisander et al., 2018) and aid

1

legitimacy work by more effectively capturing attention, making a message more intuitively

understandable, and promoting (re)evaluation of the legitimacy of a subject or sponsor.

Obviously, words evoke emotion; however, significant research shows that images more

powerfully trigger emotion (Snow et al., 2014; Wischmann, 1987; for a review see Lang,

Bradley, & Cuthbert, 1997). Images communicate instantaneously on an emotional and a

cognitive level (McQuarrie & Phillips, 2005; Rosch, 1975) and may evoke intense emotions as

viewers feel attracted to or repelled by the subject (Lang et al., 1997). Images can also increase

or decrease viewer arousal levels: those that increase arousal are noticed faster (Öhman, Flykt, &

Esteves, 2001) and viewed longer (Lang et al., 1993); and arrest individuals’ attention more

efficiently (Lang et al., 1997). This research suggests that multimodal messages that incorporate

images with words can contribute effectively to the persuasive process that underpins legitimacy

work (Doerr, Mattoni & Teune, 2013; Andersen & Boeriis, 2012; Höllerer et al., 2018).

Drawing on this research, we develop a model to describe this process and explain how the

visual/verbal structure, meaning construction, and content of a multimodal message can make

use of cultural resources in ways that are likely to evoke emotion in support of legitimacy claims.

Summarized as a research question, we ask: how do certain combinations of multimodal

messages (words and images) more forcefully evoke emotion and capture recipients’ attention,

motivate them to process those messages, and (re)evaluate the legitimacy of an organization, its

activities, and/or its industry. We conclude by discussing extensions and connections to

processes such as legitimacy work and values work.

We contribute to theory by combining four areas of existing research into a novel

theoretical framework: 1) concepts from social semiotics and visual rhetoric (Kress & van

Leeuwen, 2001, 2006; Schriver, 1997) to account for how words and images are assembled into

2

meaningful structures that evoke emotion; 2) theories about emotion to examine how individuals

emotionally experience various messages to prompt them to pay attention, motivate them to

process a message, and reconsider legitimacy evaluations (Elfenbein, 2007, 2014; Thagard et al.,

2001, 2011); 3) ideas from the rhetorical theory of identification, which explains how message

recipients respond based on their emotional reactions (Burke, 1969); and 4) models of legitimacy

evaluation to assess when individuals rely passively on cues and presumed assessments from

media, regulators, courts or others (Bitektine & Haack, 2015) and when they are motivated to

(re)evaluate a message sponsor, organization, its activities, and/or its industry for themselves.

MULTIMODAL MESSAGES, EMOTION AND LEGITIMACY WORK

Figure 1 presents our model of how multimodal messages combine texts and images to

capture recipients’ attention, provoke processing and emotional reaction, and prompt recipients

to re-evaluate the legitimacy of the targeted organization or industry.1 But first this caution: we

illustrate this model as a simple three-stage process due to the limitations of print communication

for representing the dynamic, simultaneous, and idiosyncratic process of meaning-making by

recipients. Our staged model helps make sense of the process, but it depicts it as a much more

static and sequential process than it actually is in practice.

<<Figure 1 about here>>

We begin by briefly summarizing the process. Stage one focuses on the construction of the

multimodal message: its visual/verbal structure (the relationship between images and words), its

content cued meaning structure (the form that triggers meaning making) and the content itself

(the specific words and images). This stage describes how a sponsor constructs the message to

convey “What does it mean?” and prepares us to theorize and present propositions to describe

recipients’ interpretation and response to the message in the following stages.

1 We thank Reviewer 3 for helping us succinctly summarize our theoretical story.

3

Stage two focuses on the reception of the message: how recipients process visual/verbal

structure and content to interpret the message’s meaning consruction. This stage presents various

ways that the multimodal message promotes recipients’ processing of the message and become

willing to take a stance towards the topic to consider, “What does this multimodel message mean

to me?”

Stage three focuses on recipients’ reactions to their interpretation of the multimodal

message. Their reactions arise from the interaction between their prior beliefs, values and

attitudes (stage two) and the message structure, content, and meaning (stage one), prompting a

response. In sum, stage three examines how recipients evaluate a multimodal message and

decide “What should I do?”

This process (and our model) is particularly relevant to the legitimacy work that occurs

around highly prominent organizations such as “celebrity firms” (Rindova, Pollock & Hayward,

2006). Celebrity firms are “firms that attract a high level of public attention and generate positive

emotional responses from stakeholder audiences” (Rindova et al., 2006, p. 51). These firms are

particularly vulnerable to multimodal messages intended to undermine their legitimacy and the

firms themselves (and their supporters) are more likely to produce multimodal messages

intended to garner support. The fact that they are well-known makes it more likely that recipients

will attend to the messages and re-evaluate legitimacy.

Multi-modal messages are also more likely to be produced when the activities of firms

intersect with the interests of the public. In these situations, social movement organizations are

more likely to produce multimodal messages as a part of the legitimacy work aimed at de-

legitimizing organizations. Our framework is therefore less applicable to the sorts of legitimacy

work that go on around smaller, less visible organizations whose activities are less controversial.

4

In these cases, organizations’ attempts to generate emotion through multimodal messages will be

less effective than other forms of legitimacy work, such as adopting a highly legitimate

organizational form or drawing on highly legitimate symbols.

Stage One: Construction of the Multimodal Message

We begin by exploring how visual/verbal structure creates a relationship between the

words and images in a multimodal message. Then, we examine the visual/verbal content of the

message, more precisely how elements are combined to create message meaning. The goal of

this section is to unpack the ways that the visual/verbal elements can be structured to create

persuasive multimodal arguments. We use several examples related to legitimacy judgments

about the Alberta Oil Sands as our illustrative cases.

Visual/verbal structure creates relationships. Viewers are motivated to analyze messages

by their“visual/verbal structure”. Visual/verbal structure refers to the relationship between the

images and words in a message (Phillips & McQuarrie, 2004; Schriver, 1997; for a review see

Maes & Schilperoord, 2008). When images and words are purposefully assembled by a sponsor

of an argument, they form a sign that stands for an object of thought (Peirce, 1931). The role of

message sponsor is crucial because they decide how the sign (in this case, the multimodal

message) connects with the mental object and thus has meaning. Only if the message sponsor

successfully creates meaning within a text, however, does it have the possibility of being passed

on within a dynamic social network (for the purposes of our article, as a persuasive argument

engaged in legitimacy work).

The sign structure of a multimodal message can be ‘read’ as a rhetorical figure (Kress &

van Leeuwen, 2001). For example, the image in Figure 2, “Canada [Not Just mounties and ice

hockey anymore],” is a maple leaf containing a mine site that dissolves into black droplets. We

5

might, therefore, read this sign as a visual metaphor—the maple leaf is a symbol of Canada, the

mine site creates a black liquid. However, the image alone is ambiguous; when words are added

they cue recipients as to the intended meaning, transforming it from a sign (Canada produces

black liquid) into an argument—Canada destroys its environment by producing oil —and from a

visual metaphor (Canada is an oil extraction site) into an analogical argument—Canada is anti-

environmental because its oil extraction activities worsen global warming (for more detailed

discussion of social semiotics, see Van Leeuwen, 2005).

Most research examines the complementary, mutually reinforcing roles that visual and

verbal elements play at different stages of institutionalization (Höllerer et al., 2018) such as the

theorization of an innovation (Cartel, Colombero & Boxenbaum, 2018) or the social construction

of a rational myth (Boxenbaum, Daudigeos, Pillet & Colombero, 2018). But how, exactly, may

visual and verbal elements be combined? Halgin et al. (2018) suggest a likely synchronicity

between the visual and the verbal in which each amplifies and extends the other and Meyer et al.

(2018) elucidate the complementary affordances and functionalities.

Building on this scholarship, we contend that words and images can be used in a

multimodal message to create four types of relationships2 that may involve an element of

surprise: 1) complementary, where visual and verbal components each contribute unique

information to the message based upon their affordances; 2) text infused, where multiple images

are merged and words state the meaning of those images; 3) juxtaposition, where the image and

words each have unique meanings, but combined they build a new third meaning; and 4) cued

2 Research has identified relationships between text and image beyond these four (e.g., stage setting or supplementary [Schriver, 1997]), but we have selected these four because they create the conditions that make a persuasive multimodal message possible: 1) text and image contribute more or less equally, 2) they contain a surprising element that attracts recipient attention, and 3) they generate emotion that creates the possibility for persuasion.

6

replacement, where words cue what is absent from the image (see Figure 1 for details on these

four relationships).

The four visual/verbal structures discussed in this article each contain an element of

surprise that arrests the recipients’ attention, rewards their processing (see stage two), and builds

the message’s emotional appeal. For example, Figure 2 is a text infused multimodal persuasive

message. The image fuses together scenes of an industrial site and a black liquid dripping inside

the outline of a maple leaf. The headline above and the blurb beneath contextualize the image.

The visual/verbal structure compares recipient assumptions with “reality” (to create surprise) to

question the legitimacy of the Canadian Oil Sands. Created by six environmental organizations

to challenge oil extraction in Alberta, this full-page ad appeared in April 2008 in Roll Call, a

newspaper of record on Capitol Hill in Washington, D.C., to educate U.S. decision-makers on

the negative environmental impacts of this oil extraction.

The visual/verbal structure of Figure 3 is cued replacement because it depicts

conservative/republican politicians who support and promote Keystone XL pipeline as a

beneficial construction project for American workers. Conspicuous by its absence is any

representation of the opposite viewpoint, the liberal or democrat perspective, which views the

pipeline as a potential environmental disaster, thus – latently – the cartoon communicates this

viewpoint exactly.

Visual/Verbal Meaning Construction guides interpretive action. Here meaning

construction refers to how the visual and verbal elements in the message are combined by

message sponsors to guide or cue recipients to interpret its meaning. These cues shape

individuals’ inferences as they process the relations presented (Kress, 2010; Kress & van

Leeuwen, 2001).

7

The visual and verbal cues can sponsor four interpretive actions in recipients: 1) to

establish a connection, 2) to compare similarity, 3) to contrast opposition, or 4) to imply

superiority (Maes & Schilperoord, 2008). These actions correlate with the increasing complexity

of the structural relationships between words and images; often juxtaposition or cued

replacement structures cue complex actions such as contrast of opposition or implication of

superiority, as we show in our examples.

These four interpretive actions, arising out of the visual/verbal structure, enable message

recipients to analyze its content and to construct a meaningful persuasive argument. Increasing

complexity requires more inferences of meaning and supports more indirect claims, for which no

explicit statements have been made (Smith 1991). With more indirect claims, recipients are

“invited to construct more multilayered meanings that are not actually given in a text… Indirect

persuasion attempts thus rely on consumer inference, in the sense of going beyond what is

explicitly stated in an ad (Johar, 1995; McQuarrie & Phillips, 2005, p. 8).

In multimodal messages that establish a connection between visual/verbal elements, the

sponsor creates a message to prompt recipients to draw inferences about how “A is associated

with B because…” and supports the construction of metaphors and metonyms. Additionally, “an

indirect metaphorical claim presented in a picture enjoys a further advantage because such

inferences are more likely to be generated spontaneously at the time of ad exposure” (McQuarrie

& Phillips, 2005, p. 7). Höllerer et al. (2018) demonstrate how images of corporate logos and

national flags are used as metonyms to represent all companies and countries involved in the

global financial crisis, to enhance words, and connect new phenomena (the crisis) to established

discourses (corporate power, nationalism). The image in Figure 2 establishes a connection

between the leaf, the site, and the liquid. The headline reinforces this connection by explicitly

8

naming “Canada.” Interestingly, both of our examples cued multiple actions. For example,

Figure 2 cues connection, compares similarity, and contrasts opposition.

<< Insert Figure 2 about here >>

In the second possible type of interpretive action, the sponsor creates a message that invites

recipients to compare similarity by employing analogical reasoning, a core cognitive process

(Forbus, 2001), to analyze the visual-verbal elements to identify the basis for similarity: “A is

like B because...” This process assumes that if two entities appear similar, there must be deeper

similarities (Gibbs, 1994), such as the balanced sheets and profit-and-loss accounts that illustrate

pleasing “ ‘bilateral symmetry’ – the exact identity between geometrical forms or perfect

equality between sums in columns” (Puyou & Quattrone, 2018: 737). In Figure 2, the words and

images not only establish a connection, they also compare similarity: the fused image and

headline invite recipients to see a visual analogy between Canada and an industrial site. The

words cue recipients to recognize the black substance as oil and transform the message into an

analogical argument that Canada is an environmental disaster (mining site + oil spill).

A third possible type of interpretive action that can be created by sponsors is a contrast of

opposition. Sponsors cue individuals to start with basic similarities before noting differences,

how “A is not like B because…” (Durand, 1987). The mental inferences become more complex

depending upon how dissimilar the objects are. Irony, the trope that says one thing while

meaning the opposite, requires the most complex processing because it “creates spaces for

innovation through questioning established social knowledge” Höllerer et al., 2018: 12).

However, research has shown that ironic comparisons are more difficult to recognize than

metaphorical similarity (Colston & Gibbs, 2002).

9

In addition, if a sponsor uses irony to rebut an opponent’s position, they create the

possibility of a less effective message since research suggests that they may be less liked by

message recipients, the message topic may be less favorably received, and their arguments less

well remembered (Whaley & Wagner, 2000). This research suggests that exaggerated ironic

arguments may backfire and cause the opposite response: recipients identify with the sponsors’

opponent.

In Figure 2, “Canada [Not Just mounties and ice hockey anymore],” while the fused image

compares Canada to an oil spill, the verbal elements cue a third interpretive action, the contrast

of opposites: in the image (a lovely red maple leaf versus a black oil-soaked one) and the words

(Canadians are not the law abiding hockey fans that Americans think they are). The image

contrasts what is visually expected with “reality” and tries to evoke negative emotions, and the

words explain its meaning by contesting most Americans’ assumptions about Canadians. The

verbal explanation articulates the argument that Canadians are duplicitously ruining their own

and the world’s environment to produce energy for naïve Americans, and the fused image

intensifies and supports the negative emotional appeal.

A fourth possible type of interpretive action available to sponsors is to imply superiority, a

more complex action that requires evaluation beyond noting similarities and differences. To

determine how “A is better than B because…,” sponsors invite recipients to rate A and B against

some standard to infer how one is presented as better than the other. However, implying

superiority can be tricky; research in advertising suggests that if the products being compared are

grossly different the audience will devalue the quality of the target product (Zhang, Kardes, &

Cronley, 2002). Both contrasting opposition and implying superiority can create analogies of

rebuttal—that is, analogies that support the sponsor’s position and attack the opponents’:

10

The level of attack […] relates directly to the degree of exaggeration in the base of the analogy. The more exaggerated the base, the more ridicule the analogy creates. (Whaley & Holloway, 1996, p. 165).

Such analogies, then, aim to garner support for the message while ridiculing the opposite

position. In the next section we discuss an example that uses implications of superiority. Figure

3, “No Exit: Keystone XL,” is a political cartoon that first appeared in CarBuster magazine in

2012 to foster opposition to Keystone XL pipeline. Keystone XL was to join pipelines in Canada

and the US to move Canadian oil to Texas refineries. While appearing in a US publication, this

cartoon targets both Americans and Canadians.3 The interpretive cues for the cartoon’s

visual/verbal content include contrast of opposites—the missing liberal/democrat view silently

counterpoints the pitch for good jobs for American workers, and implication of superiority—the

American worker is smarter than either politician because he can see through the hype to the

truth: Keystone XL will cause environmental pollution not economic prosperity..

<<Insert Figure 3 here >>

Content. Besides the visual/verbal structure and meaning construction, which is determined

by how the message is composed, the message sponsor decides what content (specific images

and words) to include in the message to trigger recipients’ ‘feeling rules’ (Hochschild, 1979;

Frijda, 1988; Voronov & Weber, 2016). Frijda (1986) argues that message sponsors trigger

recipients’ emotional response from the combined psychological and physiological effects from

specific content (Frijda, 1986). In Figure 2, the predominant maple leaf dripping in black with

the dissolving mining site, is designed to quickly capture recipients’ attention to revisit the

symbols of Canada. In Figure 3, Keystone XL pipeline is depicted as a sewer pipe running from

3 We would like to thank our reviewers for pointing out that the intended readers for Figure 3 could be Canadians rather than Americans, based on the fact that most Americans would be unable to recognize Stephen Harper as the Canadian Prime Minister at that time. Since CarBusters is an American publication, we would argue that the cartoon addresses readers on both sides of the border.

11

Canada’s faceless Prime Minister’s toilet to Texas refineries to trigger recipients to reconsider

the relationship between Canadian and American politicians and the effect on American workers.

Stage Two: Recipient Interpretation

In this section, we explore how recipients interpret the visual/verbal structure, meaning,

and content to decipher a multimodal messages. While Stage one focused on the sponsors’

construction of the message to trigger specific interpretations of “What could this mean?”, in

Stage two recipients must process these elements to decide, ‘What does this mean for me?’

(Hochschild, 1979; Frijda, 1988; Voronov & Weber, 2016). The process may be conscious

(Lemerise & Arsenio, 2000; Lazarus, 1991), for example, completing an appraisal checklist

(Smith & Ellsworth, 1985; Lazarus, 1991) or subconscious (Monahan, Murphy, & Zajonc,

2000); in either case, this activity is highly individual and context-specific.

First, we examine how recipients process the message meaning emotionally (see Fig 1,

Stage 2, ‘Emotional Processing’). To do this, we draw on theories that illuminate how recipients’

belief systems shape their processing by examining moral domains (Haidt & Joseph, 2008) and

emotional transfer (Thagard et al, 1997; 2001; 2011). Second, we explore the cognitive aspects

of this processing (See Figure 1, Stage 2, ‘Cognitive Processing’): specifically inferences and

analogical arguments. These roughly equate with the logic of the argument presented in the

message.

Multimodal messages capture attention: Theory from visual rhetoric suggests that the

unexpected attracts attention, and a combination of provocative or startling images and/or words

is most effective (Meyer et al., 2018; Adesope & Nesbit, 2012; Sojka & Giese, 2006;

Wischmann, 1987). Such dissonance is a source of novelty (Antal, Hutter and Stark, 2015).

These unexpected, visual elements ensure that the message stands out from purely textual

12

messages (Pineda & Sowards, 2007; see Stage 1 in Figure 1), and the more novel the visual

elements the more likely viewers will register them (Frijda, 1986; Scherer, 1984).

Foregrounded images that contain unexpected elements most efficiently capture recipients’

attention. Researchers have applied diverse labels to this phenomenon including “transgressive

images” (McNaughton, 2007; Hatfield, Hinck, & Birkholt, 2007), “representational violations”

(Gkiouzepas & Hogg, 2011), “perspectives by incongruity” (Burke, 1984), and “oxymoronic

images” (Mothersbaugh, Huhmann, & Franke, 2002; Lerner & Keltner, 2000, 2001). Unexpected

visuals most effectively create surprise, connect seemingly disparate meanings and values

(Höllerer et al., 2013), and generate both cognitive and affective dissonance to grab an

audience’s attention (Jasper, 2011). Such inconsistencies tend to surprise or interest individuals

(an emotional response) and motivate them to process that message (a further emotional

response). Summarized as a proposition:

Proposition 1: Multimodal messages that foreground unexpected images in combination with words are more likely than messages with no images or unsurprising images to prompt surprise in recipients and motivate them to process the message (see P1 in Figure 1).

Belief systems moderate recipients’ emotional reactions. As noted earlier, there is

tremendous variability in how people experience messages and how they react emotionally, with

the outcome that different people agree or disagree with the same message. Individuals’

attention, preferred emotional experiences, and emotional reactions to arguments are shaped by

their personal beliefs and values (Feinberg & Willer, 2013; Hart & Nisbet, 2012; Dodd et al.,

2012; cf. value spheres in Meyer et al., 2013). Thus, beyond simple attraction and liking based

on socio-demographic similarity, individuals are more likely to pay attention to, find emotionally

13

resonant, and endorse messages that reinforce their existing beliefs systems (Benford & Snow,

1988; De Vaujany & Vaast, 2016).

To explain how prior beliefs and values shape individual emotional responses to

multimodal messages, we draw on Moral Foundations Theory. According to this theory, moral

intuitions – feelings of approval or disapproval – are derived from innate psychological

mechanisms that have coevolved with cultural institutions and practices (Haidt & Graham, 2007;

Haidt & Joseph, 2004, 2008; tested by Graham et al., 2011). Five fundamental domains of

morality are each allied with characteristic emotional responses: harm/care concerns related to

caring for and protection of other people (emotion: compassion); fairness/ reciprocity concerns

related to treating others fairly and upholding justice (emotion: gratitude if upheld, anger/guilt if

violated); in-group/loyalty concerns related to group membership and loyalty (emotion: pride

and belongingness if upheld, anger at traitors who violate); authority/respect concerns related to

hierarchy, obedience, and duty (emotion: respect if upheld, fear if violated); and purity/sanctity

concerns related to preserving purity (emotion: reverence if upheld, disgust if violated). These

moral bases, then, help us to identify the range and type of emotional reactions cued by the

interpretive actions embedded in message content and structure.

If we apply moral foundations theory to our examples in Figure 2, “Canada [Not just

mounties and ice hockey anymore],” the multimodal message invokes three moral bases to

engage recipients’ emotion. Sponsors invoke harm/care and in-group/loyalty regarding

Canadians when they argue that this group desires to harm Americans by selling them dirty oil.

They also invoke purity/sanctity by picturing the mining site dissolving into oil (described as

dirtying “the Boreal forest”), and the words include the adjectives ‘dirtiest’ and ‘toxic’ to

reinforce the oil sands’ impurity. These moral bases, then, could provoke anger, rage, and disgust

14

at Canadians in American politicians (the target audience for Figure 2). This message aims to tap

into recipients’ ideological beliefs to provoke a negative emotional reaction by American

legislators to Canadian oil.

Emotional transfer. The visual/verbal structure and content require emotional processing

by the recipient, since multimodal messages combine the visual and the verbal to enhance the

emotional impact of the argument. As we noted in Stage one, the interpretive actions of the

recipient in comparing similarities, highlighting differences, and establishing superiority require

analogical thinking as he or she interprets the message to discern its meaning. Analogical

thinking has been argued to generate emotion or rhetorical transfer from sponsor to recipient

(Thagard, 2011; Thagard & Shelley, 2001; Barnes & Thagard, 1997; Holyoak & Thagard, 1997)

by depicting people’s emotional reactions, specific perspectives, colours, or emotional

symbolism (Höllerer et al., 2018) to create a “situated perspective, which suggest a sentiment of

involment and personal relevance” (Meyer et al., 2018, p. 16, drawing from Kress & van

Leeuwen, 2006).

Thagard et al have suggested three methods by which emotion is generated through

recipients’ interpretation of a message: 1) emulation/ persuasion,4 2) empathy, and 3) projected

empathy. Emulation designates a situation where recipients are invited to adopt a particular

emotion as described in the message from a first-person perspective. Empathy is defined as a

situation where recipients experience the transfer of the sponsors’ emotional reaction to

themselves (so they feel the emotion characterized by the subject), from a second-person

perspective. Projected empathy is defined as the process where recipients are invited to inhabit a

familiar emotional situation (say, anger at being unjustly fired) and then transfer this emotion

4 Thagard et al use “persuasion” for this concept; we have altered this label to “emulation” to differentiate it from our rhetorical use of “persuasion” in this article.

15

analogously to a new situation to understand it (anger at injustice towards someone else), first-

person perspective in another analogous situation. Because the recipient’s memory includes the

feeling of being fired, the analogical transfer is a cognitive correspondence and a projection of

emotions. Barnes and Thagard (1997) contend that “extending analogical thinking beyond verbal

thinking should not be surprising, since analogies can be visual or even auditory” (p. 12). These

three methods of evoking emotion usefully suggest how multimodal messages may generate

emotion through recipients’ interpretive processes.

Figure 3 uses three different strategies to elicit viewers’ emotional reactions towards the

three people presented. First, Stephen Harper, the Canadian prime minister (2006 – 2015), is

depicted sitting on a toilet, its sewage pipe being KeystoneXL. The concept of emulation

illuminates how this scene generates emotion: Harper appears vulnerable (pants down) and an

object of disgust (reaching for toilet paper). Recipients are invited to feel repulsed at the vulgar

scene.

The concept of empathy explains how recipients are invited to respond emotionally

towards the Republican senator, pictured as an elephant, who celebrates the pipeline and the

“Jobs!” it brings with it; however, visual details (closed eyes, sewer line, filth pouring out, and

dead fish) undermine the verbal statement. The mismatch between the visual and verbal elements

implies that recipients should share the sponsors’ anger and ridicule at the hypocrisy depicted.

Finally, the concept of projected empathy describes how recipients should respond

emotionally to the third figure, an American worker. This figure eyes the leaking pipeline and

corrects the politicians’ rosy view: “Cleanup jobs!” Recipients may recognize the situation—

cleaning up someone else’s mess—and are invited to transfer the emotions from those

experiences to this situation to feel disgust, ridicule, and anger at the duplicitous politicians.

16

When recipients project empathy onto the worker, they should see the falseness of the

politicians’ perspectives and are invited to feel superior because they have recognized the truth

(that which is missing from the politicians’ account of the pipeline—the cued replacement

structure of the message). Thagard et al’s model of emotional rhetorical transfer is useful to show

how the interplay between visual and verbal elements in a multimodal message can aim to

generate recipient emotion.

Complexity and Curiosity. As noted, multimodal messages are processed as recipients use

the interpretive actions cued by the visual and verbal structures to ascertain the message

meaning. The four relationship structures allow for multimodal messages that increase in

complexity (e.g., a complementary relationship allows simple messages; cued replacement

generates indirect and complex messages). The more indirect the message structure, the more

“open” its meaning and the greater the number of particular thoughts it may may elicit. Further:

Since meanings of verbal expressions are more specific than meanings of visual expressions … verbal meanings provide less room for particular thoughts per individual. Openness may define interference of verbal and visual expressions: Images are less open when combined with captions than they are without them. (Lagerwerf & Meijers, 2008, p. 19)

Therefore, recipients must engage more effortfully to decipher the verbal/visual relationships as

the latter increase in structural complexity (Kardes, 1988, 2013; Groupe-μ, 1992). With greater

effort, recipients grow more emotionally engaged as they participate more in creating the

message; Aristotle (1355) first identified this persuasive process in speeches, and subsequent

work has elaborated it (Van Mulken, Van Enschot, & Hoeken, 2005). Curiosity is a positively

valenced, high arousal emotion, which draws in individuals. While emotional engagement does

not guarantee that recipients find the message persuasive, it does increase their susceptibility to

17

it. Thus, beyond the initial surprise, the emotional appeal of these structures can lie in the energy

that recipients engage to infer the message argument. Summarized as a proposition:

Proposition 2: Multimodal messages that have a more complex visual/verbal structure and/or more complex content-cued meaning construction are more ‘open’ to inferences than less complex messages and more likely to prompt curiosity in recipients that motivates them to process the message (see P2 in Figure 1).

Complexity and Frustration. In the case of legitimacy work, efforts to contrast opposition

and imply superiority generally aim to emphasize the legitimacy of the message sponsor’s

organization or industry while undermining their opponents’. Research has produced several

interesting insights into the role of similarity in effecting persuasion: 1) the extent to which

recipients accept the proposed similarities highlighted between items (and ignore dissimilarities),

the more compelling they find the message—that is, simpler comparisons are easier to interpret

(Sopory & Dillard, 2002); 2) individuals’ educational level moderates their ability to process

metaphors (Lagerwerf & Meijers, 2008)—that is, with more education they can understand more

sophisticated comparisons; and finally, 3) “metaphorical objects embodied in visual form should

not be excessively dissimilar . . . if the superior impact of the visual is to be achieved”

(Gkiouzepas & Hogg, 2011, p. 114)—that is, inaccurate comparisons are less effective in

persuading recipients. This research suggests that at lower levels recipients become more

emotionally engaged as they make more inferences of meaning until the message becomes too

complex, and they give up. Summarized as a proposition:

Proposition 3: If the visual/verbal structure or content-cued meaning construction of a multimodal message becomes too complex, this may prompt frustration in recipients and demotivate them from processing the message (see P3 in Figure 1).

As noted in Stage 1, the relationships created by the sponsor among words and images in

a multimodal message allow for complex approaches to creating meaning, such as metaphor or

18

metonymy. For example, as shown in Figure 2, the text infused image depicts Canada as an

environmental disaster (maple leaf + mining site + oil spill = environmental disaster). Further,

the interpretive actions of comparing similarity and contrasting opposites in Figure 2 require

recipients to think analogically to understand the argument, in this case, juxtaposing Americans’

fantasy that Canadians are benign law-abiding hockey lovers with the reality that they

duplicitously exacerbate global warming through oil sands extraction. Third, recipients must

infer the argument conclusion: friends look out for our best interests; Canadians are not looking

out for Americans’ best interests (environmentally), so they are not ‘our friends’ (from

Americans’ perspectives). The metaphor, the analogical argument and the train of inferences

culminate in the conclusion that the facts presented in this message about Canadian oil extraction

should revise recipients’ estimations of not only the oil extraction industry in Canada as evil and

illegitimate but of Canadians themselves as malevolent and duplicitous.

Stage Three: Recipient Reaction

As we have noted, recipients may respond in one of three ways when they process a

message: 1) they identify and agree with the message’s argument; 2) they are disconcerted by the

message argument and ultimately disidentify and disagree with it; 3) they are disconcerted by the

message argument and experience ambivalent (mixed) emotions. Ambivalence requires them to

re-evaluate their thoughts and feelings about the message, its subject, and the sponsor - a process

that may cause recipients to change their minds.

As a result, a) recipients may identify and then agree with the sponsors’ stance or b) re-

evaluate but not identify or change their minds, leading to them to continue to disagree with the

sponsor. In discussing recipient reaction to a multimodal message, we address the concept of

audience. Any given message, Figure 2 for example, will elicit a broad range of responses from

19

the individuals who encounter it. Some will agree, some will disagree, some will be confused,

some will laugh, some will take offense. Responses, as we have noted, arise out of recipients’

values, beliefs, previous experiences, reactions to the message content and meaning. In the next

sections, we offer several theories to try to explain how positive and negative emotions are

linked to identification to illustrate how recipients’ evaluation of meaning can produce one of

these outcomes. First we explore the link between positive emotions and recipient identification

with the message.

Identification: Identification (the sense of sameness shared by message sponsor and

recipient) is characterized by Burke (1969) as entailing five strategies: 1) establish that the

recipient and sponsor share crucial values ; 2) illustrate how the interests of recipient and sponsor

align; 3) demonstrate that both belong to a priviledged group ; 4) create a dichotomy between

“us” and “them” (the enemy); and 5) join together to act against the enemy. Research has

suggested that positive emotion is associated with identification and negative emotion is

associated with dis-identification. Recipients who identify with people or even inanimate objects

depicted in a message are more likely to agree with the message (Hart & Nisbet, 2012).

Similarly, Delbaere, McQuarrie, and Phillips (2011, p. 121) demonstrate that photorealistic

images in advertisements showing an object engaged in human behaviour can encourage

consumers to anthropomorphize that object, leading “to more positive emotions . . . and greater

brand liking.” Thus, besides capturing attention (as discussed in stage 1), using neutral or

positive faces can result in identification and connectedness with the subject depicted in a

message, while faceless or threatening images create emotional dissonance and disconnection

(Ferri, Weinberg, & Hajcak, 2012). Thus, we propose that the emotion generated by the content

in multimodal messages can evoke identification or dis-identification (see P4 in Figure 1).

20

Proposition 4: If the content-cued meaning construction of multimodal messages include images or meanings that prompt identification (i.e., faces, recipients’ core beliefs and values), recipients are more likely to connect strongly with the message to accept the sponsor’s stance (see P4 in Figure 1).

Proposition 4 may be applied to Figure 3, “No Exit: Keystone XL Pipeline,” at least for

some recipients. As noted, three figures are depicted: Stephen Harper, a Republican senator and

a worker. Two of three figures have faces, presenting options for recipients to experience

identification (P4). Recipients are less likely to identify with the faceless Harper, even as he

represents all Canadians. Harper is depicted as vulnerable (i.e., negative), inviting recipients to

dis-identify with him. The senator has closed eyes, but recipients are less likely to identify with

him because visual details undermine his credibility, creating a negative depiction (P4). The

worker is positively depicted as recognizing the hypocrisy and presents an option for recipient

identification; they are invited to admire his honesty and feel anger at the duplicitous politicians.

The potential outcome of emotions that induce identification and dis-identification is that the

recipients find the worker’s perspective persuasive and the politicians and pipeline project of

questionable value, positions sufficient to de-legitimate the latter depending, of course, on the

recipient’s individual context and belief system.

Dis-identification. Recipients who have a different value system than that invoked by a

multimodal message are likely to feel disconnected, dis-identify with, and reject the sponsor’s

stance. In these cases, recipients are unlikely to reconsider their position on the subject, and they

will consider the message unconvincing. Research also suggests that many potential recipients

are turned off by negativity and exaggeration. When meaning is constructed to exaggerate the

contrast between the elements in an analogy (e.g., contrasting opposition or implying

superiority), it intensifies the attack, increasing the message’s persuasive power for some

recipients and for others making it repellant (Colston & O’Brien, 2000). Ironic analogical

21

arguments have also been found to reduce recipient’s liking for the message sponsor, diminish

their attitudes towards the topic, and inhibit their ability to remember the sponsor’s arguments

against their opponents (Whaley & Wagner, 2000).

This research suggests that overly exaggerated, negative analogical arguments may not

successfully de-legitimate opponents but instead cause recipients to disidentify with the

sponsors’ cause. In such cases, the recipients’ opinions may boomerang, and they may reject the

sponsors. For example, the controversial Charlie Hebdo picture depicting a drowned three-year-

old refugee boy sparked a backlash among some who were disgusted by the journalist’s decision

to use a small child’s death, yet among others it invoked care/harm and compassion, which

fundamentally changed how refugees were discussed in the media. For the purposes of symmetry

(perhaps unsurprisingly), we propose that comparisons that are too intense in a message may

motivate recipients to actively re-evaluate the sponsor and subject but not cause them to change

their position:

Proposition 5: Multimodal messages that have a highly negative meaning based on content-cued meaning construction of images and words (i.e., imply superiority, contrast opposition) and content (i.e., negative images) are more likely than somewhat more neutral or positive comparisons to prompt disconnection in recipients, such that they will dis-identify with the message and sponsor and evaluate one or both as illegitimate, (see P5 in Figure 1).

This proposition can be applied to Figure 2, “Canada: Not just . . . ,” to understand the

perspective of a Canadian message recipient. Despite the primary audience for the

advertisement being American legislators, ordinary Canadians who might encounter this

message may find exaggerated, outrageous, and excessively negative the analogies

implied in the image (mining site = Canada) and in the text (“the world’s dirtiest oil”,

“three times the global warming pollution”, “giant toxic lakes” vs. “mounties” and ice

hockey”). The one-sidedness of the advertisement could realistically be argued to cause

22

disconnection among Canadian recipients who are knowledgeable enough to understand

how false these comparisons are. Such viewers are therefore likely to disidentify with this

advertisement and find its message unpersuasive.

In sum, multimodal messages that are experienced as excessively ridiculing, especially if

it depicts people, are more likely to cause individuals to turn away (Ferri et al., 2012), be

demotivated from taking further action (Lang et al., 1997), and even dislike the sponsor (Whaley

& Wagner, 2000). When claims are too complex or when they require recipients to overlook too

much dissimilarity, recipients may become frustrated leading them to abandon the message,

losing any possibility of persuasion. If claims are exaggerated, individuals may balk, causing a

“boomerang effect” where they purposefully choose the opposite (Jones et al., 2014).

Ambivalent emotions encourage re-evaluation. To persuade effectively, one must

understand the conditions that prompt recipients to re-evaluate established ideas about a subject.

However, in cases where recipients lack a firmly established perspective on a subject (or

organization/industry), persuasion is possible if the message is constructed effectively and

matches their general beliefs and value systems (Giorgi, 2017). For example, irony is particularly

effective at communicating ridicule and other negative emotions that reveal previously

unconsidered aspects of the subject often in a humourous manner, aspects that make it

uncomfortable to maintain the unexamined belief (Höllerer et al., 2018; Graves, 2011; Colston &

Gibbs, 2002). This combination of humour and discomfort may cause recipients to change their

minds. In the case of legitimacy work, it may even motivate them to actively re-evaluate their

positions.

Thus, in addition to triggering pure emotions and identification or disidentification, as we

discuss above, multimodal messages can also evoke ambivalent emotions in recipients.

23

Ambivalence (the simultaneous experience of positive and negative emotions, such as shame

plus hope) are particularly effective in garnering attention. Emotional ambivalence can signal

unusual circumstances, create a feeling of indecision, and cause individuals to reconsider

relationships between seemingly unrelated meanings (Toubiana & Zietsma, 2016; Fong, 2006;

Maitlas et al., 2013). Individuals who experience ambivalent emotions have been found to be

more likely to perceive that they are in an unusual environment, pay attention, and be more open

to organizational messages (Fong, 2006; Maitlas, Vogus & Lawrence, 2013, Plambeck & Weber,

2009). As a result, ambivalent emotions can prompt individuals to analyze their conflicted

emotions.

With such reconsideration, recipients become more open to alternative perspectives and

better at forecasting and estimation tasks (Rees et al., 2013) and other adaptive behaviours such

as holism and compromise (Ashforth et al., 2014). Leaders who express complex, ambivalent

emotions are more likely to empower their employees to act proactively (Rothman & Melwani,

2017). Thus, messages and sponsors that evoke ambivalent emotions may result in active

evaluation, deeper consideration of the ideas, and changed attitudes towards the message,

sponsor, and/or subject:

Proposition 6: Multimodal messages that have a visual/verbal structure and/or content-cued meaning construction that prompt ambivalent (simultaneously positive and negative, rather than positive or negative) emotions in recipients are more likely to motivate them to re-evaluate the legitimacy of the message sponsor and subject (see P6 in Figure 1).

Both “Canada [Not just mounties and ice hockey anymore]” and “No Exit: Keystone XL

Pipeline,” use techniques aimed at provoking ambivalence in message recipients. In “Canada

[Not just . . .]”, sponsors contrast the intended recipients’ (American legislators) view of

Canadians as benign and law-abiding by revealing a new ‘truth’ about them. The stark contrast is

24

intended to arrest most recipients’ attention while they process this argument (emotionally and

cognitively); it is constructed to create a moment of uncertainty and ambivalence: Have I been

wrong to trust Canadians?

Turning to “No exit: Keystone XL Pipeline,” message recipients who support

environmental protection but fear high unemployment and high energy costs might feel

ambivalence: shame at the sewage pipeline to the Gulf (violates the care moral base) and

happiness that pipeline construction increases employment and secures North American energy

sources (reinforces care moral base). Other recipients might feel different ambivalent emotions:

those who support the pipeline are proud of the new jobs (reinforcing care) but fearful of

polluting their country (violating harm).

Ambivalence, therefore, can act as an agent of persuasion when recipients examine their

emotions to understand them better and thereby reach previously impossible insights. For

example, some might evaluate the environmental threats presented in Figures 2 and 3 as greater

than they had thought because of the analogy’s aptness (harm/care), or they may dismiss the

threat in disgust at the unfairness of the representation (fairness/ reciprocity) that exaggerates the

negatives and dismisses the positives. Those recipients who feel ambivalence towards the

message are a valuable target because their mixed emotions raise the possibility of changing their

positions. Recipients can also have mixed emotions about the message and the sponsor: they feel

negative about the message and dis-identify with the argument presented, but they feel positive

about the message sponsor and identify with it. We discuss this further in the next section.

THEORETICAL CONNECTIONS AND EXTENSIONS

As “comprehensive multimodal studies are yet to be conducted in organizational research”

(Boxenbaum, Jones, Meyer, and Svejenova, 2018: 603), our aim is to develop a model to allow

25

foundational examination of how certain combinations of multimodal messages (words and

images) may reliably capture recipients’ attention, forcefully evoke their emotions, and motivate

them to process those messages and (re)evaluate the legitimacy of an organization, its activities,

and/or its industry. This model enables us to respond to the request by Boxenbaum et al. (2018)

to grow propositions out of empirical data, as informed by related scholarship, to examine the

interaction between visual and verbal realms and their differentiated contributions, and the role

of visual/verbal elements in organizational research.

Our theoretical model also points to a number of important extensions of existing

scholarship. First, to create our theoretical model regarding message effectiveness, we connect

four literatures: concepts from social semiotics and visual rhetoric, theories about emotion,

rhetorical theory of identification, and models of legitimacy. Second, by examining how words

and images may be combined to capture recipients’ attention more reliably than words alone, to

forcefully evoke emotion, and to motivate processing and (re)evaluation of legitimacy, we

generate testable propositions. These propositions may allow for the examination of how

differing word/image combinations might be more or less effective, say, during various stages of

institutionalization or deinstitutionalization, or as part of other multimodal messaging including

words and images in a restaurant’s menu alongside the taste, texture, temperature, and

appearance of its food. Third, by elaborating on the ways in which multimodal messages create

identification and disidentification, we create the potential to elaborate on values work.

Examine effectiveness of multimodal messages

From our perspective, much of existing legitimacy research samples on the dependent

variable (i.e., some sort of change in a legitimacy judgment) without disentangling the relative

effectiveness of various message elements to capture attention and persuade (or not). In other

26

words, the focus is narrowly on messages that result in changes in legitimacy judgements, an

approach that misses the critical first step in terms of the characteristics of multimodal message.

Research suggests that multimodal messages that create emotional ambivalence are most

likely to provoke individuals to pay attention, consider, and more actively re-evaluate their

positions. We examine the sources of ambivalence resulting from mismatches of words and

images. Future research could examine what combinations of visual and verbal elements are

more effective in triggering ambivalence? Perhaps an unexpected pairing of an attractive image

with more negative text is best able to capture attention, promote processing, and re-evaluation.

What combination of mixed emotions are most effective? Would high-intensity, pleasant

emotions attract individuals towards a problem-solution in combination with low intensity

displeasure to neutralize the status quo? Or perhaps there is a more effective ordering: shame or

regret for our past support of the status quo, followed by motivational redemption to cleanse

ourselves from these negative emotions (Voronov & Vince, 2012; Creed, Hudson, Okhuysen, &

Smith-Crowe, 2014; Schwarz, Wong, & Kwong, 2014). What if we separate the targets of

ambivalence? We might, for example, develop a message that aims to trigger negative emotions

in response to the subject, such as anger towards organizational injustices, and positive emotions

towards the message—an invitation to join a hopeful mutiny (Huy et al., 2014). Our process

model provides a method of examining how combinations of images and words in multimodal

messages evoke ambivalent emotions, in which order, and towards which targets (sponsor,

message content, subject) to create active re-evaluation of an organization or industry, its

products, and/or its practices.

Institutional work

27

There has been a flourishing of research that examines the role of multimodal messages

during the various stages of institutionalization (Meyer et al., 2018; Höllerer et al., 2018), such as

the theorization of an innovation (Cartel, Colombero & Boxenbaum, 2017) or the social

construction of a rational myth (Boxenbaum, Daudigeos, Pillet & Colombero, 2018). We

elaborate on this research of when multimodal messages work, to examine how specific

combination of words and images may better accomplish institutional work.

Meyer et al. (2018) describe the when of multimodal texts that institutionalize new ideas,

based on which affordances of visual or verbal texts are better equipped to advance the

institutionalization of a novel idea. First, multimodal messages externalize new ideas and project

these into the social realm by initially placing the idea into existing knowledge structures,

creating exposure to make a novel idea available to a target audience, appealing to the target

audience, and mobilizing consensus. Second, to become institutionalized, the idea must acquire

typified meaning and be considered as a legitimate solution. Lastly, to diffuse the idea, it must be

understandable across audiences until it becomes taken-for-granted.

We contend that a well-crafted multimodal message has various affordances that may work

in each stage of institutionalization , given its interpretive ‘plasticity’ to differing audiences.

While this claim may seem excessive, examples exist of multimodal messages that are used so

repeatedly and extensively that they mobilize consensus, define legitimate solutions, and become

understandable and taken-for-granted globally. For example, the “hockey-stick graph” illustrates

the mean temperature record of the past 2000 years in the Northern Hemisphere, using climate

proxy records (ice cores, tree rings, sediment from Arctic lakes…). When correlated with

changes to atmospheric greenhouse gas concentrations, the hockey-stick graph became

persuasive evidence for connecting emissions to temperature change and, thus, anthropogenic

28

climate change; it created international motivation and consensus for promulgating the 1997

Kyoto Protocol, and it institutionalized regulations for greenhouse gas emissions. It

accomplished this through the sponsor’s choice of content (specific images and words) and

meaning construction of a text used to contextualize the idea into the broader social knowledge

structures. An unexpected, foregrounded image captures recipients’ attention. The visual/verbal

structure and meaning construction evoke emotions, motivate processing, and create

identification that is more intuitively understandable, which supports concensus-building. The

packaging of ideas into a multimodal message makes them portable – across multiple media such

as reports, newspapers, magazines, posters, and social media – and promotes diffusion and even

contagion.

Values work

In a parallel manner, our model also contributes to the exploration of links between

visual/verbal elements, emotion, and legitimacy work and usefully parallels related research on

values work – the ongoing development and practice of an organization’s values with “the

resulting values discourse becom[ing] available as a semiotic resource for justifying the

assessment and proliferation of values practices in disparate parts of an organization” (Gehman,

Treviño, & Garud, 2013, p. 104). Such values work enables individuals such as new-venture

entrepreneurs to recognize and even create new opportunities through their performative,

relational, and temporal efforts, by crafting the best message to resonate with desired audiences

at the best time (Garud, Gehman, & Giuliani, 2014; Zilber, 2006). By culturally aligning their

accounts with the expectations of investors, competitors, and consumers, entrepreneurs may

emphasize the legitimacy of their ventures from the evaluators’ perspective, resulting in their

greater access to resources (Lounsbury & Glynn, 2001; Martens, Jennings, & Jennings, 2007).

29

They may simultaneously create generalized legitimacy for an emergent industry (Navis &

Glynn, 2011).

Our model can be helpful to scholars engaged in values work because it enables them to

more convincingly explicate how words and images trigger the processes of identification and

associated emotional processes that underpin values work. Multimodal messages have great

capacity to evoke emotion and, during the process required to infer meanings, are more likely to

trigger re-evaluation of the subject matter by the recipient than textual messages or images alone.

If we apply the concepts from our theory of visual/verbal structure, meaning consruction,

and content to this work, we can examine the relative persuasiveness of variations in these

concepts, with differing audiences, and to what effect. In addition to emotional engagement with

interesting messages, recipients may also identify with the message content and meaning; the

more they resemble the subject matter and values characterized in the message, the more likely

recipients are to feel emotional identification and, ultimately, be persuaded by the message. For

example, Hart and Nisbet (2012) and Feinberg & Willer (2010) demonstrate how careful

tailoring of climate change messages to particular values can persuade science skeptics to

support climate change policies. The ideas from our framework can also be applied to extend

such studies. In particular, our work raises the possibility that the extent to which different

audiences experience identification and values resonance (Snow & Benford, 1988; Giorgi, 2017)

might be measured by introducing messages with varying subjects, by examining which

messages evoke characteristic emotional responses, and by determining which messages result in

recipients accepting, rejecting or changing their evaluations.

30

CONCLUDING REMARKS

We are swimming in a sea of words and images designed to legitimate or delegitimate, yet

organizational theorists have limited tools available to analyze such multimodal texts. To begin

to address this gap, we have developed a process model to describe how multimodal messages

evoke emotion to persuade message recipients of the (il)legitimacy of an organization and its

practices and, perhaps, motivate re-evaluation of the organization’s legitimacy. In doing so, we

connect and extend existing theory in two important ways. We leverage research on

multimodality to examine how such messages motivate cognitive and emotional processing to

present the sponsors’ critique or support of an organization or industry’s legitimacy. And we

extend the model to explain the conditions under which individuals will accept, reject or rethink

the stance presented in the message. We hope that this model will better equip researchers to

understand how multimodal messages ‘work’ within organizations, within industries and fields,

and within society as sponsors vie for definitional authority.

31

REFERENCES

Adesope, Olusola O. & Nesbit, John C. (2012). Verbal redundancy in multimedia learning environments: A meta-analysis. Journal of Educational Psychology, 104(1), 250-263.

Andersen, Thomas Hestbæk & Boeriis, Martin (2012). Relationship/participant focus in multimodal market communication. HERMES-Journal of Language and Communication in Business, 25(48), 75-94.

Aristotle (1355). The Art of Rhetoric. H. Lawsen-Tancred (trans 1985). New York: Penguin Books.

Ashforth, Blake E., Rogers, Kristie M., Pratt, Michael G. & Pradies, Camille (2014). Ambivalence in organizations: A multilevel approach. Organization Science, 25(5), 1453-1478.

Barnes, Allison & Thagard, Paul (1997). Empathy and Analog. Dialogue: Canadian Philosophical Review, 36(4), 705-720.

Bitektine, Alex & Haack, Patrick (2015). The macro and the micro of legitimacy: Towards a multi-level theory of the legitimacy process. Academy of Management Review, 40(1), 49-75.

Boxenbaum, Eva, Jones, Candace, Meyer, Renate E. & Svejenova, Silviya. (2018). Towards an articulation of the material and visual turn in organization studies. Organization Studies, 29(5-6), 597-616.

Burke, Kenneth (1984). Permanence and change: An anatomy of purpose. University of California Press.

Cartel, Melodie, Colombero, Sylvain & Boxenbaum, Eva (2017). Towards a multimodal model of theorization processes. In Markus A.Höllerer, Thibault Daudigeos, & Dennis Jancsary (Eds.), Multimodality, Meaning, and Institutions (pp. 153-182). Emerald Publishing Limited.

Christiansen, Lærke Højgaard (2018). The use of visuals in issue framing: Signifying responsible drinking. Organization Studies, 39(5-6) 665–689.

Colston, Herbert L. & Gibbs, Raymond W. (2002). Are irony and metaphor understood differently? Metaphor and Symbol, 17(1), 57-80.

Colston, Herbert L. & O'Brien, Jennifer (2000). Contrast and pragmatics in figurative language: Anything understatement can do, irony can do better. Journal of Pragmatics, 32(11), 1557-1583.

Creed, W.E. Douglas, Hudson, Bryant, Okhuysen, Gerardo A. & Smith-Crowe, Kristin (2014). Swimming in a sea of shame: Incorporating emotion into explanations of institutional reproduction and change. Academy of Management Review, 39(3), 275-301.

De Vaujany, François-Xavier & Vaast, Emmanuelle (2016). Matters of visuality in legitimation practices: Dual iconographies in a meeting room. Organization, 23(5), 763-790.

Delbaere, Marjorie, McQuarrie, Edward F. & Phillips, Barbara J. (2011). Personification in advertising. Journal of Advertising, 40(1), 121-130.

Delmestri, Giuseppe & Goodrick, Elizabeth (2016). Looking away: denial and emotions in institutional stability and change. In Joel Gehman, Michael Lounsbury & Royston Greenwood (Eds.), How Institutions Matter: From the Micro Foundations of Institutional Impacts to the Macro Consequences of Institutional Arrangements (pp. 233-271). Emerald Group Publishing Limited.

Dodd, Michael D., Balzer, Amanda, Jacobs, Carly M., Gruszczynski, Michael W., Smith, Kevin B. & Hibbing, John R. (2012). The political left rolls with the good; the political right

32

confronts the bad. Philosophical Transactions of the Royal Society of London, Biological Sciences 367(1589), 640–649.

Doerr, Nicole, Mattoni, Alice & Teune, Simon (Eds.). (2013). Advances in the Visual Analysis of Social Movements. Emerald Group Publishing Limited.

Durand, Jacques (1987). Rhetorical figures in the advertising image. In J. Umiker-Sebeok (Ed.), Marketing and Semiotics: New Directions in the Study of Signs for Sale. New York: Mouton de Gruyter, 295-318.

Elfenbein, Hillary Anger (2007). Emotion in Organizations: A Review and Theoretical Integration. Academy of Management Annals, 1(1), 315-386.

Elfenbein, Hillary Anger (2014). The many faces of emotional contagion An affective process theory of affective linkage. Organizational Psychology Review, 4(4), 326-362.

Feinberg, Matthew & Willer, Rob (2010). Apocalypse soon? Dire Messages Reduce Belief in Global Warming by Contradicting Just-World Beliefs. Psychological Science, 22(1), 34-38.

Feinberg, Matthew & Willer, Rob (2013). The moral roots of environmental attitudes. Psychological Science, 24(1), 56-62.

Ferri, Jamie, Weinberg, Anna & Hajcak, Greg (2012). I see people: The presence of human faces impacts the processing of complex emotional stimuli. Social Neuroscience, 7(4), 436-443.

Fong, Christina Ting (2006). The effects of emotional ambivalence on creativity. Academy of Management Journal, 49(5), 1016-1030.

Forbus, Kenneth D. (2001). Exploring Analogy in the Large, In Dedre Gentner, Keith J. Holyoak, & Boicho N. Kokinow (Eds.), The Analogical Mind: Perspectives from Cognitive Science (pp. 199-253), Cambridge, MA: The MIT Press.

Frijda, Nico H. (1986). The Emotions. Cambridge: Cambridge University Press.Frijda, Nico H. (1988). The laws of emotion. American Psychologist, 43(5), 349-358.Garud, Raghu, Gehman, Joel, & Giuliani, Antonio Paco (2014). Contextualizing entrepreneurial

innovation: A narrative perspective. Research Policy, 43(7), 1177-1188. Gehman, Joel, Treviño, Linda K., & Garud, Raghu (2013). Values work: A process study of the

emergence and performance of organizational values practices. Academy of Management Journal, 56(1), 84-112.

Gibbs Jr., Raymond W. (1994). The Poetics of the Mind. Cambridge: Cambridge University Press.

Giorgi, Simona (2017). The Mind and Heart of Resonance: The Role of Cognition and Emotions in Frame Effectiveness. Journal of Management Studies, 54(5), 711-738.

Gkiouzepas, Lampros & Hogg, Margaret K. (2011). Articulating a new framework for visual metaphors in advertising. Journal of Advertising, 40, 103-120.

Graham, Jesse, Nosek, Brian A., Haidt, Jonathan, Iyer, Ravi, Koleva, Sspassena & Ditto, Peter H. (2011). Mapping the moral domain. Journal of Personality and Social Psychology, 101(2), 366-385.

Graves, Heather (2011). ‘The Women’s Parliament:’ Political Oratory, Humor, and Social Change. Gender Forum, 33, 41-51.

Groupe-μ. (1992). Traité du signe visuel. Pour une rhétorique de l’image. Paris: Editions du Seuil.

Haidt, Jonathan & Graham, Jesse (2007). When morality opposes justice: Conservatives have moral intuitions that liberals may not recognize. Social Justice Research, 20(1), 98-116.

Haidt, Jonathan & Joseph, Craig (2004). Intuitive ethics: How innately prepared intuitions generate culturally variable virtues. Daedalus, 133(4), 55-66.

33

Haidt, Jonathan & Joseph, Craig (2008). The moral mind: How five sets of innate intuitions guide the development of many culture-specific virtues, and perhaps even modules. The Innate Mind, 3, 367-391.

Hart, P. Sol & Nisbet, Erik C. (2012). Boomerang effects in science communication: How motivated reasoning and identity cues amplify opinion polarization about climate mitigation policies. Communication Research, 39(6), 701-723..

Hatfield, Katherin.L., Hinck, Ashley & Birkholt, Marty J. (2007). Seeing the Visual in Argumentation: A Rhetorical Analysis of UNICEF Belgium's Smurf Public Service Announcement. Argumentation and Advocacy, 43(3-4), 144-151.

Höllerer, Markus A., Jancsary, Dennis, Meyer, Renate E. & Vettori, Oliver (2013). Imageries of Corporate Social Responsibility: Visual Recontextualization and Field-Level Meaning. In Michael Lounsbury & Eva Boxenbaum (Eds.), Institutional Logics in Action, Part B. Research in the Sociology of Organizations, 39, 139-174. Emerald.

Höllerer, Markus A., Jancsary, Dennis & Grafström, Maria (2018). ‘A Picture is Worth a Thousand Words’: Multimodal Sensemaking of the Global Financial Crisis. Organization Studies, 39(5-6), 617–644.

Holyoak, Keith J. & Thagard, Paul (1997). The analogical mind. American Psychologist, 52(1), 35-44.

Hochschild, Arlie R. (1979). Emotion work, feeling rules, and social structure. American Journal of Sociology, 85(3), 551-575.

Huy, Quy Nguyen (2012). Emotions in Strategic Organization: Opportunities for impactful research. Strategic Organization, 10(3), 240-247.

Huy, Quy Nguyen., Corley, Kevin & Kraatz, Matthew (2014). From Support to Mutiny: Shifting Legitimacy Judgments and Emotional Reactions Impacting the Implementation of Radical Change. Academy of Management Journal, 57(6), 1650-1680.

Jancsary, Dennis, Höllerer, Markus. A. & Meyer, Renate E. (2016). Critical analysis of visual and multimodal texts. In Ruth Wodak, & Michael Meyer (Eds.), Methods of critical discourse studies (3rd ed., pp. 180–204). London: SAGE Publications.

Jancsary, Dennis, Meyer, Renate E., Höllerer, Markus A. & Boxenbaum, Eva (2018). Institutions as multimodal accomplishments: Towards the analysis of visual registers. Research in the Sociology of Organizations, 54(A), 87–117.

Jasper, James M. (2011). Emotions and Social Movements: Twenty years of theory and research. Annual Review of Sociology, 37, 285-303

Johar, Gita Venkataramani (1995). Consumer Involvement and Deception from Implied Advertising Claims. Journal of Marketing Research, 32(3), 267-279.

Jones, Michael A., Taylor, Valerie A. & Reynolds, Kristy E. (2014). The Effect of Requests for Positive Evaluations on Customer Satisfaction Ratings. Psychology & Marketing, 31(3), 161-170.

Kardes, Frank R. (1988). Spontaneous inference processes in advertising: The effects of conclusion omission and involvement on persuasion. Journal of Consumer Research, 15(2), 225-233.

Ketelaar, Paul E., and Van Gisbergen, Marnix N. (2006). Openness in Advertising: Occurrence and Effects of Open Advertisements in Magazines. Sociale Wetenschappen, Radboud Universiteit Nijmegen, Proefschrift.

Kress, Gunther R. & Van Leeuwen, Theo (2001). Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold Publishers.

34

Kress, Gunther R. & van Leeuwen, Theo (2006). Reading Images: The Grammar of Vvisual Design (2nd Ed.). London et al.: Routledge Publishers.

Kress, Gunther (2010). Multimodality: A social semiotic approach to contemporary communication. London et al.: Routledge Publishers.

Lagerwerf, Luuk & Meijers, Anoe (2008). Openness in Metaphorical and Straightforward Advertisements. Journal of Advertising, 37(2), 19–30.

Lang, Peter J., Greenwald, Mark K., Bradley, Margaret M., & Hamm, Alfonso. (1993). Looking at pictures: Affective, facial, visceral, and behavioral reactions. Psychophysiology, 30(3), 261-273.

Lang, Peter J., Bradley, Margaret M. & Cuthbert, Bruce N. (1997). Motivated attention: Affect, activation, and action. In Peter J. Lang, Robert F. Simons, Marie Balaban & Robert Simons (Eds.), Attention and orienting: Sensory and motivational processes (pp. 97-135). Hillsdale, NJ: Lawrence Erlbaum Associates.

Lazarus, Richard S. (1991). Emotion and adaptation. Oxford: Oxford University Press.Lefsrud, L.M., Graves, H. & Phillips, N. (2016). Analyzing Visual Rhetoric in Organizational

Research. In Kimberly D. Elsbach & Roderick M. Kramer, R.M. (Eds.), Handbook of Qualitative Organizational Research: Innovative Pathways and Methods (pp. 225-237). London et al.: Routledge Publishers.

Lemerise, Elizabeth A. & Arsenio, William F. (2000). An integrated model of emotion processes and cognition in social information processing. Child Development, 71(1), 107-118.

Lerner, Jennifer S. & Keltner, Dacher (2000). Beyond valence: Toward a model of emotion-specific influences on judgement and choice. Cognition & Emotion, 14(4), 473-493.

Lerner, Jennifer S. & Keltner, Dacher (2001). Fear, anger, and risk. Journal of Personality and Social Psychology, 81(1), 146-159.

Lounsbury, Michael & Glynn, Mary Ann. (2001). Cultural entrepreneurship: Stories, legitimacy, and the acquisition of resources. Strategic Management Journal, 22(6-7), 545-564.

Maes, Alfons & Schilperoord, Joost (2008). Classifying Visual Rhetoric: Conceptual and Structural Heuristics. In Edward F. McQuarrie& Barbara J. Phillips (Eds.), Go figure! New Directions in Advertising Rhetoric (pp. 227-253). Armonk, NY: ME Sharpe.

Maguire, Steve & Hardy, Cynthia (2009). Discourse and Deinstitutionalization: The Decline of DDT. Academy of Management Journal, 52(1), 148-178.

Maitlis, Sally, Vogus, Timothy J. and Lawrence, Thomas B. (2013). Sensemaking and emotion in organizations. Organizational Psychology Review, 3(3), 222-247.

Martens, Martin L., Jennings, Jennifer E., & Jennings, P. Devereaux (2007). Do the stories they tell get them the money they need? The role of entrepreneurial narratives in resource acquisition. Academy of Management Journal, 50(5), 1107-1132.

McNaughton, Melanie J. (2007). Hard cases: Prison tattooing as visual argument. Argumentation and Advocacy, 43 (3-4), 133-143.

McQuarrie, Edward F. & Phillips, Barbara J. (2005). Indirect Persuasion in Advertising: How Consumers Process Metaphors Presented in Pictures and Words. Journal of Advertising, 34(2), 7-20.

Meyer, John W. & Rowan, Brian, (1977). Institutionalized organizations: Formal structure as myth and ceremony. American Journal of Sociology, 83(2), 340-363.

Meyer, Renate E., Höllerer, Markus A., Jancsary, Dennis., & van Leeuwen, Theo (2013). The visual dimension in organizing, organization, and organization research: Core ideas, current developments, and promising avenues. The Academy of Management Annals, 7, 489-555.

35

Meyer, Renate E., & Höllerer, Markus A. (2016). Laying a smoke screen: Ambiguity and neutralization as strategic responses to intra-institutional complexity. Strategic Organization, 14(4), 373-406.

Meyer, Renate E., Jancsary, Dennis, Höllerer, Markus A., & Boxenbaum, Eva (2018). The role of verbal and visual text in the process of institutionalization. Academy of Management Review, 43(3), 392-418.

Moisander, Johanna. K., Hirsto, Heid & Fahy, Kathryn M. (2016). Emotions in institutional work: A discursive perspective. Organization Studies, 37(7), 963-990.

Monahan, Jennifer L., Murphy, Sheila T. & Zajonc, Robert B. (2000). Subliminal mere exposure: Specific, general, and diffuse effects. Psychological Science, 11(6), 462-466.

Mothersbaugh, David L. , Huhmann, Bruce A., & Franke, George R. (2002). Combinatory and separative effects of rhetorical figures on consumers’ effort and focus in ad processing. Journal of Consumer Research, 28(4), 589–602.

Navis, Chad & Glynn, Mary Ann. (2011). Legitimate distinctiveness and the entrepreneurial identity: Influence on investor judgments of new venture plausibility. Academy of Management Review, 36(3), 479-499.

Öhman, Arne, Flykt, Anders & Esteves, Franscisco (2001). Emotion drives attention: Detecting the snake in the grass. Journal of Experimental Psychology General, 130(3), 466–478.

Peirce, Charles Sanders (1931). Collected papers of Charles Sanders Peirce, 1860–1911 (Vol. I–VIII), Charles Hartshorne, Paul Weiss and Arthur Burks (Eds.).Cambridge: Harvard University.

Phillips, Barbara J. & McQuarrie, Edward .F. (2004). Beyond visual metaphor: A new typology of visual rhetoric in advertising. Marketing Theory, 4(1-2), 113-136.

Phillips, Barbara J. (2000), The Impact of Verbal Anchoring on Consumer Response to Image Ads. Journal of Advertising, 29(1), 15–25.

Pineda, Richard D. & Sowards, Stacey K. (2007). Flag waving as visual argument: 2006 immigration demonstrations and cultural citizenship. Argumentation and Advocacy, 43(3-4), 164-174.

Plambeck, Nils & Weber, Klaus (2009). CEO ambivalence and responses to strategic issues. Organization Science, 20(6), 993–1010.

Puyou, François-Régis & Quattrone, Paolo (2018). The Visual and Material Dimensions of Legitimacy: Accounting and the Search for Societies. Organizaition Studies, 39(5-6), 721-746.

Rees, Laura, Rothman, Naomi B., Lehavy, Reuven., & Sanchez-Burks, Jeffrey. (2013). The ambivalent mind can be a wise mind: Emotional ambivalence increases judgment accuracy. Journal of Experimental Social Psychology, 49(3), 360-367.

Rindova, Violina P., Pollock, Timothy G. & Hayward, Mathew L.A. (2006). Celebrity firms: The social construction of market popularity. Academy of Management Review, 31(1), 50-71.

Rindova, Violina., Dalpiaz, Elenia, & Ravasi, Davide. (2011). A cultural quest: A study of organizational use of new cultural resources in strategy formation. Organization Science, 22(2), 413-431.

Rosch, Eleanor. (1975). Cognitive representations of semantic categories. Journal of Experimental Psychology: General, 104(3), 192-233.

36

Rothman, Naomi B. & Melwani, Shimul. (2017). Feeling mixed, ambivalent, and in flux: The social functions of emotional complexity for leaders. Academy of Management Review, 42(2), 259-282.

Scherer, Klaus R. (1984). On the nature and function of emotion: A component process approach. In Klaus R. Scherer, & Paul Ekman (Eds.), Approaches to Emotion (pp. 293-318). Hillsdale, NJ: Lawrence Erlbaum Associates.

Schriver, Karen A. (1997). Dynamics in Document Design: Creating Texts for Readers. New York: Wiley.

Schwarz, Gavin M., Wong, Kin Fal Ellick & Kwong, Jessica Y.Y. (2014). The Role of Regret in Institutional Persistence and Change. Journal of Change Management, 14(3), 309–333.

Shelley, Cameron (2001). Aspects of visual argument: A study of the March of Progress. Informal Logic, 21(2), 85-96.

Shortt, Harriet L. & Warren, Samantha K. (2018). Grounded visual pattern analysis: Photographs in organizational field studies. Organizational Research Methods, 1094428117742495.

Smith, Craig A. & Ellsworth, Phoebe C. (1985). Patterns of cognitive appraisal in emotion. Journal of Personality and Ssocial Psychology, 48(4), 813-838.

Smith, Ruth Ann (1991), The Effects of Visual and Verbal Advertising Information on Consumers' Inferences, Journal of Advertising, 20 (4), 13-24.

Snow, David A. & Benford, Robert D. (1988). Ideology, frame resonance, and participant mobilization. International Social Movement Research, 1(1), 197–217.

Snow, David A. Benford, Robert, McCammon, Holly, Hewitt, Lyndi, & Fitzgerald, Scott (2014). The Emergence, Development, and Future of the Framing Perspective: 25+ Years Since ““Frame Alignment”. Mobilization: An International Quarterly, 19(1), 23-46.

Sojka, Jane Z. & Giese, Joan L. (2006). Communicating through pictures and words: Understanding the role of affect and cognition in processing visual and verbal information. Psychology & Marketing, 23(12), 995-1014.

Sopory, Pradeep & Dillard, James Price. (2002). The persuasive effects of metaphor: a meta‐analysis. Human Communication Research, 28(3), 382-419.

Suchman, Mark C. (1995). Managing legitimacy: Strategic and institutional approaches. Academy of Management Review, 20(3), 571-610.

Suddaby, Roy & Greenwood, Royston (2005). Rhetorical Strategies of Legitimacy. Administrative Science Quarterly, 50(1), 35-67.

Thagard, Paul & Shelley, Cameron (2001). Emotional analogies and analogical inference. In Dedre Gentner, Keith J. Holyoak, Keith James Holyoak, & Boicho N. Kokinov (Eds.), The analogical mind: Perspectives from Cognitive Science (pp. 335-362). Cambridge: MIT Press.

Thagard, Paul (2011). The brain is wider than the sky: Analogy, emotion, and allegory. Metaphor and Symbol, 26(2), 131-142.

Toubiana, Madeline & Zietsma, Charlene (2017). The Message is on the Wall? Emotions, Social Media and the Dynamics of Institutional Complexity. Academy of Management Journal, 60(3), 922-953..

Treviño, Linda Klebe, den Nieuwenboer, Niki A., Kreiner, Glen E., & Bishop, Darren G. (2014). Legitimating the legitimate: A grounded theory study of legitimacy work among Ethics and Compliance Officers. Organizational Behavior and Human Decision Processes, 123(2), 186-205.

37

Vaara, Eero, Tienari, Janne & Laurila, Juha (2006). Pulp and Paper Fiction: On the Discursive Legitimation of Global Industrial Restructuring. Organization Studies, 27(6), 789-810.

Van Leeuwen, Theo (2005). Introducing Social Semiotics. New York: Routledge.Van Mulken, Margot, Van Enschot, Renske & Hoeken, Hans (2005). Levels of Implicitness in

Magazine Advertisements: An Experiment into the Relationship Between Complexity and Appreciation in Magazine Advertisements. Information Design Journal + Document Design, 13(2), 155–164.

Voronov, Maxim & Vince, Russ (2012). Integrating emotions into the analysis of institutional work. Academy of Management Review, 37(1), 58-81.

Voronov, Maxim & Weber, Klaus (2016). The heart of institutions: Emotional competence and institutional actorhood. Academy of Management Review, 41(3), 456-478.

Whaley, Bryan B. & Holloway, Rachel L. (1996). ‘Rebuttal Analogy’: A theoretical note. Metaphor and Symbolic Activity, 11(2), 161-167.

Whaley, Bryan B. & Wagner, Lisa Smith (2000). Rebuttal Analogy in Persuasive Messages Communicator Likability and Cognitive Responses. Journal of Language and Social Psychology, 19(1), 66-84.

Wischmann, Lesley (1987). Dying on the front page: Kent State and the Pulitzer Prize. Journal of Mass Media Ethics, 2(2), 67-74.

Zhang, Shi, Kardes, FrankR. & Cronley, Maria L. (2002). Comparative Advertising: Effects of Structural Alignability on Target Brand Evaluations. Journal of Consumer Psychology, 12(4), 303-311.

Zietsma, Charlene, Toubiana, Madeline, Voronov, Maxim & Roberts, Anna (2018). Emotions In Organization Theory. Cambridge: Cambridge University Press.

Zilber, Tammar B. (2006). The work of the symbolic in institutional processes: Translations of rational myths in Israeli high tech. Academy of Management Journal, 49(2), 281-303.

38

Figure 1 – Process model for multimodal messages invoking emotions and legitimacy work

39

Figure 2 – Environmental organizations’ “Canada [Not just mounties and ice hockey anymore]”

40

Figure 3 – Carbusters Journal of the Carfree Movement “No exit: Keystone XL Pipeline” Available online at: http://carbusters.org/2012/02/12/no-exit-keystone-xl-pipeline/

41