congruence of meaning between facial expressions of ...nalvarado/pdfs/alvarado1996motemotion.pdf ·...

Motivation and Emotion, Vol. 20, No. 1, 1996

Congruence of Meaning Between Facial Expressions of Emotion and Selected Emotion Terms 1

Nancy A l v a r a d o 2

Universify of California, Irvine

How similar are the meanings of facial expressions of emotion and the emotion terms frequently used to label them? In three studies, subjects made similarity judgments and emotion self-report ratings in response to six emotion categories represented in Ekman and Friesen's Pictures o f Facial Affect, and their associated labels. Results were analyzed with respect to the constituent facial movements using the Facial Action Coding System, and using consensus analysis, mult idimensional scaling, and inferential statistics. Shared interpretation o f meaning was found between individuals and the group, with congruence between the meaning in facial expressions, labeling using basic emotion terms, and subjects' reported emotional responses. The data suggest that (1) the general labels used by Ekman and Friesen are appropriate but may not be optimal, (2) certain facial movements contribute more to the perception of emotion than do others, and (3) perception of emotion may be categorical rather than dimensional

1Preparation of this article was supported in part by National Institute of Mental Health grant MH18931 to Paul Ekman and Robert Levenson for the NIMH Postdoctoral Training Program in Emotion Research. Portions of this work were completed as part of a doctoral dissertation (see references). I gratefully acknowledge the assistance of Louis Narens, William H. Batchelder, Virginia Lee, and Paul Ekman in the completion of this work. I also thank William Irwin for his help with FACS coding, and Shirley Wang and the other UC Irvine Semiotics Laboratory research assistants for their help with data collection. Jerome Kagan, Nancy Etcoff, Kimberly Jameson, and several anonymous reviewers provided helpful comments on earlier versions of this manuscript.

2Address all correspondence concerning this article to Nancy Alvarado, Department of Psychology, Harvard University, 33 KirHand Street, Cambridge, Massachusetts 02138.

33

0146-7239/9~0300-0033509.50/0 0 1996 Plenum Publishing Corporation

34 Alvarado

Isomorphism between the meaning conveyed by facial expressions of emotion and that conveyed by emotion terms has been suggested by Osgood (1966), Numraenmaa (1964), and more recently by Rnssel, Lewicka, and Niit (1989), following multidimensional scaling (MDS) studies of facial expression by Schlosberg (1954), and Abelson and Sermat (1962). These authors have noted the similarity of interpretable dimensions across the two domains of meaning (Osgood, 1966), the roughly circular shape when large sets of items are scaled (later called the emotion circumplex by Russell, 1980), and the tendency to assign emotion terms from one portion of space in the verbal domain to facial expressions inhabiting the corresponding portion of space in the visual domain (Nummenmaa, 1964).

However, Osgood (1966) noted, "despite this isomorphism, there is obviously a rather loose coupling between names and things in this semantic domain . . ." (p. 27). Osgood suggested that the domain of meaning in facial expressions of emotion is less finely differentiated than that of emotion terms, and that terms include "denotative differentiations within constant affective cores," whereas facial expressions "communicate affective meaning primarily and denotative or referential meaning only secondarily" (p. 27). More recent authors have made similar statements about the relationship between emotion terms and facial expressions in particular (cf. Ortony & Turner, 1990) as well as between emotion terms and emotional states (Levy, 1984). Further, based upon this loose coupling, some researchers have questioned whether variability of response exists in language or in the phenomena that language is used to describe (Russell, 1994; Or- tony & Turner, 1990; Turner & Ortony, 1992). Fridlund (1994) has suggested that this loose coupling supports his view that so-called emotional facial expressions convey not emotions but social inclinations.

This paper presents a statistical demonstration of the congruence of meaning between selected facial expressions of emotion and certain associated verbal labels. The method demonstrated here provides a quantitative measure of the similarity of shape in the patterns of meaning within com- parable sets of items. Sets of items with the most similar patterns of meaning will show the greatest congruence across domains, and their MDS plots should look the most similar. While the separate elements of this methodology are well known, their combined application is new, and is patterned after a paradigm developed by Shepard and Cooper (1992) to study the second-order isomorphism between mental representations of color (color term labels) and the colors themselves (Munsell chips). Like that study, this work compares the "functional relation between two internal representations" (Shepard & Chipman, 1970, p. 2) as expressed in similarity judgments among items in two conditions: (1) when stimuli are actually presented, and (2) when they are only named (Shepard & Cooper, 1992).

Testing Congruence of Emotional Meaning 3S

As Shepard and Chipman (1970) noted, "we can do this even though (a) we have never before compared the two representations in question, and even though (b) we may be unable to communicate anything about the absolute nature of either of the two representations taken separately" (p. 2). It is the relationship among the meanings of the items that is being measured, independent of what those meanings might be or how they are represented.

Shepard and Cooper (1992) selected their color names to correspond one-to-one with the colors presented. This paper tests the correspondence between a set of proposed emotion-term names and associated facial expressions that have evoked controversy in the literature (Ekman, 1994; Russell, 1994). It proposes that the second-order isomorphism between a set of mental representations in the form of semantic labels and a set of external objects, in this case facial expressions, is a measure of the similarity of meaning between the internal mental representations evoked by the labels and the objects. Therefore, if the facial expressions used in this study are appropriately labeled, the result will be a similarity of meaning relationship among the items within each domain, Shepard and Chipman's (1970) second-order isomorphism.

It is worthwhile pointing out differences between this and previous scaling studies of emotion. First, previous studies were designed to explore the dimensionality of emotion space. This research is not concerned with dimensionality, nor with interpreting the adjacencies of emotions in either domain. Second, this research uses objectively described stimuli. In early work, meaningful comparisons of results across studies were difficult because, without a system for describing facial movement, like the current Facial Action Coding System (FACS), there was no way to determine whether one researcher's "anger" photo presented the same facial activity as the next researcher's "anger" photo. Third, while many studies have presented individual exemplars of a range of facial expressions of emotion, few have examined whether similarly labeled expressions with different constituent facial movements convey the same affective meaning. This research compares multiple exemplars of a similarly labeled emotion, to see whether similar meaning is derived by viewers, and explores whether that meaning results from similarity of facial movement.

It is also worthwhile to address in advance several technical issues related to the use of MDS. The researchers cited above did not validate their solutions, nor did they report any method of assuring that their group solutions provided a meaningful picture of individual response patterns. This research presents MDS plots to illustrate the patterns existing in the data (and as a reference point for comparisons with previous work) but makes its arguments using traditional inferential statistics. Further, the congruence

36 Alvarado

between domains is demonstrated not by the shapes of the MDS plots, but on an individual-by-individual basis with reference to each subject's responses. Where MDS plots based on aggregated data are presented, their validity is confirmed using consensus analysis, a quantitative measure of the homogeneity of response within a group. When consensus exists among subject responses, the group plot based upon their averaged or aggregated data can be expected to have roughly the same shape as most of the plots based upon individual data. The consensus model also provides a theoreti- cal grounding that links the results for a particular random sample to the population from which it is drawn.

Roberts and Wedell (1994) presented an interesting demonstration of how stimulus sampling can affect similarity judgments made by subjects, and thus the configuration of items within an MDS plot. 3 Their findings are important for studies of the dimensionality of emotion space or of the relationship between items within that space (i.e., for theories like the circumplex model of emotion, Russell, 1980). However, their findings are irrelevant to the work presented here because this work makes comparisons across domains, not within the same domain. Because the number and type of constituent items are held constant within each domain, stimulus sampling can be expected to have the same effect on both domains. If it does not, it will work against a finding of congruence of meaning. Further, certain context effects created by stimulus sampling, as noted by Roberts and Wedell, have their greatest impact on ambiguous items, in paradigms where the context of judgment is not varied. The work described here avoids these problems by maximally varying the context of judgment through the use of a triad task presented in a complete design or in a balanced incomplete block design. This design is similar to the paired-comparison method in which Roberts and Wedell found that context effects were eliminated. If a context effect exists, it should also work against any finding of similarity of meaning among similarly labeled items within a category.

Three studies are presented below. The first two demonstrate congruence of meaning across the visual and verbal domains for six emotion terms and six facial expressions of emotion purported to display basic emotions by Ekman and Friesen, selected from their published Pictures of Facial Af- fect (Ekman & Friesen, 1978a). The third study presents three exemplars of each of the six emotion categories, to determine whether similarly labeled facial expressions are treated similarly by subjects in the same judgment task. The results were analyzed with respect to facial movements using Ekman and Friesen's Facial Action Coding System. The stimuli used in these studies were selected for three reasons: (1) They were created by

3I am indebted to James Russell for bringing this work to my attention.

Testing Congruence of Emotienal Meaning 37

asking an expressor to pose specific sets of facial movements objectively classified using Ekman and Friesen's FACS (Ekman & Friesen, 1976); (2) they have been extensively tested in empirical judgment studies and have high English-language reliability ratings for the emotion each purports to display; (3) their relationship to the emotion terms used during such testing is being questioned in the literature (see Ekman, 1994; Russell, 1994). The labels presented are the accompanying, loosely coupled, emotion-category names attached to the faces by Ekman and Friesen (1978a).

EXPERIMENTS 1 AND 2

Two studies tested whether congruence of meaning exists between facial expressions of emotion and the emotion terms used to label them. Ex- periment 2 repeated the procedures of Experiment 1 using facial expressions posed by a different expressor. Because the expressions used in Experiment 2 had slightly lower reliability judgments and the emotion terms were slightly different from those used in Experiment 1, the second study provided a test of the sensitivity of the method and the impact upon the measurement of congruence between domains under conditions where the stimuli varied in subtle ways. 4

Method

Subjects

For Experiment 1, subjects were 36 male and female undergraduate student volunteers at the University of California, Irvine, of diverse ethnic backgrounds (for detail see the Results section below). For Experiment 2, subjects were 29 male and female volunteers recruited from the same s o u r c e .

Materials

Stimulus items consisted of six black and white photographs exhibiting posed facial expressions previously classified by Ekman and Fdesen (1978), and their associated verbal labels. For Experiment 1, the labels were: fear,

4See Alvarado (1993) for a demonstration of the lack of congruence across domains when names and faces do not match, as when facial expressions are tested against the given names of the expressors.

38 Alvarado

anger, joy, surprise, disgust, and sadness. For Experiment 2, the term happy was used instead of joy, and sad was used instead of sadness. For Experi- ment 1, the six photos showed the highest overall percentage of judgments for the intended labels of any available set, as rated during validation studies conducted by Ekman and Friesen. The photos portraying joy, anger, and surprise were rated at 100%. For Experiment 2, the set producing the second-highest reliability ratings was used. In Experiment 1, the expressor was a young woman; in Experiment 2, a middle-aged man. Within each experiment, the same person was shown in each photo; thus all details of appearance (e.g., sex, skin and hair color, hair style) except facial expression were held constant.

Procedure

Three tasks were presented during a single session: (1) a visual triad task using photos of basic expressions, (2) a verbal triad task using the corresponding labels for those expressions, and (3) an emotional response rating task. The order of presentation of the visual and verbal triad tasks was counterbalanced. The emotional response rating task was always presented last, to avoid biasing subject responses in the visual triad task by mentioning emotion. In Experiment 2, the emotional response rating task was presented twice, the second time in reverse order from the first, with a demographic questionnaire intervening. This permitted a test of the effect of presentation order upon emotional response. With six stimulus items, a complete triad design was used in which each item appeared with each other item four times, and all possible combinations of three items were presented (a total of 20 trials). Item position within a triad was balanced.

For the visual triad task, subjects were given written instructions which asked them to select the picture that was most different among the three. No further judgment criteria were considered necessary because facial expression was the only element varying. The word emotion was never used, nor was any synonym for it (e.g., feelin~ affect). Subjects were then presented by the experimenter with a series of sets of three photos.

For the verbal triad task, subjects were given written instructions which asked them to select the word that was most different among the three based on "meaning." An example was provided involving concrete objects. Although a different random order was used for each subject, to maintain a consistent context of judgment across the two triad tasks, emotion terms were presented in exactly the same order as the corresponding visual items in the visual triad task. Terms were presented in sets of three words per line in a questionnaire booklet.

Testing Congruence of Emotional Meaning 39

During the emotional response rating task, subjects were presented with the same six photos, one at a time, and asked to inspect each photo carefully and notice both the quality and intensity of their own emotional response to that photo. They were then asked to quantify that response on a rating scale from I to 5, where 1 indicated a strong negative response, 5 indicated a strong positive response, and 3 indicated a neutral response.

In Experiment 1, subjects were tested individually and a different random order of triads was presented. In Experiment 2, subjects were tested in small groups of 4 to 15 subjects, with the same random order of triads presented within each group, but a different order across groups.

Results and Discussion

Consensus Analysis

Consensus analysis (Batchelder & Romney, 1988, 1989) was performed using the multiple-choice model (Borgatti, 1990). Consensus analysis produces a probabilistic estimate of the extent to which each subject accesses shared knowledge (called a competence rating), plus confidence estimates for the correctness of each potential response, using a quantitative model that incorporates concepts from test theory, latent class analysis, and signal detection theory. It assumes that if responses across subjects are correlated, it is because the responses are also correlated with latent shared knowledge accessed by subjects. Thus the pattern of correlations can be used to figure out what subjects know. Idiosyncratic responses are readily apparent in the measures provided by this modeling tool (i.e., the individual competence ratings,and the mean competence for the group); thus it is useful for de- termining homogeneity of group response, as well as for studying how diversity affects response in a particular context.

Consensus analysis results for all three experiments are summarized in 1hble I. Results are shown for the facial expression triad task (hereafter called the visual task), for the emotion term triad task (hereafter called the verbal task), and for the emotional response rating task (see Alvarado, 1993, for a more extensive analysis of the emotional response ratings). For both Experiments 1 and 2, consensus analysis showed a good fit for the model and homogeneity of response, permitting the MDS plots based upon group results to be interpreted confidently. Further, consensus results generated sufficiently high confidence levels for the estimated answer key that the data from this sample can be generalized to other random samples.

A discussion of how emotion-related meaning is represented for the two judgment conditions can become quite complex and is unnecessary in

40 Alvarado

Table L Comparison of Consensus Analysis Mean Competence Rating Scores

Experiment Visual Verbal Emotional No. Group triads triads response

1 All subjects .646 .699 .557 Native English speakers (n = 24) .664 .771 Nonnative English speakers (n = 12) .616 .554

2 All subjects .567 .557 .520 (second rating) .526

Native English speakers (n = 20) .555 .612 .738 ~

Normative English speakers (n = 9) .587 .446

3 All subjects .694 .632 (second rating) .656

aReanalyzed with divergent subjects omitted.

order to test the goodness of fit between the meanings evoked by the facial expressions and the tested labels. However, consensus analysis results suggest that subjects did not perform both tasks using the same semantic representations. In other words, it is unlikely that subjects performed the visual task by spontaneously labeling each stimulus with one of the semantic terms used in the verbal task, then comparing their self-generated terms as they did in the verbal task. Nor did they perform the verbal task by visualizing facial expressions equivalent to each term, then comparing the resulting images.

That the representations are not identical for the two sets of stimuli can be seen most readily through a comparison between bilingual and monolingual subjects. If representations are semantic for both sets of stimuli, then diversity of English-speaking ability should affect consensus for performance of both tasks. Inclusion of diverse subjects from differing cultural backgrounds and speaking different native languages encourages lesser consensus through wider choice of meanings and consequent labels for visual stimuli, and through the use of languages other than English in the visual domain. Under such circumstances, any consensus or congruence of meaning across the visual and verbal domains becomes a stronger finding.

Experiments 1 and 2 present a comparison between consensus results for native speakers of English and a mixed group speaking a variety of other native languages, for whom English was a second language. For both Experiments 1 and 2, significant differences due to nonnative English- speaking ability were found in the verbal triad task consensus analysis re-


suits, but not in the visual triad task results (see 'lhble I). Subjects were divide~ into two subsets based upon whether English was their native language (first language learned after birth). For Experiment 1, nine subjects learned a language other than English and three were bilingual from birth. Their native languages were: 1 Vietnamese, 2 Korean, 2 Spanish, 4 Chinese (3 Mandarin, 1 Cantonese), 2 ~galog, 1 Marathi (Bombay, India). These 12 subjects were analyzed separately from the 24 native speakers of English. Bilingual subjects were included with normative speakers to provide a larger sample for consensus analysis. Their presence in the nonnative group re- duces the likelihood of finding differences between the two groups. Diver- sity was similar among subjects for Experiment 2, with 20 native speakers of English and 9 normative speakers.

For Experiment 1, a one-tailed, paired-sample t-test showed no significant difference between the visual and verbal task consensus analysis competence ratings for normative speakers [difference M = .062, t(11) = .50, p = .628], but significantly higher verbal competence scores than visual scores for the native speakers [difference M ffi -.107, t(23) = -1.94, p = .033]. Similarly, Experiment 2 showed no significant difference between the visual and verbal task consensus analysis competence ratings for normative speakers [difference M = .141, t(8) ffi 1.04, p = .327], but significantly higher verbal competence scores than visual scores for the native speakers [difference M = .156, t(17) -- -2.23, p = .039].

For Experiment 1, a between-group, single-factor analysis of variance (ANOVA) showed no significant difference in mean competence between the native and normative speakers on the facial expression triad task IF(l, 34) ffi .233, p ffi .632], but significantly greater mean competence for the native speakers (M ffi .771) compared to the normative speakers (M = .554) [F(1, 34) = 5.682, p = .023] on the emotion-term triad task. Similarly, Ex- periment 2 showed no significant difference in mean competence between the native and normative speakers on the facial expression triad task [F(1, 25) ffi .354, p = .557], but significantly greater mean competence for the native speakers (M ffi .698) compared to the normative speakers (M ffi .446) [F(1, 25) = 5.357, p -- .029] on the emotion-term triad task. No significant differences were found for the emotional response rating task.

Whether subjects responded directly to features of the visual stimuli, monitored some internal subjective affective response to the facial expressions, or generated verbal labels as a mediator of their response, the lower consensus among native speakers on the visual task compared to the verbal task suggests that the meaning evoked by the visual stimuli is less constrained or more complex than is the meaning of the emotion terms used in the verbal task. This is consistent with Osgood's (1966) view that the meaning of facial expressions may be primarily affective while that of emo-

42 Aivarado

tion terms may be primarily denotative. It also suggests that affective meaning is less impacted by cultural diversity than is denotative meaning. Con- sensus among both native speaking and normative groups was not substantially different on the visual task, but was different on the verbal task. Further, although negative competence scores were produced on the verbal task in Experiment 2, 5 no negative competence scores were found for any subjects performing visual tasks, where greater divergence might be expected. These fmdings suggest that whatever cultural and language differences existed among the normative speakers, they did not affect their agreement with native speakers about the meanings of the visual stimuli. Obviously, the diversity of backgrounds among the normative speakers made this far from the kind of cross-culturfl comparison group that might be used in a study designed to explore this issue (as this was not), yet these findings of no impact on the visual triad and emotional response rating tasks are nonetheless interesting.

Multidimensional Scaling

Multidimensional scaling was used as a convenience, to make more evident any similarity in the patterns existing in the verbal and visual triad data. Note that these patterns were then confirmed on an individual-by- individual basis, with respect to the data itself (see the Data Analysis section below).

Two-dimensional solutions for the visual and verbal triad data collected in Experiment 1 were plotted using MDS(X) Minissa nonmetric multidimensional scaling routines (Borgatti, 1990), 6 and are shown in Figs. 1 and 2, respectively. Stress and dimensionality were not analyzed because, with six items, low stress and low dimensionality are to be expected regardless of the characteristics of the domain. The unambiguous labeling of the six photos in experiments conducted by Ekman and Friesen (1978a) indicates that the items used in this study were sufficiently distinct from each other

5Three divergent subjects showed negative competence scores in Experiment 2. Two were native speakers who showed nearly significant correlation (r = .38) in their responses. Examination of their responses showed a tendency to select a noun as most different among two adjectives, and vice ve~a. With these three subjects omitted, mean consensus improved from .611 to .738, a figure more consistent with Experiment 1.

6A two-dimensional solution differs from a plot of the first two dimensions of a mul t id imensional solution (as typically presented by previous researchers) . The two-dimensional solution presents all information fit into two dimensions, whereas a plot of the first two dimensions of a multidimensional solution presents only the information from the first two dimensions, not that related to higher dimensions. This is especially relevant to comparisons between Fig. 4 and previous research.

T~ting Congruence of Emotional Meaning

stress = 0.005 1.5

43

1 .0 ~

.5"

0 .0 '

-.5"

-1.0"

Joy

surprise

sadness

f e a r R anger

disgust

- 1 . 5 . . . . . . . . . . . . . . . .

-1.5 -I".0 -;5 0'.o .~ 1'.o 1.s

Fig. I. Multidimensional sealing plot of basic emotion facial expression similarity data--Experiment 1.

that the relative positions of items in the MDS plot can be meaningfully interpreted.

Similarity between the emotional response ratings and visual meaning was tested by superimposing the consensual emotional response rating for each item (determined from the consensus analysis estimated answer key) onto the position of that item within the MDS plot of visual triad task responses. 7 Inspection of the plot then revealed a clear split along the dimension of pleasantness, evaluation, or hedonic tone (negative and positive emotion). The photos of joy and surprise received positive ratings while the remaining photos (fear, anger, sadness, disgust) all received negative emotional response ratings.

The similarity of shape between the two plots shown in Figs. 1 and 2 suggests a similarity of meaning-relations across the visual and verbal do-

7This procedure is similar to that used in property fitting except that properties are treated as categorical rather than dimensional. Here, property fitting supports the same interpretation.

44

1.5 ¸

St ress = 0 . 0 0 0

Alvarado

1 . 0

.s joy

o.o. surp=rise

- .S'

-1 .0 '

-1.S -1.5

fear m

sadness o

anger disgust

.1.o -?s olo .i 11o 1.s

Fig. 2. Multidimensional scaling plot of basic emotion term similarity data-- Experiment 1.

mains. Reflection of either plot about the x-axis produces almost exactly the same configuration as is shown in the remaining plot. In other words, the emotion terms bear the same relationship to each other as the facial expressions do.

A two-dimensional solution for the visual triad data collected in Ex- periment 2 is shown in Fig. 3. The plot of the verbal data for Experiment 2 is nearly identical to that for Experiment 1 and is therefore not shown (see Fig. 2 for comparisons across domains). Note that, in Experiment 2, the position of surprise varies considerably between the visual and verbal plots (compare Figs. 3 and 2). In the visual task plot (Fig. 3), surprise is closest to fear whereas surprise was closest to joy in Experiment 1 (see Fig. 1). Although the surprise expressions in both Experiments 1 and 2 obtained that label with 100% judgment ratings in studies conducted by Ekman and Friesen (1978a), the difference in their positions in Figs. 1 and 3 indicates a qualitative difference in appearance between the two surprise expressions that is not captured by the verbal label surprise. In the verbal

Testing Congruence of Emotional Meaning

Stress = 0,000

1.5

1.0

o.o surprise

-,5'

-1.0

-1.5 -1.5 -1'.0

happy

sad

fear disgust

anger

-:s 010 .~ 11o 1.s

Fig. 3. Multidimensional scaling plot of basic emotion facial expression similarity data--Experiment 2.

45

tasks and in the emotional response ratings in this study, surprise items were consistently treated as positive by subjects in both experiments.

Analysis using Ekman and Friesen's (1976, 1978b) Facial Action Cod- ing System suggests that the main difference between the surprise expressions used in Experiments 1 and 2 is the presence of ALl 12 (a slight pulling upward of the corner of the lips) found in the surprise photo used in Ex- periment 1. Otherwise, the surprise expressions both share the raised brows, widened eyes, and open mouth found in the fear expressions used in both experiments. While the upturned mouth may be a positive cue, the incon- sistent response to the surprise expression without the upturned mouth con- stitutes an ambiguity, consistent with the view that surprise is an affeetively neutral state depending upon context for its valence. Subject response to the surprise photo in Experiment 1 (with upturned mouth) suggests that additional facial movement, combined with surprise action units, may provide such a context, along with the other contextual elements typically ex- perienced under more natural viewing conditions.

46 Alvarado

Data Analysis

To confirm the patterns suggested by the MDS plots, the percentage of matches, on a subject-by-subject basis, between the verbal and visual triad responses was compared in the following manner. The items in the independent verbal and visual triad tasks were always presented in exactly the same order to the same subject. Selection of a particular facial expression (e.g., sad), and its corresponding verbal label (e.g., sadness) in the respective visual and verbal triad sets constituted a "match."

For Experiment 1, the mean percentage of matches was 68%, significantly greater than chance (33%), z = 19.86, p < .001. This justifies con- cluding a similarity of structure between the verbal and visual domains for these faces and terms. For several subjects, the match was 100%, but for several others, it was near chance. The mode for native English speakers was 14 matches (70%). That some native speakers were near chance may be due to individual differences in decoding subtle differences among the negative facial expressions. (Failure to differentiate between positive and negative expressions would appear as greater divergence during consensus analysis.) Reduction in the percentage of matches also occurred due to error, and due to the greater difficulty of some triad sets compared to others. If items are readily distinguishable but nearly equidistant in either domain (as are joy, fear, and sadness in Fig. 1), then the basis for choice among them becomes less clear and may approach chance. Even if the same pattern exists across domains, the likelihood of a match when there is little basis for preferring any of three equally different items, is at chance (33%). Shepard and Cooper (1992) found a .97 correlation between color names and color terms, among subjects with normal vision, suggesting that some error is to be expected, even when subjects are assumed to be drawing upon the same mental representation in both domains. The lack of such a strong correlation in this study provides additional evidence that subjects may not be drawing upon the same representation in both domains. How- ever, finding any subjects with 100% matches seems extremely unlikely without the presence of a naming function that links facial expressions with semantic labels.

For Experiment 2, the mean percentage of matches was 43%, still significantly greater than chance (p < .001). With nonnative English speakers eliminated from the sample, this percentage improved to 46%. These un- impressive percentages indicate a weaker link between the interpretation of meaning across the visual and verbal domains than was found in Experiment 1, suggesting a less good fit between the labels and the faces presented in Experiment 2. Most of this difference in results between Experiments 1 and 2 was due to the surprise facial expression, as noted above. In the verbal

Testing Congraeace of Emotioaal Meaaing 47

domain, surprise is unifomdy considered a positive term and is considered similar to happy whenever it appears, whereas in the visual domain, surprise tends to be considered negative and is considered similar to such expressions as fear and anger, leaving happy or sad the most different item. Of 20 triad sets, 13 were divergent between the visual and verbal domains. Of these, seven showed differential responses to the surprise stimulus item. A rough calculation assuming the seven trials with surprise photos were handled the same in both experiments yields 70% agreement (i.e., 20 - 6 = 14, 14/20 = .70) compared to 68% for Experiment 1.

It might be tempting to conclude from this that Experiment 2 repre- sents a failure of replication of the results in Experiment 1. That would be true if the same items had been used. The consensus analysis results indicate that with the same items, close if not identical results would be produced in Experiment 1 by any random sample of subjects drawn from the same population. The analysis above suggests instead that qualitative differences in the visual items was the larger source of difference between the two studies. It seems likely that the verbal label for the "surprise" expression does not convey the same meaning as the facial expression itself. Yet, 100% of subjects chose the label surprise in a forced-choice paradigm. This mismatch between the verbal, visual, and emotional response for this particular expression-term combination (surprise) illustrates the sensitivity of this methodology to the "fit" between the labels and expressions presented to subjects. A difference between photos too subtle to be measured by the forced-choice recognition studies has a large impact on results in this paradigm.

EXPERIMENT 3

If the emotion-term labels applie~ to certain facial expressions of emotion by Ekman and Friesen (1975, 1978a) are appropriate, it would be expected that similarly labeled facial expressions should evoke similar responses in a judgment task. Further, if subjects respond to facial expressions based upon the affective meaning of those expressions rather than the perceptual similarity of the faces, then it would be expected that simi- laxly labeled items need not contain the same facial movements (action units). In other words, perceptual similarity need not correspond to similarity of meaning. Experiment 3 tested these premises using the same procedures as used in Experiments 1 and 2, but with multiple exemplars of similarly labeled facial expressions of emotion. Experiment 3 also overcame the methodological objection that the use of only six stimulus items may not produce valid results. To introduce intensity as a dimension of judg-

48 Aivarado

ment, allowing a circumplex to emerge if possible, the category neutral was also included. Because multiple, empirically tested, alternative labels for the variant exemplar photos were not available, the verbal triad task was omitted.

Method

The methodology for Experiment 3 was the same as for Experiments 1 and 2, except as noted below.

Subjects

Subjects were 42 male and female undergraduate student volunteers at the University of California, Irvine. Ethnic diversity was similar to that described for Experiment 1. No comparisons between native and nonnative speakers of English were made.

Materials

Stimulus items consisted of 21 Pictures of Facial Affect listed in Table II with the judgments ratings reported by Ekman and Friesen (1978a). Three photos from each category of emotion, plus three neutral photos, were selected. Photos with the highest possible judgment ratings were selected, under the constraints of minimal duplication of individuals and inclusion of equal numbers of male and female expressors, balanced among emotion categories.

Procedure

A visual triad task and two emotional response rating tasks were presented. The emotional rating tasks were presented after the triad task, and the order was reversed for the second rating. Due to the large number of stimulus items, a balanced incomplete block design with lambda of 1 was used. Each item was paired with each other item at least once and all items appeared exactly 10 times. The third item in each triad set was randomly varied under the constraint that each item appear the same number of times. Instructions read to subjects were similar to those used in Experi- ments 1 and 2, except that, because multiple expressors appeared, subjects were asked to specifically disregard details such as sex, age, hairstyle, and any variations introduced by the slide projector bulbs. Subsequent exami-

Testing Congruence of Emotional Meaning

Table II. Experiment 3 Stimulus Items

Item Emotion ID numbe¢ % Judgments

1 Neutral JB-1-03 78 2 Disgust MO-2-18 100 3 Neutral PE-2-04 63 4 Fear J J-5-13 96 5 Sadness PF-2-12 100 6 Happiness MF-1-06 100 7 Neutral EM-2-04 69 8 Fear PE-3-21 92 9 Sadness JM-3-11 96

10 Surprise GS-I-16 100 11 Anger MF-2-07 100 12 S u r p r i s e SW-I-16 100 13 Disgust WF-3-11 97 14 Happiness JB-1-09 100 15 S a d n e s s EM-4-24 97 16 Happiness PF-1-06 100 17 Fear PF-2-30 100 18 Anger NR-2-07 79 19 Anger MO-2-11 1130 20 Disgust JB-I-16 100 21 Surprise A-1-24 97

aCorresponds to listing in P/ctures of Facial Affect (Ekman & Friesen, 1978a).

49

nation of the MDS plot reveals that they were able to do so (i.e., there was no tendency for items to be grouped by sex, hair color, age, etc.).

Results and Discussion

Consensus Analysis

Consensus analysis results are shown in "l~ble I. No negative competence scores were found. Presence of consensual response validated the MDS solution and confirmed that individual plots were likely to be similar to the group plot analyzed below.

Multidimensional Scaling

The two-dimensional MDS solution based upon the triad similarity data is shown in Fig. 4. Stress was high (.223) and improved slightly when

5O

Stress = 0.223

1.5-

Alvarado

1.0.

.5-

0.0-

-.5-

-1.o-

happiness-14 happiness-6

happiness*16 •

neutral-7

neutral-3 ii neutral-1

sadness-5

sadness-15 sadness-9 •

surpr~e-12 surprise21,

surpdse-lO

fear-4

f'e~.pa;-17

ange r- 19 .~ anger-18

disgust 0 •

disgust- 13 disgust-2 • anger-11

-1.5 -t.s -i.o -:s o:o .~ ,:o 1~

Fig. 4. Multidimensional scaling plot of multiple exemplars of basic facial expressions--Experiment 3.

a third dimension was added, but the interpretability of the plot also di- minished. Comparison of stress values with previous research is inappro- priate because different numbers of items were used, and because different MDS computer packages apply different algorithms for calculating stress. Because stress was high, relationships among items suggested by the MDS plot were also confirmed by inspection of the consensus modeling answer key produced for the triad data. No artifacts of the MDS routine were found.

If similarly labeled photos are perceived to have similar meaning, then they should cluster together in the MDS plot shown in Fig. 4. This is the case, with two exceptions: (1) one surprise photo appears equidistant from the fear and surprise groupings, and (2) one anger photo appears closer to the disgust grouping than to the other anger photos (the disgust photo is similarly closer to the anger photo than to the remaining disgust photos). FACS scoring was applied to determine whether differences in facial movement might explain these discrepancies, as described below.

Testing Congruence of Emotional Meaning Sl

Because the balanced incomplete block design used in the triad task did not present aH possible combinations of three items, an additional triad task was constructed to further test the discrepancies noted in Fig. 4. The triad sets listed in Table III were presented to 49 additional male and female subjects. Results showed high mean consensus (M = .626) with no negative competence scores. The item considered most different by subjects is denoted by an asterisk in Table III.

In all cases, fear was selected as most different when grouped with two surprise photos. No surprise photo was considered most similar to fear. However, Surprise Photo 12 was considered most different when grouped with the two other surprise photos. This supports the FACS scoring, which showed that there is a qualitative difference in Surprise Photo 12.

The disgust/anger triad sets show a discrepancy between labeling and similarity judgments in the triad data. Anger Photo 11 is considered most different in the triad set consisting of two anger photos and a disgust photo (11, 19, and 20), whereas the disgust photo should have been considered most different. This violates emotion category boundaries. From this data it is clear that the MDS plot accurately depicts subject responses and that the anomalous grouping is not due to the stress required to produce a two-dimensional MDS solution.

FA CS Analysis

Ekman and Friesen's Facial Affect Coding System, non-baseline-cor- rected, was used to describe qualitative differences in the stimuli, and to compare the amount of similarity between multiple exemplars depicting the

Table Ill. Experiment 3--Additional Test Triad Sets

A B C

18--Anger 19--Anger 20--Disgust* 19--Anger ll--Anger* 20--Disgust 20--Disgust 13--Disgust ll--Anger* 20--Disgust 11--Anger* 19--Anger 18--Anger 19--Anger ll--Anger*

21 --Surprise I2--Surprise 4--Fear* 4--Fear* 10--Surprise 12--Surprise

10--Surprise 12--Surprise* 21--Surprise 21--Surprise 12--Surprise 8--Fear*

*Indicates photo considered most different; numbers correspond to stimulus items shown in Fig. 4. Mean consensus = .626 (N = 46).

52 Alvarado

same emotion. Two coders evaluated each item, with intercoder reliability of .76 (at the level recommended by Ekman and Friesen). Resulting FACS action units (AUs) for each photo were superimposed onto the item positions in Fig. 4 to allow visual inspection of the correspondence between patterns of codes and grouping of items, as shown in Fig. 5.

In general, each category of emotion shows the combination of codes required to specify that emotion (Ekman & Friesen, 1975, 1978b), together with varying combinations of other codes less important to defining a particular emotion. For example, the happiness photos all show different intensities of AU 6 + 12 (Duchenne smiles showing upturned mouth, raised cheeks, and crow's feet at the eyes). The sadness photos all show AU 1 + 4 (oblique eyebrows), with various combinations of other codes. The downturned mouth (AU 15) intuitively associated with sadness is irrelevant to this classification; only one photo shows it, while Sadness Photo 5 even shows an upturned mouth (AU 12). Anger Photos 11 and 19 both display AU 4 + 7 (lowered brow with squint) at different intensities, and all three

Stress = 0.223

1.5-

1.0-

.5" no AUs

R12A

no AUs 0.0- •

-.5-

-1.0.

-1.5 ......... -1.5 -1'.0

6C+12D+26

6D+12C~26

6C+12C

1E+2E+5B+26 1B+2B+5C÷26

1B+2D÷12A+26

1C+2C+4C+5D+10A+20B÷26

~C+2C+5D+2~+26

1C+2B+4A+5E+20B+26

1A+4B+12A

1C+4C+25 18+4B+lSA+41C =

4B+SC+7B+15A+23 • 4B+20B+23+25+29

4B+7C+10B •

10C+17B+25 • 4C+5C+7D+10A+23+26

9D+R101~+17B+25 •

-.'5 01o .k tlo 1.5

Fig. 5. Facial Action Coding System codes superimposed on multiple exemplar item positions (see Fig. 4).

Testing Congruence of F_~motional Meaning 53

anger photos display AU 23 (lips tightened). Disgust photos all display AU 10 (upper lip raised). Surprise photos all display AU 1 + 2 + 5 + 26 (eyebrows raised, eyelids wide, mouth dropped open). Fear photos all display AU 1 + 2 + 5 + 20 (same as surprise but with lips pulled back lat- erally).

With FACS coding, the anomalous positions of certain items can be explained. In Surprise Photo 10 (equidistant between fear and surprise), the widening of eyes and the shape of the expressor's mouth is similar to the movement in the fear expressions. Surprise Photo 12 contains no AUs suggesting happiness, but has the lowest-intensity AU 5 (eye widening), dis- tancing it from fear. The Disgust Photo 20 shares AU 4 + 7, elements of anger, with the anger photos to which it is closest. Similarly, Anger Photo 11 shares a slight AU 10 with the disgust photos, but its open mouth (AU 26) distances it from Photos 19 and 20. Thus, Anger Photo 11 is selected as most different both due to similarities between Anger Photo 19 and Disgust Photo 20, and due to its own qualitatively distinct mouth movement.

Inspecting the plot as a whole, all photos to the extreme right have open mouths (AU 26). Those in the lower half of the plot involve lowered or oblique brows (either AU 4, AU 1 + 4, or AU 9). In a less clear-cut manner, all photos in the upper half show no brow movement or lifted brows (AU 1 + 2). The raising of cheeks and tightening around the eyes in AU 6 are also related to the eyes (e.g., when we speak of "laughing eyes"). However, these tenuous regularities are insufficient to fully characterize the positions of items in the plot. The presence of AU 10 is needed to distinguish disgust from anger, and the presence of ALl 20 may be needed to distinguish fear from surprise. Further, it seems likely that AU 12 (upturned lips) and AU 15 (downturned lips) alone are insufficient to fully characterize happiness and sadness, respectively, except perhaps at high intensity.

Distance Analysis

The results presented above suggest that not all AUs contribute equally to the perception of an expression and its description with an emotion term label. Components appear to be important only in certain configurations or patterns. To illustrate this, consider the anger/disgust grouping in the lower right quadrant of Fig. 5. Calculation of the percentage agreement between items is a measure of the similarity between items in terms of constituent muscle movements. If such movements contribute equally to perceived similarity, then items closest to each other in the plot

54 Alvarado

should show greater percentages of agreement than items more distant. This is not the case.

Figure 6 shows the computed percentages of agreement and the MDS distances for the anger and disgust photos. High percentages should link close items, while low percentages should link distant items. Inspection of the plot reveals that this is only inconsistently true. A Spearman rank order correlation of ordering of pairs by their distances with the ordering of the same pairs by their percentage agreement was nonsignificant and tended in the negative direction (r = -0.12).

While it is possible that descriptions based upon AUs (muscle move- merits) do not include the right components, this seems unlikely given that FACS was designed to record all possible facial movement independent of any theory. It seems more likely that meaning is derived from larger patterns that encompass less than the whole face but more than one group of muscles (AU). Further, these larger patterns do not appear to be dimensional.

o.o

-.5

-1.0

anger

d i s g u s t ~

.20 disgust anger disgust

anger

-.5 olo .~ 11o

Fig. 6. Facial Action Coding System code computed similarity in relation to distances between selected items (see Fig. 5).

1.5


Certain definitional patterns of movement appear to be systematically related to verbal labels. Note that Anger Photo 19 and Disgust Photo 20 are quite close in the plot, and showed 50% agreement in terms of facial movements, indicating that subjects considered them similar, yet 100% of Ekman & Friesen's subjects agreed that they be labeled anger and disgust, respectively. AU 10 thus seems definitional to the disgust label because Photo 20 has it and Photo 19 does not. Anger Photo 11 and Disgust Photo 20 share more muscle movements (.66), yet there was also 100% agreement on their respective labels as well, and subjects did not consider them to be similar. Taken together with the findings above, this suggests categorical perception of emotion triggered by key patterns of AUs.

Critics might argue that the meaning found in this experiment is simply perceptual similarity, rather than emotional meaning. If that were so, perceived similarity should correspond more directly to the observable physical similarity among photos, yet the determinants of judged similarity appear to be more complex, and, for the anger/disgust stimuli, are even uncorre- lated with the amount of physical similarity. Thus the decisions made by subjects cannot be solely perceptual. Item labels are provided in Table II so that readers can confirm this themselves with reference to the photos used. Despite these anomalies, the congruence between emotional meaning conveyed through facial expressions and their associated emotion term labels is obvious from the clear groupings of items by emotion category shown in Fig. 4.

GENERAL DISCUSSION

By demonstrating a second-order isomorphism between selected facial expressions of emotion and emotion terms, Experiments 1 and 2 showed that (1) subjects derived meaning from facial expressions of emotion that was similar to the meaning conveyed by certain emotion terms; (2) that the meaning derived from facial expressions of emotion was shared among subjects rather than idiosyncratic; and (3) that visual meaning appeared to be less affected by differences in English-speaking ability, and associated cultural diversity, than was the meaning derived from emotion terms, at least for this particular group of heterogeneous subjects. However, Osgood (1966) noted that high reliabilities are more likely with fewer items. Ex- periment 3 showed that interpretation of meaning in facial expression re- mained consensual and was consistent with emotion-term labeling, even when 21 items were presented. Experiment 3 found that similarly labeled facial expressions were considered most similar to each other when viewed in the absence of their verbal labels, further supporting the contention that

Al~rado

the facial expressions convey the same meaning as the labels. Subjects grouped items by that meaning without being exposed to the emotion-term labels at any time during the experiment, and without the word emotion being used in the instructions.

In Experiment 3, analysis using FACS coding suggests that the perceived similarity of meaning among similarly labeled expressions bears a complex relationship to facial movement in which not all types of movement have equal importance. Perceived meaning may be related to key configurations of muscle movements that may convey specific emotional meaning. Perceived similarity of meaning and number of similar facial movements appeared to be unrelated, suggesting that some movements are necessary to the specification of emotion, some necessary but not sufficient, and some irrelevant to the perception of emotion. Further, these results suggest that categorical perception may exist, in the sense that interpretation of meaning bears no linear relationship to the amount of facial movement along any continuous dimension.

The stimuli presented here were those previously used by Ekman and Friesen and others in a variety of forced-choice experiments investigating the universality of perception of emotion in facial expressions. These findings suggest that their match between English terms and facial expressions has been appropriate, but not optimal. The match appears better for the stimuli presented in Experiment 1 than for those used in Experiment 2, primarily due to differences in the appearance of the stimuli used across experiments. Of course, no cross-cultural work was performed here, and results for English- versus non-English-speaking samples are presented because they bear upon semantic processing, not as a test of universality, though I believe they are interesting in the context of that debate.

The analysis of similarity with respect to FACS codes presented in the context of Experiment 3 has important implications for theories suggesting that expressions of emotion are made up of combinations of constituent components (cf. Ortony & Turner, 1990). A mix-and-match theory of blend- ing of facial movements into an endless variety of facial expressions of emotion is only possible if all components carry somewhat equal weight, or contribute their share to the interpretation of that expression. The results of Experiment 3 show that certain movements contribute very little to meaning in one configuration of movements, but quite a lot in a different configuration (for example, in Fig. 5, amount of eye widening may help distinguish surprise from fear, but adds little if anything to the interpretation of anger).

Shepard and Cooper (1992) suggested that the same representation of color is accessed for similarity judgments involving MunseU chips and color terms and obtained a 97% correlation between tasks for subjects with


normal vision. In the work presented here, congruence of meaning was found, but not perfect congruence. How much congruence across domains is needed in order to assert that the meaning conveyed by a face is the same as the meaning conveyed by an emotion term? Why is the congruence between domains imperfect, the percentage of matches less than 100%? Technical reasons are suggested in the context of Experiment 1. As noted above, the .97 correlation obtained by Shepard and Cooper cannot be expected in this context unless we also assume that the underlying representation was the same for both domains. It seems more likely that, while good enough to achieve some congruence, the words were not the best possible labels for the facial expressions used in this study. Ekman (1993) suggested that facial expressions are best regarded as representing families of emotions labeled by more generic emotion terms such as anger or fear. To the extent that such words are general, they may lessen the likelihood of a match between visual and verbal stimuli in this paradigm. Experiment 2 demonstrated the sensitivity of this measure to slight mismatches. The quite general emotion terms used by Ekman and Friesen to classify both families of terms and of expressions may not capture the more complex meaning conveyed by a particular facial expression as well as some more specific term might do (Ekman, 1993).

Whether a closer fit between each facial expression and its assigned label can be attained is an empirical question. Alvarado and Jameson (1996) have suggested a method for identifying and testing the best verbal labels for specific facial expressions of emotion. This method uses paired comparisons to determine an interval scale that ranks emotion terms against exemplar photos (or vice versa), applying Batchelder and Bershad's numerical rating system. This method permits large numbers of items to be tested using small groups of subjects (typically 15 to 20), making it prac- tical to use. Fine-tuning of the match between verbal and visual items should improve the percentage of matches, as it has done in similar work in a different domain (see Alvarado, 1993). However, Osgood's (1966) idea that facial expressions convey primarily affective meaning whereas emotion terms convey primarily denotative meaning implies that a perfect congruence between facial expressions and emotion terms cannot be achieved. As Ekman (1993) noted, emotion terms may reference behavior, context, or metaphorical aspects of meaning that are beyond the meaning conveyed by a facial expression.

Because lower consensus was achieved in all three experiments for facial expression tasks than for verbal tasks, it appears that visual meaning is less constrained than is verbal meaning. Even if subjects spontaneously generate verbal labels when performing the visual task, similar visual meanings should result in verbal labels with similar meanings. Regardless of Eng-

58 Al~rado

lish-language ability, there was less agreement about the relative meaning of the facial expressions than about the meaning of the emotion terms. It may be that the facial expressions are more complex, or it may be that there are greater individual differences in decoding emotional meaning (Ekman, 1993).

Some researchers have suggested that nothing new can be learned from the multidimensional scaling of facial expressions and emotion terms) With increased sophistication in the use of well-described, systematically varied, and controlled stimuli, I believe the potential of this method is far from exhausted. Use of an objective descriptive system like FACS is essen- tial to this effort. A triad judgment task forces subjects to emphasize different aspects of stimuli over others in the differing contexts presented during the course of the task (Nosofsky, 1986). This permits the salience of certain facial movements to the perception of emotion to emerge in the MDS plot. That certain patterns of movement distinguish certain emotions seems obvious from the plot shown in Fig. 4, yet this is new information because previous researchers did not apply such detailed descriptive methods to their stimuli. By and large, the studies described here revealed the patterns of movement linked to specific emotions by Ekman and Friesen (1975), who used their descriptive system to specifically characterize the emotions. However, the work presented here is a preliminary step in the more detailed description of how facial movement relates to the perception of emotion.

Etcoff and Magee (1992) suggested that the perception of emotion is categorical. The results presented here suggest that perception of emotion may be categorical with respect to certain key patterns of facial movement, rather than the overall percentage of an expression represented in a face, as tested by Etcoff and Magee. Note that Etcoff and Magee found the same anomalous results with respect to the surprise/fear expressions and the anger/disgust expressions as were found here. With the methods presented here, exploring this issue empirically becomes more straightforward, and it seems likely that uncovering the reasons for these anomalies will yield greater understanding of the meaning that observers derive from facial movement.

Those who have never questioned whether facial expressions convey information about the emotional state of the expressor may find this research self-evident. However, recent theorists have raised the possibility that emotion itself is a culture-dependent construct and that facial expressions instead serve social purposes unrelated to the experience of emotion (cf. Fridlund, 1994). In this study, stimulus sampling created a demand to

Sl thank the anonymous reviewer of a previous draft for this observation.


interpret the facial expressions in terms of emotional meaning, but only to the extent that subjects were already able to perceive such meaning in the faces shown. The larger question of whether facial expressions serve only social purposes and are only incidentally labeled using emotion terms is untestable within the context of this culture. This study does demonstrate, however, that English-speaking subjects, including those born in other areas of the world, do interpret facial expressions of emotion similarly to emotion terms, even when given the opportunity to do otherwise.

This research has several methodological consequences. First, the quality of labeling need not be a confound in studies of the interpretation of meaning in facial expressions of emotion. This method presents a forced- choice format in which mismatches between items will be obvious. Second, using this methodology, the meaning of verbal items is tested independently of visual items, and thus presentation of verbal labels does not bias subject responses in the visual domain, circumventing another criticism of previous research. Note that in this study no order effects were found across presentation conditions (visual items first vs. verbal items first). Third, because this method provides a quantitative measure of the relationship between visual and verbal items, it may be used to study the extent of hypocognition or hypercognition within specific languages and cultures (Levy, 1984). Fourth, by varying the constituent facial movements in exemplar photos, this method can be used for the controlled study of the contribution of those movements to the perception of specific emotions. Fifth, use of this paradigm need not be limited to facial expressions and emotion terms, but can be extended to any type of stimuli that can be readily presented in triads (or pairs) to yield similarity data.

APPENDIX

The following description of the consensus model is adapted from Bor- gatti (1993). The model uses the following notation:

L 1 /L

1 - 1 /L

m O

the probability that a subject knows the right answer to a given question the probability that the subject doesn't know the answer the number of response options to a given question the probability that the subject will guess the correct answer the probability of guessing the incorrect answer the probability that two subjects i and j give the same answer to a given question

60 Aivarado

The parameter di is the subject's competence rating. It is readily cal- culated if the answer key is known because it is the percentage of correct questions answered minus a correction for guessing. If the answer key is not known, the parameters are estimated using the following equations:

m# - (dflj - 1)/L (1) d ~ = (Lm# + 1)/L = m*# (2)

where m*# is a rescaling to correct for chance guessing of the observed values m#. Equation (2) is solved for d via minimum residual factor analysis to yield a least-squares estimate of the d parameter (competence rating) for each subject. Bayes' theorem is then used to estimate the answer key confidence levels, given the estimated values of d.

REFERENCES

Abelson, R., & Sermat, V. (1962). Multidimensional scaling of facial expressions./ournal of Experimental Psychology, 63, 546-554.

Alvarado, N. (1993). The labeling of emotion. Doctoral dissertation, University of California, Irvine. (University Microfilms No. 9323902).

Alvarado, N., & Jameson, IC (1996). New findings on the contempt expression. Cognition and Emotion, I0, 379-407.

Batchelder, W. & Romney, A. (1988). Test theory without an answer key. Psychometrika, 53, 71-92.

Batchelder, W., & Romney, A. (1989). New results in test theory without an answer key. In E. Roskam (Ed.), Mathemat/calpsycho/ogy/n progress (pp. 229-248). Heidelberg: Springer Verlag.

Borgatti, S. (1990). Provisional documentation; Anthropac 3.0. Unpublished manuscript. Borgatti, S. (1993). Anthropac 4.0. Columbia, SC: Analytic Technologies Inc. EL, nan, P. (1993). Facial expression and emotion. American Psychoiogist, 48, 384-392. Ekman, P. (1994). Strong evidence for universals in facial expressions: A reply to Russell's

mistaken critique. Psychological Bulletin, 115, 268-287. Ekman, P., & Friesen, W. (1975). Unmasking the face. Palo Alto, CA: Consulting Psychologists

Press. Ekman, P., & Friesen, W. (1976). Measuring facial movement. Environmental Psychology and

Nonverbal Behavior, 1, 5675. Ekman, P., & Friesen, W. (1978a). Pictures of facial affect. Palo Alto, CA: Consulting

Psychologists Press. Ekman, P., & Friesen, W. (1978b). Facial Action Coding System manual Palo Alto, CA:

Consulting Psychologists Press. Etcoff, N., & Magee, J. (1992). Categorical perception of facial expressions. Cogn/~n, 44,

22%240. Fridlund, A. (1994). Human facial expression: An evo/utionary v/ew. San Diego, CA: Academic

Press. Levy, R. (1984). Emotion, knowing, and culture. In R. Shweder & R. LeVine (Eds.), Cu/ture

theory: Essays on min~ self, and emotion (pp. 214-237). New York: Cambridge Univemity Press.

Nosofsky, R. (1986). Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: Genera~ 115, 39-57.

Nummenmaa, T. (1964). The language of the face. Jyvaskyla studies in education, psychology, and social research (Vol. 9). Jyvaskyla, Finland: Jyvaskylan Yliopistoyhdistys.

Ortony, A., & Turner, T. (1990). What's basic about basic emotions? Psychological Review, 9Z 315-331.


Osgood, C. (1966). Dimensionality of the semantic space for communication via facial expressions. Scandinavian Journal of Psychology, 7, 1-30.

Roberts, J., & Wedell, D. (1994). Context effects on similarity judgments of multidimensional stimuli: Inferring the structure of the emotion space. Journal of Experimental Social Psychology, 30, 1-38.

Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psycholo~, 39, 1161-1178.

Russell, J. A. (1994). Is there universal recognition of emotion from facial expression? A review of cross-cultural studies. Psychological Bulletin, 115, 102-141.

Russell, J., Lewieka, M., & Niit, T. (1989). A cross-cultural study of a circumplex model of affect. Journal of Personali~ and Social Psychology, 57, 848-856.

Schlosl~rg, H. (1954). Three dimensions of emotion. Psychological Review, 61, 81-88. Shepard, R., & Chipman, S. (1970). Second-order isomorphism of internal representations:

Shapes of states. Cognitive Psychology, 1, 1-17. Shepard, R., & Cooper, L. (1992). Representation of colors in the blind, color-blind, and

normally sighted. Psychological Science, 3, 97-104. Turner, T., & Ortony, A. (1992). Basic emotions: Can conflicting criteria converge?

Psychological Review, 99, 566-571.

congruence of meaning between facial expressions of ...nalvarado/pdfs/alvarado1996motemotion.pdf ·...

Documents