single and multichannel signal composition: facial ... · single and multichannel signal...

Single and Multichannel Signal Composition: Facial Expressions and Vocalizations of RhesusMacaques (Macaca mulatta)Author(s): Sarah R. PartanReviewed work(s):Source: Behaviour, Vol. 139, No. 8 (Aug., 2002), pp. 993-1027Published by: BRILLStable URL: http://www.jstor.org/stable/4535968 .Accessed: 28/07/2012 15:18

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

BRILL is collaborating with JSTOR to digitize, preserve and extend access to Behaviour.

http://www.jstor.org

http://www.jstor.org/action/showPublisher?publisherCode=bap

http://www.jstor.org/stable/4535968?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp

SINGLE AND MULTICHANNEL SIGNAL COMPOSITION: FACIAL EXPRESSIONS AND VOCALIZATIONS OF RHESUS

MACAQUES (MACACA MULATTA)

by

SARAH R. PARTAN1 2)

(Animal Behavior Group, University of California, Davis)

(Acc. 7-IV-2002)

Summary

A method for simultaneously examining visual and vocal components of expressive behavior is described, compiled from video recordings of social behavior of a free-ranging group of rhesus macaques on Cayo Santiago, Puerto Rico. I developed a catalog of expressive movements, and chronicled detailed information on visual and vocal components of 1215 individual behaviors. Two thirds of the events recorded were silent, supporting the idea

1) Department of Psychology, University of South Florida St. Petersburg, 1400 South Seventh Avenue, St. Petersburg, FL 33701, USA. e-mail: [email protected] 2) This work is based on my dissertation research in Peter Marler's laboratory. I would like to give my heartfelt thanks to Dr. Marler for his encouragement, support, and advice, as well as for his comments on several drafts of the manuscript. I am grateful to Marc Hauser for encouraging me to work on rhesus monkeys, and to William Mason for helping me to understand rhesus behavior during pilot observations at the California Regional Primate Research Center. This project was developed during discussions with Drs. Marler, Mason, Hauser, Christopher Evans and Joseph Macedonia. I would like to thank Charles Snowdon, Correigh Greene, Jill Soha, Katya Partan, and two anonymous reviewers for comments on the current manuscript; Dr. Mason, John Endler, and Arlene Alvarado for comments on prior versions; and J.A.R.A.M. van Hooff for a suggestion on terminology. Neal Willits and Tim Allis provided key statistical advice; Renee Allen and Marc Fourrier gave indispensable assistance in the field, and Jeannine Logan, Rebecca Wylie, and Virginia Price in the laboratory; Bill Patrick and Todd Hughes helped produce Figure 4. Finally, I would like to thank John Berard, Matt Kessler, the University of Puerto Rico, and the Caribbean Primate Research Center for providing access to the field site and the genealogical data, and Edgar Davila for introducing me to the monkeys. This study was funded by grants from the National Science Foundation, the L.S.B. Leakey Foundation, the Animal Behavior Society, and Sigma xi.

( Koninklijke Brill NV, Leiden, 2002 Behaviour 139, 993-1027 Also available online -

994 PARTAN

that visual behaviors are primary for short distance communication in these macaques. Clusters of expressive components detected by Principal Component Analysis and Multiple Correspondence Analyses corresponded to threatening, submissive, and affiliative behaviors described previously, providing quantitative support both for these previous descriptions and for the suggestion that these three poles of behavior are important in daily social interaction. Silent expressions involved a greater variety of mouth positions than did vocalizations, which were produced with stereotyped mouth shapes. Other components of the face, not involved with articulation, were nonetheless associated with particular vocalizations: specific associations were found among barks, ears retracted, and head lowered on the one hand, and pant-threats, ears forward, and head raised on the other. Screams and squeaks were highly stereotyped, combined with prototypical grimace mouth positions, crouching and retreating. Girney vocalizations were accompanied by lipsmacking. Grunts were unaccompanied by other expressive components, evoking the suggestion that they may be predominantly neutral in valence.

Introduction

Although communication involves the use of multiple sensory channels, most researchers focus on one sensory channel at a time, in isolation from the full repertoire of the animal. The combination of channels, however, can have important ramifications for signal meaning and efficacy (Marler, 1965; Partan & Marler, 1999; Rowe, 1999). Human facial expression and visual articulatory movements, for example, play a role in speech perception (McGurk & MacDonald, 1976; Massaro, 1998). In birds, odor cues from prey can interact with visual stimuli of particular colors to produce food aversions that do not occur without the odor (Rowe & Guilford, 1996), and odor plays an important role in combination with other cues during sexual behavior in baboons (Goldfoot, 1982). Signal components in multichannel displays can be redundant (e.g. Conner, 1987), or each component may play an independent role (e.g. Fusani et al., 1997). It is difficult, however, to determine the functional role of each component of multicomponent signals, as Green (1975, p. 87) mentioned for Japanese macaques: "...their vocal

behavior is inextricably tied to simultaneous olfactory, tactile, and visual signals, hence consideration solely of evoked responses cannot disentangle the roles of the concurrent signals available by different sensory moda- lities ..

Before parsing the role of each component in communication, one needs to determine which particular components are combined simultaneously into multimodal signals during signal production. Although researchers studying

SIGNAL COMPOSITION 995

the receptive side of communication have developed sophisticated methods for measuring the perception of multichannel signals (e.g. visual influences on human speech perception, Massaro, 1998), there are to date few established methods for quantifying natural multichannel signal production.

This study describes how natural visual signals of facial expression and body posture are associated with vocalizations of rhesus macaques (Macaca mulatta), bridging the separate analyses of visual and vocal signals of this species (e.g. primarily visual analyses by Hinde & Rowell, 1962; van Hooff, 1962; Maxim, 1982, 1985; Zeller, 1986, 1996; Maestripieri & Wallen, 1997; and primarily vocal analyses by Rowell, 1962; Gouzoules et al., 1984; Hauser, 1991, 1996; Hauser & Marler, 1993). Although some authors have described visual and vocal signals together (e.g. Altmann, 1962; Rowell & Hinde, 1962; Lindberg, 1971; Mason, 1985; Kalin et al., 1992), none have reported the frequencies with which particular visual components are associated with particular vocal components. This is an important first step towards an understanding of the multimodal nature of communication. A similar approach has been taken by Adams & Schoel (1982) in a study of stumptail macaque (Macaca arctoides) communication: vocalizations were analyzed separately but in parallel with the simultaneous facial and postural components of behavior. My work also builds on the work of Hauser et al. (1993), who suggested that each class of rhesus vocalization is accompanied by an unique articulatory gesture. Here I not only examine mouth position, but also include facial expressions and body postures uninvolved with actual phonation that nonetheless predictably accompany each vocalization.

One goal is to explore the variability in facial expressive components used during communication. I constructed a catalog of expressive movements of the face and head, grouping behaviors into 'morphological' categories (classified by body part: eye, ear, head, etc.; cf. Reynolds, 1976). Within each category, I defined logical, mutually exclusive states for each body part (e.g. the ear category includes earsforward, retracted, orflapped back andforth). This categorization scheme is more detailed than previous rhesus catalogs with regard to facial movements (see Zeller, 1986, 1996, for a detailed look at the face of other Macaca species) but may not be as detailed with regard to gross body movements. Reynolds (1976) thoroughly documented and cross-referenced terms used by early rhesus monkey observers, and I attempt here to use many of the terms of my predecessors in the hope of achieving marginal consistency and common understanding. My terms for

996 PARTAN

the vocalizations follow most closely those used by Rowell & Hinde (1962) and Hauser et al. (1993).

The behavior of the animals was recorded action by action in sequence. An 'action' was identified as a discrete unit of behavior, which may include multiple simultaneous components. This designation was based on my impression of a behavioral 'unit.' Behavior, although it occurs in continuous streams, can be described as being made up of sequences of more or less discrete actions. The identification of these action units can, however, be difficult. I separated the behavioral stream into discrete actions based partially on physical changes in the body (e.g. the head moving from raised to lowered), and partially on timing considerations (if two components of behavior occurred simultaneously, such as head lowered and bark, I considered them to be components of the same action, whereas if there was a time lag between them, they were considered to be two separate actions). Green (1975) also emphasized dynamic transitions in patterns as an aid to classification.

Drummond (1981) provides an extensive and interesting discussion of how we categorize and describe behavior. He notes that the stream of behavior can be segmented at many different levels, each appropriate to a different type of analysis. A common definition of a 'unit' is as a reliably recurring phenomenon (Drummond, 1981, p. 13-14). Ethologists may have to use intuition to determine where the breaks in behavior occur, but our intuition is based on tractable data such as regularity in patterns or bouts of behavior (Drummond, 1981; see also Marler & Hamilton, 1966, Chapter 20). The ultimate test of our intuition is to ask the animals themselves how they categorize their own signals (see Marler, 1982).

Describing real, complex behavior, involving multiple simultaneous signal components, is a challenge. To simplify, I included only the visual and acoustic channels. In addition, I typically analyzed only two classes of behavior simultaneously: vocalizations, along with one class of visual behaviors (e.g. eye gaze, or mouth position). My rationale was to document the bimodal visual and vocal behaviors of these animals. Detailed quantitative analyses of the compositions of facial and vocal expressions are presented in an effort to describe how visual and auditory signal components are combined in this population of rhesus monkeys. For further suggestions of how to analyze complex multicomponent behaviors, see van Hooff (1982), Brad- bury & Vehrencamp (1998), and Deputte (2000).


TABLE 1. Demographics of main study animals

Study Subjects

Group Y Group V

Adults (6 yrs +) Females 19 13 Males1) 12 8

Subadults (4-5 yrs) Females 8 4 Males 8 3

Juveniles (1-3 yrs) Females 15 6 Males 14 6

Infants (<1 yr) Females 11 6 Males 9 3

Total 96 49

1) The number of adult males was approximate since males occasionally transferred among groups during the study.

Methods

Subjects and study site

The subjects were free-ranging male and female rhesus macaques, Macaca mulatta, on Cayo Santiago, a 15.2-hectare island off the southeast coast of Puerto Rico (Rawlins & Kessler, 1986). The monkeys ranged freely and formed social groups, were provisioned with food, and had no natural predators. All of the animals were trapped as yearlings and given tattoos and ear notches for identification. At the time of the study, there were six social groups on the island, ranging in size from approximately 49 to 354 animals per group. The study was conducted on two groups: group 'Y' and group 'V' (see Table 1 for age demographics of the groups). Twenty-two adult females and 13 adult males were the primary focal subjects. Occasional samples were taken from animals in neighboring groups.

Videotape recording methods

All data were collected on audio and videotape for subsequent analysis. Adult animals were filmed during 30-minute focal-animal follows. The primary focal animals had at least four samples each. Two simultaneous video recordings of the animals were taken: one camera filmed a close-up shot of the facial expressions of the focal animal, and the other took a wide-angle view that also encompassed the surrounding animals. I collected 270 hours of video tape footage on VHS & SVHS tape, summing from both cameras, along with supplementary audio cassette tape recordings, during March and September-December of 1994. Close-up videos used a Panasonic AG460 movie camera and wide-angle videos used a Panasonic AG455. Both cameras were mounted on Bogen 3179 tripods. A Sennheiser MKH 816 shotgun microphone with windcover was strapped to the close-up video camera and fed into the external audio input for better audio recordings on the videotape. I filmed only when the focal animal was interacting with other monkeys at relatively close range, so signals

998 PARTAN

propagated by solitary monkeys, such as branch-shaking and food-calling (coos), were not common. Signals concerned with mating were infrequent because I collected data primarily during the nonbreeding season. (Published records, such as Rawlins & Kessler, 1985, indicate that the Cayo Santiago breeding season was from approximately July to November in the 1970's to early 80's. However, the season has shifted forward slowly over the years, currently beginning in May and mostly over by September [M. Gerald, pers. comm., December 2001].)

Data collection from videotape

Of the 270 hours recorded on both cameras, I watched 230 hours of tape (approximately 130 hours of real time). In many instances, the two videos had to be watched one at a time because they displayed different views of the same scene, and included different players. While watching the tapes, I sampled for social interactions using the 'behavior-sampling' method (Martin & Bateson, 1993, p. 87). Specifically, I sampled for any interaction involving vocal or visual communication between two or more individuals. This broad sampling rule included all social interactions, with the exception of grooming interactions already underway at the time the camera started to film, since they usually did not include vocal or visual signals. If two animals being filmed began to groom during the sample period, however, this behavior was included.

Once an interaction was located and data collection began, data were collected continu- ously until either (a) the individuals ceased interacting, or (b) one or both ran off screen and out of view of both cameras. Since I was recording in the field, with trees, bushes, and in- tervening topography, many interactions in the database ended prematurely (i.e. the animals went out of sight before they were done interacting). However, this is not problematic for the purposes of this study, since I am interested here in the structure of visual and auditory signals, rather than the outcomes of the interactions.

I described 402 social interactions (both agonistic and affiliative), watching the interactions in detail, often proceeding frame by frame to identify the components used in the signals. The individual actions that each animal performed (which often included multiple simultaneous components, as defined in the introduction) were logged sequentially, totaling to 1215 actions. The median number of actions per interaction was 2 (the range was 1-26; the mean was 3). This means that most interactions were short, with one animal emitting a display and the other a response. Focal animals were all adults, but younger animals were included when they interacted with focal subjects. Although the primary focal animals each had 4-10 focal follows, the total number of actions performed by each focal animal ranged from 5 to 65 (data on the number of times each individual contributed available on request to the author).

The database contained one entry for each action, with the following fields: the time of the event (video frame); the identities of the actor and recipient(s) involved, including their sex and age; and a set of eight components of the signaler's behavior. These included: vocalization type, mouth position, eye gaze, ear position, head position, tail position, body posture, and overall movement. If a particular body part was out of view, as could happen if the animal's face was on the camera screen but its tail was not, I left blank the corresponding cell in the database. I did not include observations with blank cells in my analyses.

For each of the eight components of the signal, I compiled a list of possible expressive positions or states (Table 2). This list provides a detailed repertoire of rhesus monkey communicative behavior, focused largely on movements of the face and head. I avoided using


TABLE 2. Repertoire of communicative behaviors of rhesus macaques: expressive components of the face, voice and body

Vocal behavior: No vocalization silence Bark loud, voiced, harsh (broadband) sound Pant-threat softer, more breathy harsh sound usually emitted in a rapid

bout of three units Scream loud sound, can be harsh or tonal (includes noisy, arched and

tonal screams) Squeak short punctuated harsh (pulsed) scream Bark/Scream mixed call that starts with bark and ends with scream Grunt quiet, harsh sound usually not repeated Girney frequency modulated, quiet sound, may be narrow- or

broadband Coo tonal (narrow-band) sound Gecker staccato harsh sounds, usually from infants Copulation call short harsh scream-like calls given by some males during

copulation Other vocalization vocalization which cannot be put in any of the above classes

Mouth position: Neutral mouth is closed, lips closed, loose Open mouth lower jaw dropped so lips form 'o' shape, upper teeth

covered Grimace lips retracted horizontally to expose teeth, jaws can be

together or apart Tense mouth mouth is closed, lip corners drawn back to form straight line Lipsmack (LS) lips moved repeatedly together and apart; may be audible LS with tongue same as above but with tongue rhythmically moving in and protrusion out of mouth Chin-up1) mouth is closed, lips together and often pursed, chin is

angled up Chin-up & LS same as above, but with lips making small, rapid movements Teeth-chatter mouth rapidly opens & closes; teeth exposed; may be audible Tooth chomp jaw opens and closes but lips remain closed; may be audible Puckered lips lips drawn forward together, cheeks furrowed Yawn mouth opens widely in stereotyped gaping movement Bite mouth opened to bite other monkey Mouth matches mouth opens only enough to emit a sound, and only for the vocalization duration of the sound, then closes

Eye gaze2): Neutral eyes relaxed, may be half shut, not looking in any particular

direction

1000 PARTAN

TABLE 2. (Continued)

Stare direct, prolonged, unwavering look at specific individual Look at look in direction of specific individual Look away look in direction obviously other than that of specific

individual Look between look rapidly between two or more other individuals Look around look rapidly in several directions, often at distant

individual(s)/group(s)

Ear position: Neutral ears relaxed, in medium position (about 45 degrees from

head) Ears forward ears pointing perpendicularly out from

head Ears back ears retracted tightly to scalp Ear flap ears move quickly backwards and forwards more than once

Eyebrow position: Neutral eyebrows relaxed Eyebrows raised eyebrows raised and remain up Eyebrows lowered eyebrows lowered and remain down Eyebrow flash eyebrows move quickly up and down

Head position3): Neutral head relaxed Head raised head held up in high position Head lowered head held in low position, neck usually angled forward Head jerk head moved quickly and abruptly up and down Head bob head moved slowly and smoothly up and down, often

repeated

Tail position: Neutral tail held in relaxed, low position Tail vertical tail held up perpendicular to body, straight from base to tip Tail up looped tail held straight up from base, but tip is curled in tight loop Tail up crooked tail held straight up from base, but tip is bent over Tail 45 degrees tail held up at 45 degrees from base, tip often slightly bent Tail parallel tail held straight out parallel to body Tail wave tail held low, waving

Body positions & movements: Neutral posture relaxed Crouch posture with belly low, arms and legs bent Lunge sudden and quick movement of upper or entire body towards

recipient Mount mount other monkey in typical posture Lean away lean whole body away from another monkey Slap ground quick hit to ground, usually one hand only



Dip-turn stereotyped movement of approaching another individual very closely, then suddenly breaking approach by bending arms, turning quickly around and moving away (head usually swings around last, so that actor is looking at recipient until last moment)

Branch shake bounce up and down on branch or other object, holding on with feet so that the substrate moves as well

Freeze body suddenly becomes still Rough grab use hand to grab hold of other monkey Swat use handlarm to hit at other monkey Gentle touch use hand to gently touch other monkey Piloerection hair is raised on body Fight fast-paced agonistic interaction involving chasing, tumbling,

biting, barking, screaming all at once Present-sexual present own body for mounting usually by moving tail aside

and presenting rear end to other monkey Present-for-grooming present own body for grooming usually by moving neck or

side or back towards other monkey, looking away, and becoming still

Groom hands systematically combing through other monkey's hair

Overall movement: No change remain stationary Approach move towards other monkey Retreat move rapidly away from other monkey Chase run after other monkey Go in several directions move back and forth Pass by pass another without stopping Leave leave another with whom it had been associating (not rapidly) Follow walk close behind another monkey, same direction

1) Identical to 'muzzle-up' (Partan, 1998). 2) Eye gaze direction was determined by head orientation as well as eye gaze. 3) Includes head movements in the vertical, rather than horizontal, domain.

names with functional connotations to describe the elements of the expressions, with the exception of pant-threat. I kept the 'threat' as a part of this name to be consistent with the previous literature (e.g. Rowell, 1962; Rowell & Hinde, 1962; Hauser et al., 1993).

All expressive components were defined as a departure from a neutral state (based on Sade, 1973). A neutral expression was defined as the countenance of a relaxed monkey (Fig. la). Neutral positions of each body part were recorded along with expressive positions (see Table 2). Expressive positions are mutually exclusive within categories. For example, when an individual's head is raised, it cannot simultaneously be lowered; or if an animal is barking, it cannot be also cooing. The only category for which internal mutual exclusivity does not hold is the 'body' category, because it is possible, for instance, for an animal tofreeze and crouch at the same time. In these cases I entered the most pronounced behavior, putting

1002 PARTAN

(t) ~~~~~~(b)

Fig. I. (a) Sub-adult rhesus male (X85) with neutral facial expression. Photo by author. (b) Adult female (E76) with silent stare and open-mouth. Digitized from videotape using

Adobe Premiere software and Panasonic editing stations.

the secondary behavior in an 'other' category. I chose to lump all body postures into this one category because I wanted to focus specifically on facial expressions and vocalizations. Body posture was of secondary interest here, although certainly of great importance in social behavior.

Eye gaze direction is difficult to assess on videotape. I have followed Altmann (1962) in his distinction of stare from look at, and I have included three other active states of the

eyes (see Table 2): look betweeni, look arounid (perhaps similar to Altmann's unit #35, 'looks apprehensively'), and look away (similar to Altmann's unit #34, 'avoids staring at'). In each of the latter three cases the eyes move quickly between, among, or away from social targets in the environment, respectively. In contrast, eyes nieutrail were relaxed eyes, often half shut or gazing off into space, not moving quickly. In cases where I could not assess eye direction from the videotape, I left the corresponding data cell blank.

Head position was troublesome to categorize because the head often moves in conjunction with other behavior, such as yawning (Deputte, 1994). Movements of the head are also incorporated into gaze behavior. I decided to prioritize mouth and eye behavior, since I was primarily interested in facial expression. Therefore yawn was categorized as a mouth behavior, and all eye gaze directions involving both head and eye movements were categorized as eye behaviors (e.g. look arounid and look between). Head movements that occurred in the vertical domain (raised, lowered, bobbed, jerked) were included in the head position category. My use of the term head bob refers to a slow, rhythmic up and down motion, whereas head jerk is a sudden sharp movement, not repeated. I follow Hinde & Rowell (1962) and Reynolds (1976) in their use of the term 'jerk' for the latter behavior (departing from Altmann, 1962, and Drickamer, 1975, in their use of the term 'bob' to refer to quick head movements during threat).

Vocalizations were classified by ear from the audio track of the videotape, after categories were established via spectrographic analysis and verified by comparison to published


spectrograms (Rowell & Hinde, 1962; Hauser & Marler, 1993) and discussions with other researchers (P. Marler, W. Mason, & M. Hauser, pers. comm.). Although rhesus screams have been subdivided into five categories by Gouzoules et al. (e.g. 1984), I could reliably distinguish only two types by ear. One type, which I labeled scream, included 'noisy', 'tonal', and 'arched' screams (Gouzoules et al., 1984); the second type, which I called squeak, included scream-like vocalizations that were very short in duration ('pulsed' screams, Gouzoules et al., 1984). If a vocalization had several syllables (as was common with screams and pant-threats), I entered it only once into the database, to avoid inflating the number of vocalizations.

Pilot observations of mouth positions during vocal behavior indicated that some vocalizations were produced with a utilitarian or purely articulatory use of the mouth: the mouth opened only to the degree and duration necessary for sound emission. Other vocalizations, however, were accompanied by elaborate mouth movements that exceeded either the duration of the call or the degree of opening necessary to produce the sound. I distinguished between these possibilities by including a category called mouth matches vocalization (Table 2), along with the expressive mouth positions. This allowed me to set aside articulatory movements for particular analyses (noted below). Note that all vocalizations are 'simultaneously multimodal' in that each occurs with a particular mouth position, providing both visual and vocal stimuli, regardless of whether the mouth is open as long as or longer than the call. A finer- grained temporal analysis might consider the latter cases to include unimodal signaling of mouth opening, followed by multimodal signaling during the call, followed again by the unimodal mouth position after the call has ended. However, here the behavioral unit included the vocalization in the context of the visual signals that encompassed it; any action that included a vocalization was considered 'multimodal'.

Data analysis

All data analysis was done using SAS software. Any entries (observations) with missing data for the behavior being analyzed were omitted prior to analysis. This reduced the sample size differentially for the different analyses, depending on which behaviors were being assessed.

I carried out three main analyses. First I examined the entire data set to determine which components co-occurred, using Principal Component Analysis (PCA) to examine overall relationships among the variables. Since PCA requires numeric data, I transformed the nominal categories into a series of indicator variables which read '1' or '0' depending on whether the particular behavior was present or absent (this method is also used and described by Deputte, 2000). I used all eight behavioral classes (vocalizations, eye, ear, head position, etc.). When I transformed the data, there were a total of 57 columns, because the 8 classes included 57 unique behavioral components or elements that were included in the analysis. (All behavioral components in Table 2 were included in the PCA except articulatory mouth matches vocalization movements and some other rare components.) The raw data matrix for the PCA analysis consisted of the 57 columns corresponding to each behavioral component, and 1215 rows corresponding to each event. The analysis therefore produced a correlation matrix with 57 columns and 57 rows, correlating each of the behavioral elements with each other element. This matrix was then analyzed to produce 57 principal components, the top six of which had eigenvalues large enough (> 1.9) to warrant discussion.

The second analysis was a comparison of the silent and vocal behaviors of the animals, based on the frequencies of occurrence of each expressive component during silent and vocal behavior as a whole.

1004 PARTAN

In the third set of analyses I examined in detail the structure of the bimodal, visual/vocal expressions. I conducted Multiple Correspondence Analyses (MCA's) to explore the distribu- tion of components two or three classes of behavior at a time, and presented the results graph- ically. The frequency data were taken from columns in the main database, with behavioral components across the columns, and one composite action per row. A correspondence analysis, like a Principal Component Analysis, detects trends in the data by establishing which components are associated (see van der Heijden et al., 1990, for detailed explanation). The analysis then defines 'dimensions' to describe the strongest associations. Like the PCA, the correspondence analysis has many dimensions, but only the two most important ones are plotted in the graphs. Strong associations between variables are shown in two ways on the MCA graphs: by distance from the origin, and by trajectory. If two behavioral components plot out on the same trajectory, they are closely associated, and the farther they are away from the origin, the stronger the association. (Diagonal lines drawn on the graphs make it easier to see which variables are associated.) Note that for the correspondence analysis of mouth positions and vocalizations, I excluded the articulatory mouth matches vocalization category.

Finally, I documented which visual components were associated with each vocalization, using frequency matrices and histograms.

For all analyses, behavior was pooled across individuals. Although I am aware of the potential dangers of pooling data for parametric tests (Machlis et al., 1985; Leger & Didrichsons, 1994), my data are nominal frequencies that cannot be averaged. I cannot, for example, find the mean of 10 open mouth threats and 5 fear grimaces to come up with an 'average expression' for a given individual who produced these 15 expressions. Furthermore I collected data on myriad components of expression, with the individuals involved in many different possible dyadic combinations, making the concept of an 'average' even less appropriate. Instead, the focus of analysis is on the frequencies with which multiple expressive components are combined, across all cases of expression.

Results

Overall relationships among the behaviors

Principal Component Analysis yielded six top components (with largest eigenvalues; Table 3). The clusters revealed by the PCA correspond to behavioral suites that have been reported previously in a more anecdotal fashion.

Component 1

The most striking association in the data was among open mouth, stare, ears forward, and head lowered (Fig. lb depicts the open mouth and stare). This suite of behaviors corresponds to descriptions of silent threat from the literature. Grimace did not occur with this cluster.


TABLE 3. Principal Component Analysis

Component Factor Weight Variable Coefficient Interpretation

PC 1 3.67 stare 0.38 Silent threat head low 0.31 open mouth 0.28 ears forward 0.28 [grimace] -0.23

PC 2 2.75 bark 0.31 Vocal fight look-around 0.27 scream 0.23 move around 0.23

PC 3 2.59 tail up 0.34 Approach approach 0.31

PC 4 2.32 girney 0.28 Affiliation lipsmack 0.24 ears back 0.24 tail wave 0.23 [grimace] -0.22

PC 5 2.08 look-at 0.38 Aggressive approach lunge 0.35 approach 0.32

PC 6 1.94 look-at 0.35 Submission grimace 0.31 /Male solicitation retreat 0.24 ears back 0.21 chin-up 0.21 [bark] -0.22

Top six components. All variables with Icoefficientl >0.2 are listed for each component. A negative coefficient indicates that the variable was negatively associated.

Component 2

The second principal component involved bark, scream, looking around, and moving in several directions. This is a typical suite of behaviors involved in agonistic interactions.

Component 3

Steady, walking approach with tail raised. Typical behavior of dominant males (Hinde, 1966; Lindburg, 1971).

1006 PARTAN

TABLE 4. Comparison of visual components associated with silent and vocal expressions

SILENT VOCAL

N Row % Column % N Row % Column % Total

Mouth: Neutral 249 98.03 44.15 5 1.97 2.02 254 Open Mouth 108 63.53 19.15 62 36.47 25.00 170 Grimace 88 58.28 15.60 63 41.72 25.40 151 Lipsmack 69 82.14 12.23 15 17.86 6.05 84 Chin-up 16 100.00 2.84 0 0.00 0.00 16 Yawn 9 100.00 1.60 0 0.00 0.00 9 Bite 9 100.00 1.60 0 0.00 0.00 9 Tooth Chomp 8 100.00 1.42 0 0.00 0.00 8 Tongue Protrusion 8 88.89 1.42 1 11.11 0.40 9 Matches Sound n/a 0.00 0.00 102 100.00 41.13 102 Column Total 564 69.46 100.00 248 30.54 100.00 812

Eye: Look-at 252 73.68 38.83 90 26.32 33.21 342 Stare 161 73.18 24.81 59 26.82 21.77 220 Neutral 99 89.19 15.25 12 10.81 4.43 111 Look-between 64 65.98 9.86 33 34.02 12.18 97 Look Away 40 83.33 6.16 8 16.67 2.95 48 Look-around 33 32.35 5.08 69 67.65 25.46 102 Column Total 649 70.54 100.00 271 29.46 100.00 920

Ear: Neutral 458 72.01 73.63 178 27.99 68.46 636 Back 100 66.67 16.08 50 33.33 19.23 150 Forward 55 75.34 8.84 18 24.66 6.92 73 Flapped 9 39.13 1.45 14 60.87 5.38 23 Column Total 622 70.52 100.00 260 29.48 100.00 882

Head: Neutral 575 69.70 81.91 250 30.30 88.34 825 Lowered 82 80.39 11.68 20 19.61 7.07 102 Jerked 20 80.00 2.85 5 20.00 1.77 25 Raised 13 68.42 1.85 6 31.58 2.12 19 Bobbed 12 85.71 1.71 2 14.29 0.71 14 Column Total 702 71.27 100.00 283 28.73 100.00 985

Body: Neutral 515 69.69 71.63 224 30.31 74.92 739 Lunge 45 59.21 6.26 31 40.79 10.37 76 Lean Away 30 90.91 4.17 3 9.09 1.00 33 Crouch 22 50.00 3.06 22 50.00 7.36 44



SILENT VOCAL

N Row % Column % N Row % Column % Total

Freeze 22 91.67 3.06 2 8.33 0.67 24 Slap Ground 17 89.47 2.36 2 10.53 0.67 19 Gentle Touch 16 94.12 2.23 1 5.88 0.33 17 Rough Grab 15 100.00 2.09 0 0.00 0.00 15 Present 11 100.00 1.53 0 0.00 0.00 11 Swat 10 76.92 1.39 3 23.08 1.00 13 Shift Position 9 90.00 1.25 1 10.00 0.33 10 Dip-turn 5 100.00 0.70 0 0.00 0.00 5 Fight 2 25.00 0.28 6 75.00 2.01 8 Piloerection 0 0.00 0.00 4 100.00 1.34 4 Column Total 719 70.63 100.00 299 29.37 100.00 1018

Movement: None 411 71.98 54.15 160 28.02 51.95 571 Approach 172 78.54 22.66 47 21.46 15.26 219 Retreat 98 63.64 12.91 56 36.36 18.18 154 Pass By 25 100.00 3.29 0 0.00 0.00 25 Chase 22 53.66 2.90 19 46.34 6.17 41 Back & Forth 15 37.50 1.98 25 62.50 8.12 40 Follow 11 91.67 1.45 1 8.33 0.32 12 Leave 5 100.00 0.66 0 0.00 0.00 5 Column Total 759 71.13 100.00 308 28.87 100.00 1067

There were a total of 760 silent behaviors and 356 vocal ones, but the totals for each section above are smaller, owing to data unavailable because a particular body part could not be seen in certain cases.

Component 4

Girney, lipsmack, ears back, tail wave. This is typical affiliative behavior. Grimace does not occur with these behavioral elements.

Component 5

Lunge, chase, look at. This is typical of an aggressive approach.

Component 6

Retreat, grimace, look at, ears back, chin-up. The first two behaviors are typical of submission. The last three are components of a solicitation behavior described by Altmann (1962, p. 378), in which a male approached a female with a stereotyped posture, head tilted back, chin up, and lips pursed

1008 PARTAN

ii .

_ i I .. i,,~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ....... . _ .... ...... ............ .. . . ...... .,, ..

_ ~~~~~~~~~.... ..... U _.,..... .....,,,t, ,

| ~ ~ ~~~~~~~~~d I'l" ...... ......._ .: ... .F;},... ,~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~. ..

Fig 2. Images from videotape of vocalizing animals. Spectrograms were collected from

videotape at the same frame as the picture, using a Kay Digital Sona-Graph model 7800. Horizontal bars mark I-kHz intervals (1-8 kHz). (a) Adult male barking, with open mouth and ears back. (b) Adult female (845) giving a pant-threat vocalization. (c) Sub-adult female (X70) giving a broadband ('noisy') scream. (d) Adult male (C78) grunting. (e) Sub-adult male girneying and waving his tail. Although this girney contains primarily broadband components, girneys can also include narrow-band sounds (e.g. see spectrogram in Kalin et al., 1992). Figure 2a reprinted with permission from: Partan, S. & Marler, P.: Communication

goes multimodal - Science 283(5406), p. 1272-1273; copyright 1999 American Associa- tion for the Advancement of Science.

and sometimes rapidly smacking. He typically approached very close to the female, almost touched his face to hers, and then immediately tured away with a stereotyped 'dip' (arms bend quickly down), and walked away. Only males were seen performing the chin-up and dip-turn; when a recipient was identified, it was female (Partan, unpublished data).

Comparison of silent and vocal expressions

Four of the top six Principal Components were silent, and two were vocal. To

compare silent and vocal behavior, I explored which visual components were


associated with vocalizations. The presence or absence of a vocalization could be reliably determined for 11 16 out of the 1215 total events recorded. Of these 1116 events, 68% (760) were silent and 32% (356) included vocalizations. Silent and vocal expressions were differentially accompanied by the various visual signal components (Table 4). The 'Vocal' column in Table 4 lumps the six major vocalizations, each of which occurred 20 times or more: barks, pant-threats, screams, squeaks, grunts, and girneys (see Fig. 2 for spectrograms). Omitted from the table are vocalizations that were recorded fewer than ten times (bark-scream mixes, coos, geckers, copulation calls, harmonic arches, and unclassifiable calls).

The 'row %' column in Table 4 compares the silent to vocal behavior. Silent expressions involved more visual components than vocal expressions. This can be demonstrated by a quick scan over the 'row %' columns to see the disparity in the number of behaviors at 100% for the silent group compared to the vocal.

The mouth was most often in a neutral position during silence, but many other mouth positions were used. All possible eye positions were recorded during silent expressions, the most common being looking at the recipient, followed by staring and neutral eye position. All ear and head positions were also recorded during silent expressions, with the neutral position predominant in both cases. Head bobs were predominantly silent. Many body and arm postures occurred during silent expressions, although most of the time the body was neutral. Presents, grabs, and dip-turns were always silent; lean away, touch, freeze, and slap ground usually so. During most silent expressions the animal was stationary. Pass by and lunge were always silent; follows usually were as well.

Expressions including vocalizations incorporated fewer visual components than did silent expressions overall. Mouth positions during the six major vocalizations included open mouths, grimaces, lipsmacks, and mouth matching the vocalization. There were no vocalizations during yawns, bites, tooth chomps, or chin-up movements.

Looking around occurred more during vocalizations than during silent behavior; looking away, however, predominantly occurred during silent behavior. Ear-flapping, lunging, crouching, fighting, and piloerection occurred pro- portionately more often during vocal than silent behaviors, as did retreating, chasing, and moving back andforth.

1010 PARTAN

Structure of multimodal signals: Visual behaviors associated with each vocalization

I used Multiple Correspondence Analysis to examine in greater detail how behaviors were associated when considering not the entire data set, as in the above PCA analysis, but just two of the eight major categories at a time (e.g. vocalization and mouth position). Associations among vocalizations and mouth positions are shown in Fig. 3, using only 'expressive' mouth positions (all positions except mouth matches vocalization). The components split into three groups, each of which contains behavioral elements devoted to one of three functions described previously in the literature: threat, submission, and affiliation (Altmann, 1962; Hinde & Rowell, 1962; Rowell & Hinde, 1962; Lindburg, 1971). Threat included bark, pant-threat, bark-scream, and

* , :

4 ,J:A . .....' ! 146,

* : .; , . .. .. ..; .,: . .:.fs..e.j. t: ..'* ".'

Fig. 3. Multiple correspondence analysis of vocalizations (circles) and mouth positions (triangles). Associations among variables are shown by common trajectory from the origin. Aggressive components clustered on upper left; submissive ones on right; neutral and friendly

ones on lower left. Diagonal lines are included to facilitate grouping of the variables. ones on lower left. Diagonal lines are included to facilitate grouping of the variables.


* 1 3-U

11..~~N"

1: ..._^K 1. ...

Fig. 4. Multiple correspondence analysis of vocalizations (circles), mouth positions (triangles), and eye gaze (squares).

open mouth; submission included scream, squeak, and grimace; affiliation included no vocalization, girney, grunt, coo, lipsmack, and chin-up.

To test whether these three clusters still appeared when combined with a visual component uninvolved with phonation, I added eye gaze direction to the vocal and mouth data and ran another MCA (Fig. 4). The same three clusters occurred. Stare fell into the aggressive cluster; look around and look between fell with submission; look at and neutral eye with affiliation.

Along with the above associations predicted from the literature, the next MCA analyses revealed several new and unexpected associations. These depict a difference in visual accompaniments of the two vocalizations considered to be aggressive, bark and pant-threat. Associations among vocalizations and ear position are shown in Fig. 5a. Threatening behaviors are on the right, and nonthreatening (affiliative and submissive) behaviors are on the left. The nonthreatening vocalizations were loosely associated with

PARTAN

' I . ' ' , '' ",;-'*'' . '*.';:*/ .' " ji '-' .:': "* ' ' y v l:****:' ..

* t . ; *' -* w * -, .'' i **r *'<'.' } *::'*. -.w !; :y_ , * ̂ .+i9 . :' **** * . :* . ' v -: * .' '-:

i . e! ! |l , . ^* l i j i ^ ̂ ! * ' *'* ^ '^ * * ' :" ' j .E X

* .-- C» AN' ,,i

'" '

.:.*' _'; * . *

^ *''i, *:,n:,

_-'' -1 ... i: s'

i-I,

-4,

- .. . ' .. 1

'-'

i.

*

:

'

'

"¢

' f:

!" .' .

'

., ,-

t.;. A t .. . ...... . . .. .. ,' . l . .','' .'.e ·

'

'- : : ^

.^ .'*.l ^ i

i: ' '.;*':; ;- . --.'.', .'s1V;

. ........

(a) . .. .. . .. .' ' ' "t ii; -* > ji:^'i

Fig. 5. Multiple correspondence analysis of (a) vocalizations (circles) and ear positions (triangles), and (b) vocalizations (circles) and head positions (triangles). In both graphs, aggressive behaviors fell to the right of the origin, and submissive and affiliative ones fell

primarily to the left.

neutral ear position. The interesting finding is that the threat vocalization

bark was associated with ears back, while pant-threat was with ears forward.

Figure 5b shows the associations among vocalizations and head position. Affiliation and submission are on the left, weakly associated with neutral

head position. On the right, pant-threat vocalizations were associated with

head raised or forward, whereas barks were with head lowered.

_0 - F- · ;-- ---;·~ · . -~;~r CL- .rcu·* 1.L-~~;-i,l;i:ii~P I~Li ·~+

_,, . . : . , . .. _ . i .

1012

SIGNAL COMPOSITION

': :..., .:.'.:. ::'-

:

.. e ...

." , "' 7 .. ... .

: , - - ' . . . '. . . . . . .. :. . . . ..

: -:...:..-: , ,-' ::..: ''" ...:.

', T,. · .

,

,. ,: . ,, ,:, ',,

.S,i:-:t ."'".... ...' ,',.:.:..:''.. ; ..

-I,

U

* .' r~~~~~~~~~~ 'jit :

Fig. 5. (Continued).

I systematically documented the visual components that occurred simultaneously with each of the six major types of vocal behaviors (Fig. 6). These are presented below, organized according to the visual components.

Mouth position

Mouth position was tightly linked to vocalization type (Fig. 6a). Barks and pant-threats were accompanied by rounded open mouths, screams and

squeaks were accompanied by grimaces, and girneys (and, to a lesser extent, grunts) were accompanied by lipsmacks. Each vocalization also had a significant proportion of mouth shapes that simply matched the sound,

1013

. .- .~ .·

^.^

*.,... :

.,., ..-"

;...... :¥ . . '"~:

~. ~.'.. '".: :, .-. ..... .. .::. .*., .- ,:.[

~.. '...4..

: ..'-'^; *. . . .... ;..

· .. . ...

'..:':.'.. '.. :".. --I...f.L','

(b.)

. . -.& -0-00- . . . - . - -

1 -- t - - : : il - uy l[ ·, . . .. -~ :~ .. -1 .711.M.

·WqL' Ti- i· -i · - · IT; i ~ · , ·· I- -. · -· · .- .. I :i-· 7 · .. - - I i- ·.l l

.:.

. ._ F-.s

'i

VE Ak':

1014 PARTAN

synchronized in time, opening and closing without any further expressive posture. For grunts, almost 80% involved no mouth expression other than opening slightly for the grunt. The lowest matching scores were found among screams and, especially, squeaks, which often occurred in the midst of prolonged silent grimaces. These vocalizations were still simultaneously multimodal: the squeaks co-occurred in time with grimaces.

Eye gaze

All six vocalizations were accompanied by substantial proportions of looking directly at the recipient and of looking around (Fig. 6b). The girney involved a higher percent of looking at the recipient (67% of all girneys), and a lower proportion of looking around (only 17 %). Barks and pant-threats were accompanied by a high proportion of staring (37% of barks, and 50% of pant-threats). Screams and squeaks had the highest proportions of looking between the recipient and a third party (13% of screams; 30% of squeaks) and of looking away from the recipient (5% of screams; 15% of squeaks).

Ear position

Ear position (Fig. 6c) was predominantly (>80%) neutral for screams, squeaks, girneys, and grunts. Barks and pant-threats differed dramatically. Ears were neutral in only 51% of barks and 40% of pant-threats. During barks, the most common nonneutral position was retracted (28%), but they could also be juttingforward (10%) orflapped back and forth (11%). In pant- threats the ears were most often forward (35%) but also could be retracted (15%) orflapped (10%).

Head position

Head position (Fig. 6d) was always neutral for screams, squeaks, and grunts. During girneys, the head was predominantly (92%) neutral, but in the remainder of the cases showed a distinctive bobbing movement not seen during any other vocalizations. Barks and pant-threats were most variable in head position. Barks were accompanied 16% of the time by head lowered, 3% by head jerked, and 2% by head raised. Pant-threats were accompanied more often than barks by head raised (20%) and head jerked (10%), and less often by head lowered (10%).


Body posture

A wide array of body postures was observed (Fig. 6e). Grunts, girneys, and pant-threats were given at least 95% of the time with neutral body position. The remaining girneys were accompanied by reaching out and gently touching the recipient. The remaining pant-threats involved body crouches. Barks, screams, and squeaks were relatively active. Barks were accompanied 16% by lunges; 4% by crouching, and 3% by piloerection. Screams were given 16% with crouching, 12% with lunging, 7% with tumbling during fighting, 4% with swatting, and 2% with leaning away. Squeaks were emitted 7% with lunging or crouching, and 4% with leaning away, freezing, or shifting position.

Overall movement

Movement (Fig. 6f) showed similar patterns to body posture, except for girneys. 40% of girneys were stationary; 52% involved approaching the recipient. Grunts and pant-threats were 90% stationary; for the remainder, the actor approached during grunts, and either approached or chased the recipient during pant-threats. Barks were heard during chases or approaches 13% each, during retreats 5%, and 6% while moving back andforth. Screams and squeaks were commonly accompanied by retreats (42% of screams; 36% of squeaks). They also were heard during approaches (12% of screams; 16% of squeaks) and while moving back andforth (14% of screams; 10% of squeaks).

Discussion

This study provides quantitative support for two areas of literature. First, the data support the idea of three 'poles' of behavior important during social communication: aggression, submission, and affiliation. Deputte (2000, p. 118) calls this the "3 A's concept (affiliation, aggression, and avoid- ance)," and Mason (1985) also singles out these three motivational states. In the present study this was shown most clearly in the Multiple Correspon- dence Analyses of the associations among mouth position, vocalization, and eye gaze (see Figs 3 & 4). Previous work identified behavior involved in these three domains in rhesus (i.e. open mouths and barks being aggressive;

1016 PARTAN

(a) 100% [ eta t U~~~~~~~~~~~~~~~~~~~~~~OTongue protr.

13Lipsmack

80% *~~~~~~~~~~~~~~~~~~~~0Grimace 10% 12 Ope~~~~~~~~~~~~~~~Qno mouth E Matches Sound

60%

40%

20%

0f% -

BARK (102) PANT(20) SCREAM(62) SQUEAK(25) GIRNEY(20) GRUNT (19)

(b)K (111)OPANT (20) SCREAM(59)SQUEAK(27)GIRNEY(23)GRUNT(20)Neutral 12Look Away

13 Look-between Fig. 6 Visual expressive components associated with each major Stare

1U Look-around * Look-ut

60%

40%

20%

0% BARK (1 13) PANT (25) SCREAM (67) SQUEAK (27) GIRNEY (24) BRUNT (25)

(c) 10% Neutral

9S% * Flapped U2 Forward

80% E Back

70%

60%

50%

40%

30%

20%

10%

BARK (It ) PANT (20) SCREAM (09) SQUEAK (27) GIRNEY (23) GRUNT (20)

Fig. 6. Visual expressive components associated with each major vocalization. (a) Mouth position; (b) eye gaze direction; (c) ear position; (d) head position; (e) body posture; (f) overall movement. The N's are based on the number of times that the vocalization was


(d) 100% Nur

90% OBobbed

80% B~~~~~~~~~~~~~~~~~~~~Jerked

80%

U Raised 70% *Lowered

60%

50%

40%

30%{

20% --

10% I

BARK (110) PANT (20) SCREAM (75) SQUEAK (29) GIRNEY (24) GRUNT (20)

(e) 700 O Neutral

90% ( Shift position MuGentle Touch

80% El~~~~~~~~~~~~~~~~~~~~UFreeze EoSlap Ground

70% 12Lean Away M Swat 13Piloerection

60% U Fight

U3 Crouch 50% U Lunge

40%

30%

20%-

10%

0% BARK (120) PANT (20) SCREAM (86) SQUEAK (29) GIRNEY (24) GRUNT (20)

7\ 100% (1) ~~~~~~~~~~~~~~~~~~ONone

90% * Follow 13Chaue

eO% HEBach & FoSth

60% U~~~~~~~~~~~~~~~~~~~~~~~~1 Retreat

70% N Approach

00%

BARK (1 20) PANT (20) SCREAM (92) SQUEAK (31) GIRNEY (25) GRUNT (20)

Fig. 6. (Continued) recorded in conjunction with any position of the mouth, eyes, etc. The N's differ slightly among panels owing to cases lost hecause the particular visual component

could not he seen.

1018 PARTAN

screams, squeaks, and grimaces submissive; and girneys and lipsmacks affiliative; Altmann, 1962; Hinde & Rowell, 1962; van Hooff, 1962, 1967; Rowell & Hinde, 1962; Marler, 1965; Lindburg, 1971; Drickamer, 1975; Redican, 1975; Mason, 1985; Boccia, 1986; Kalin et al., 1992; Maestrip- ieri, 1997; Maestripieri & Wallen, 1997; Partan, 1998). Van Hooff (1973) provided quantitative evidence for five behavioral states important in chimpanzees, including the three discussed here as well as play and excitement. He pointed out that the determination of these groupings depends partially on the degree of lumping and splitting of the initial behavioral elements that make up the catalog. Adams & Schoel (1982) found six motivational states in male stumptail macaques, including the three discussed here, and defense (which I included with submission), sexual behavior, and display.

Second, these data provide quantitative support for the previous descriptions of behavioral clusters in rhesus monkeys. For example, the top components of the Principal Component Analysis, each of which involved a suite of behaviors that clustered together, corresponded to behaviors reported (usually in a nonquantitative fashion) to have been observed together.

Comparison of silent and vocal expressions

Silent visual expressions dominated the communication signals used by these monkeys at the close distances at which they were observed. This agrees with Rowell's (1962) suggestion that the visual channel is primary for rhesus macaques, and supports the argument that, owing to selective pressures imposed by the environment, terrestrial primates such as rhesus macaques rely more heavily on vision while arboreal ones rely more on audition (Altmann, 1967; Redican, 1975). A higher proportion of behaviors in this study were vocal (31.9%), however, than was true of Altmann's (1962, 1965, 1967) study, in which only 5.1% of behaviors included vocalizations. These figures are difficult to compare, particularly because Altmann split some categories of visual behaviors more finely than I did, increasing the proportion of visual to vocal signals in his study relative to mine.

The silent expressions reported here were highly complex; in fact, silent expressions as a whole incorporated a wider range of visual components than did expressions accompanied by vocalizations (see Table 4). This may partly be due to physical constraints placed on the mouth when an animal vocalizes, inhibiting the concurrence of vocalizations with certain mouth movements


such as yawning, biting, tooth chomping, or pursing or compressing the lips. However, these particular mouth movements do not entirely preclude vocalizing: yawns have been observed to coincide occasionally with barks in grey-cheeked mangabeys, for example (Deputte, 1994).

In addition to mouth positions, some body postures never occurred while an animal vocalized, but did occur during silent behavior (see Table 4). All occasions when an individual presented its body to another, whether for grooming, occasional sexual interactions, or to indicate submission, were silent. Also silent were chin-up and dip-turn, which Hinde and Rowell referred to as 'dancing' (see Fig. 8b & c in Hinde & Rowell, 1962, p. 17). They did not mention any vocalizations occurring during this behavior. Altmann also described this behavior in detail, although he did not name it. He considered it to be an 'extreme form' of 'smacks lips at' (Altmann, 1962, p. 378), and it did not occur in combination with any vocal units. This behavior may be analogous to the 'stylized trot' described by Green (1975, p. 65) for Japanese macaques. The facial component may be similar to the 'LEN' observed in pigtail macaques (Lips protruded-Eyebrows raised- Neck forward; Jensen & Gordon, 1970, p. 268) and the 'pucker' described by Maestripieri & Wallen (1997). The males I studied were silent during the chin-up and dip-turn behavior regardless of whether or not they were lipsmacking. Hauser (1993) has reported that male rhesus monkeys who vocalize during copulation, perhaps analogous, are more likely to be attacked by other males than those who copulate silently.

Visual signals associated with vocalizations

Multimodal signals are an important part of the rhesus macaque repertoire: in this study they made up just over 30% of all behaviors in the database. For example, bark and pant-threat vocalizations were accompanied by particular visual components (open mouth, staring, ears neutral or back, head neutral or lowered, body neutral or lunging, and approaching, chasing or remaining stationary) as suggested in the literature (Altmann, 1962; Hinde & Rowell, 1962; van Hooff, 1962, 1967; Rowell & Hinde, 1962; Redican, 1975; de Waal et al., 1976; Boccia, 1986).

Novel findings include a contrast in the frequencies of the visual accompaniments of the two aggressive vocalizations. Pant-threats were more often accompanied by staring, ears in the forward position, and head raised

1020 PARTAN

or forward than were barks. Ear position during threat has been described by others to be retracted (e.g. Drickamer, 1975), but the two threatening vocalizations have not been distinguished previously in terms of these associated visual signals. Staring or looking at another individual may be a method of indexing with the eyes, indicating to whom a vocalization is directed (Itani, 1963; Altmann, 1967; Mitchell, 1972; Argyle & Cook, 1976; Green & Marler, 1979). Ears in the forward position may serve a similar function, indicating the intended addressee (Partan & Marler, 2002). The correlations between pant-threat and indexical eye and ear positions suggest that pant-threats are typically directed at one or more targeted individuals, whereas barks are propagated more generally. Barks were more often accompanied by looking around and looking between other individuals than were pant-threats. 'Broadcast' signals are not directed to any recipient in particular but are presented generally for all to see or hear. Other exam- ples of broadcast signals in rhesus include branch shaking, yawning, and tail raising. In the stumptail macaque, broadcast signals such as branch shaking (called 'bouncing') and patrolling were also associated with barks (Adams & Schoel, 1982).

The visual accompaniments of the two submissive vocalizations, screams and squeaks, differed. Both were generally accompanied by grimacing, looking at, around, or between two other animals, neutral ears, heads, and bod- ies, and retreating or remaining stationary (Fig. 6), in agreement with previous studies (Altmann, 1962; Hinde & Rowell, 1962; van Hooff, 1962, 1967; Rowell & Hinde, 1962; de Waal et al., 1976; Maestripieri & Wallen, 1997). Not previously documented, however, are that the ears were never held forward or flapped during submissive vocalizations, and that screams and squeaks differed in the proportions of their visual accompaniments. Squeaks (shorter than screams) occurred with prolonged, stereotypical grimacing mouth positions, and were more often accompanied by looking between the recipient and a third animal than were screams. Squeaks never occurred during actual fighting, however; they were more likely to occur during breaks in fighting or after the fight was over. Since looking between can be used for recruitment, squeaks may have been used more often than screams when soliciting support from allies present at the scene.

Two acoustically distinct vocalizations, girney and grunt, were the only ones to be sometimes accompanied by the affiliative behaviors of lipsmacking and, in the case of girneys, head bobbing (Fig. 6). Vocalizing animals


usually looked at the recipient, had neutral ear, head, and body positions, and either approached (in the case of girneys) or remained stationary. The ears, as with submissive behaviors, were never heldforward or flapped during these two vocalizations. Girneys and grunts differed in the proportions of their visual accompaniments. Girneys were most often given with lipsmacking, an affiliative signal (Hinde & Rowell, 1962; Mason, 1985; Maestripieri & Wallen, 1997). Grunts were much more likely than girneys to be given without any mouth expression at all. Grunts were also more likely than girneys to be accompanied by neutral positions of the eyes, ears, head, and body, and grunts were usually not accompanied by movement in any particular direction. Adams & Schoel (1982) found that stumptail macaque grunts were likewise not significantly associated with any particular acts or postures.

Rowell & Hinde (1962) suggested that rhesus girneys (or 'girns') are affiliative, and Kalin et al. (1992) provided empirical evidence for this in infant monkeys. Hauser et al., (1993) classified both girneys and grunts as affiliative. Hauser & Marler (1993) stated that grunts occur in multiple contexts, including both affiliative and food-related. Kaldor (1996) lumped grunts with girneys in her analyses and empirically found them together to be affiliative. My data are consistent with the suggestion that girneys are affiliative, but I have little evidence that grunts have any emotional valence whatsoever.

Mouth position

Mouth position differs from the other visual expressions discussed in that it is intricately tied with vocal production. Although vocalizations and mouth shapes must to some extent be associated, there is disagreement about the order of events on an evolutionary timescale. Darwin (1872, p. 91) mentioned this as an "obscure point, namely, whether the sounds which are produced under various states of the mind determine the shape of the mouth, or whether its shape is not determined by independent causes, and the sound thus modified". Rowell (1962) suggested that vocalizations are secondary to visual signals, postures, and breathing patterns of rhesus monkeys. Andrew (1963) agreed that vocalizations are dependent on mouth shapes, although he suggested that the mouth shapes originated primarily as protective responses toward noxious environmental stimuli. Ohala (1984), however, indicated that the vocalizations were primary: he suggested that

1022 PARTAN

mouth shape originated in order to produce specific types of sound. Ohala based his ideas on Morton's (1977) 'motivational-structural rules,' which predict that an aggressive animal will attempt to appear large by using sounds of lower frequencies, and submissive animals will try to appear small, using higher-pitched sounds (but see Hauser et al. 1993). Ohala suggested that large size could also be communicated by a large resonant chamber, such as that created by the rounded mouth of a macaque displaying an open-mouth threat. Conversely, smallness could be conveyed by high resonance, which would occur if the monkey pulled its lips back, shortening the length of the resonance chamber. Ohala suggested that through ritualization, the mouth shapes, originally in service of vocalizations, became independent and are now communicative on their own.

I distinguished between articulatory and expressive movements of the mouth. I found that the most common mouth shapes accompanying each vocalization either matched the vocalization exactly (i.e. the mouth was opened only to the degree necessary to produce the sound) or were of a shape perhaps constrained by the requisite sound production (grimace for screams and squeaks; rounded open mouth for barks; neutral or lipsmacking for girneys). This provides quantitative support for the illustrations in Hauser et al. (1993) of typical articulatory gestures accompanying eight main vocalizations of rhesus macaques. These data suggest that the mouth and the voice are not emancipated during bimodal (vocal) production. During unimodal, silent (visual) behavior, the mouth took on a variety of postures and movements not observed during vocalizations, although the silent open mouths, grimaces, and lipsmacks were similar in gross structure to the vocal ones.

I compared the proportion of each vocalization that was accompanied by matching mouth shapes (see Fig. 6a). Grunts had the highest scores for mouth positions exactly matching the vocalization, indicating that there was no particular silent expressive shape of the mouth that accompanied grunting. Hauser et al. (1993) found, similarly, that grunts were produced with little separation of the lips. I found that screams and squeaks, in contrast to grunts, had low scores for mouth matching, indicating that these vocalizations were often accompanied by extended mouth expressions, in this case grimaces. Girneys also had low scores for mouth matching, because they were often produced simultaneously with extensive lipsmacking.


Matching mouth positions may carry no new information, being purely redundant with the vocalization, whereas an extended expressive mouth position may provide additional information. For example, an expressive mouth position such as a prolonged, exaggerated grimace might reflect a higher intensity response than a more utilitarian (smaller, briefer) grimace that simply matched a scream (see Maestripieri, 1997; Partan, 1998). Marler (1992) discussed the idea that certain components of a signal may reflect intensity of response while others reflect referential information.

Conclusion

The detailed associations among visual and vocal components of rhesus monkey expressions demonstrate a method for quantifying the structure of multichannel signals. The results provide new insights into signal structure and quantitative support for previous descriptions of rhesus behavior found in the literature. That particular vocalizations are not always paired with the exact same visual expressions reflects the fact that all behavior is probabilistic (as Altmann, 1965, discussed for sequences of behavior). However, some vocal expressions were consistently paired with predictable visual components; this was more often the case for screams and squeaks than for barks and pant-threats. Silent expressions were the most variable in terms of the numbers of associated visual components.

References

Adams, D.B. & Schoel, M.W. (1982). A statistical analysis of the social behavior of the male stumptail macaque (Macaca arctoides). - Am. J. Primatol. 2, p. 249-273.

Altmann, S.A. (1962). A field study of the sociobiology of rhesus monkeys, Macaca mulatta. - Ann. N.Y. Acad. Sci. 102, p. 338-435.

- (1965). Sociobiology of rhesus monkeys. II: Stochastics of social communication. J. theoret. Biol. 8, p. 490-522.

- (1967). The structure of primate social communication. - In: Social Communication Among Primates (S.A. Altmann, ed.). The University of Chicago Press, Chicago, p. 325-362.

Andrew, R.J. (1963). The origin and evolution of the calls and facial expressions of the primates. -Behaviour 20, p. 1-109.

Argyle, M. & Cook, M. (1976). Gaze and mutual gaze. - Cambridge University Press, Cambridge.

Boccia, M.L. (1986). Grooming site preferences as a form of tactile communication and their role in the social relations of rhesus monkeys. - In: Current perspectives in primate social dynamics (D.M. Taub & F.A. King, eds). Van Nostrand Reinhold Co., New York.

1024 PARTAN

Bradbury, J.W. & Vehrencamp, S.L. (1998). Principles of animal communication. Sinauer Associates, Inc., Sunderland, Massachusetts.

Conner, W.E. (1987). Ultrasound: Its role in the courtship of the arctiid moth, Cynia tenera. - Experientia 43, p. 1029-103 1.

Darwin, C. 1872/1897. The expression of the emotions in man and animals. - Second edition. D. Appleton & Co., New York.

Deputte, B.L. (1994). Ethological study of yawning in primates. I. Quantitative analysis and study of causation in two species of Old World monkeys (Cercocebus albigena and Macacafascicularis). - Ethology 98, p. 221-245.

- (2000). Primate socialization revisited: Theoretical and practical issues in social ontogeny. - Adv. Study Behavior 29, p. 99-157.

Drickamer, L.C. (1975). Quantitative observation of behavior in free-ranging Macaca mulatta: Methodology and aggression. -Behaviour 55, p. 209-236.

Drummond, H. (1981). The nature and description of behavior patterns. - In: Perspectives in ethology, Vol. 4 (P.P.G. Bateson & P.H. Klopfer, eds). Plenum, New York, p. 1-33.

Fusani, L., Hutchison, R.E. & Hutchison, J.B. (1997). Vocal-postural co-ordination of a sexually dimorphic display in a monomorphic species: The Barbary dove. - Behaviour 134, p. 321-335.

Goldfoot, D.A. (1982). Multiple channels of sexual communication in rhesus monkeys: Role of olfactory cues. - In: Primate communication (C.T. Snowdon, C.H. Brown & M.R. Petersen, eds). Cambridge University Press, Cambridge, p. 413-428.

Gouzoules, S., Gouzoules, H. & Marler, P. (1984). Rhesus monkey (Macaca mulatta) screams: Representational signaling in the recruitment of agonistic aid. - Anim. Behav. 32, p. 182-193.

Green, S. (1975). Variation of vocal pattern with social situation in the Japanese monkey (Macaca fuscata): A field study. - In: Primate behavior (L.A. Rosenblum, ed.). Academic Press, New York, p. 1-102.

- & Marler, P. (1979). The analysis of animal communication. - In: Handbook of behavioral neurobiology, Vol. 3; Social behavior and communication (P. Marler & J.G. Vandenbergh, eds). Plenum Press, New York.

Hauser, M.D. (1991). Sources of acoustic variation in rhesus macaque (Macaca mulatta) vocalizations. - Ethology 89, p. 29-46.

- (1993). Rhesus monkey copulation calls: Honest signals for female choice? - Proc. R. Soc. Lond. B 254, p. 93-96.

- (1996). Vocal communication in macaques: Causes of variation. - In: Evolutionary ecology and behavior of macaques (J.E. Fa & D. Lindburg, eds). Cambridge University Press, Cambridge.

- , Evans, C.S. & Marler, P. (1993). The role of articulation in the production of rhesus monkey, Macaca mulatta, vocalizations. - Anim. Behav. 45, 423-433.

- & Marler, P. (1993). Food-associated calls in rhesus macaques (Macaca mulatta): I. Socioecological factors. - Behav. Ecol. 4, p. 194-205.

van der Heijden, P.G., de Vries, H. & van Hooff, J.A.R.A.M. (1990). Correspondence analysis of transition matrices, with special attention to missing entries and asymmetry. - Anim. Behav. 40, p. 49-64.

Hinde, R.A. (1966). Ritualization and social communication in rhesus monkeys. - Phil. Trans. Roy. Soc. London 251, p. 285-294.


& Rowell, T.E. (1962). Communication by postures and facial expressions in the rhesus monkey (Macaca mulatta). - Proc. Zool. Soc. Lond. 138, p. 1-21.

van Hooff, J.A.R.A.M. (1962). Facial expressions in higher primates. - Symp. Zool. Soc. Lond. 8, p. 97-125. (1967). The facial displays of the Catarrhine monkeys and apes. - In: Primate ethology (D. Morris, ed.). Aldine Pub. Co., Chicago, p. 7-68.

- (1973). A structural analysis of the social behaviour of a semi-captive group of chimpanzees. - In: Social communication and movement: Studies of interaction and expression in man and chimpanzee (M.V. Cranach & I. Vine, eds). Academic Press, London, p. 75-162. ( 1982). Categories and sequences of behavior: Methods of description and analysis.-

In: Handbook of methods in nonverbal behavior research (K.R. Scherer & P. Ekman, eds). Cambridge University Press, Cambridge, p. 362-439.

Itani, J. (1963). Vocal communication of the wild Japanese monkey. - Primates 4, p. 11-66. Jensen, G.D. & Gordon, B.N. (1970). Sequences of mother-infant behavior following a facial

communicative gesture of pigtail monkeys. - Biol. Psychiatry 2, p. 267-272. Kaldor, E.L. (1996). The meaning and social function of grunt and girney vocalizations in

Macaca mulatta. - Msc thesis, University of California, Los Angeles. Kalin, N.H., Shelton, S.E. & Snowdon, C.T. (1992). Affiliative vocalizations in infant rhesus

macaques (Macaca mulatta). - J. Comp. Psychol. 106, p. 254-261. Leger, D.W. & Didrichsons, I.A. (1994). An assessment of data pooling and some alternatives.

- Anim. Behav. 48, p. 823-832. Lindburg, D.G. (1971). The rhesus monkey in North India: An ecological and behavioral

study. In: Primate behavior: Developments in field research (L.A. Rosenblum, ed.). Academic Press, New York, p. 1-106.

Machlis, L., Dodd, P.W.D. & Fentress, J.C. (1985). The pooling fallacy: Problems arising when individuals contribute more than one observation to the data set. - Z. Tierpsy- chol. 68, p. 201-214.

Maestripieri, D. (1997). Gestural communication in macaques: Usage and meaning of nonvocal signals. Evol. Communic. 1, p. 193-222. & Wallen, K. (1997). Affiliative and submissive communication in rhesus macaques.

- Primates 38, p. 127-138. Marler, P. (1965). Communication in monkeys and apes. - In: Primate behavior: Field

studies of monkeys and apes (I. DeVore, ed.). Holt, Rinehardt and Winston, New York, p. 544-584.

- (1982). Avian and primate communication: The problem of natural categories. Neurosci. & Biobehav. Rev. 6, p. 87-94.

- (1992). Functions of arousal and emotion in primate communication: a semiotic approach. - In: Topics in primatology (T. Nishida, W.C. McGrew, P. Marler, M. Pickford & F.B. deWaal, eds). Univ. of Tokyo Press, Tokyo, p. 225-233.

- & Hamilton, W.J. III (1966). Mechanism of animal behavior. - John Wiley & Sons, NY.

Martin, P. & Bateson, P. (1993). Measuring behaviour. - Cambridge University Press, Cambridge, UK.

Mason, W.A. (1985). Experiential influences on the development of expressive behaviors in rhesus monkeys. - In: The development of expressive behavior: Biology-environment interactions (G. Zivin, ed.). Academic Press, New York, p. 117-152.

1026 PARTAN

Massaro, D.W. (1998). Perceiving talking faces: From speech perception to a behavioral principle. The MIT Press, Cambridge, Massachusetts.

Maxim, P.E. (1982). Contexts and messages in macaque social communication. - Am. J. Primatol. 2, p. 63-85.

- (1985). Multidimensional scaling of macaque social interaction. - Am. J. Primatol. 8, p. 279-288.

McGurk, H. & Macdonald, J. (1976). Hearing lips and seeing voices. - Nature 264, p. 746- 748.

Mitchell, G. (1972). Looking behavior in the rhesus monkey. J. Phenomenol. Psychol. 3, p. 53-67.

Morton, E.S. (1977). On the occurrence and significance of motivation-structural rules in some bird and mammal sounds. - Am. Nat. 111, p. 855-869.

Ohala, J.J. (1984). An ethological perspective on common cross-language utilization of Fo of voice. - Phonetica 40, p. 1- 18.

Partan, S.R. (1998). Multimodal communication: The integration of visual and vocal signals by rhesus macaques. - Dissertation, University of California.

- & Marler, P. (1999). Communication goes multimodal. - Science 283, p. 1272-1273. - & (2002). The Umwelt and its relevance to animal communication. Introduction

to special issue. -J. Comp. Psychol. 116, p. 116-119. Rawlins, R.G. & Kessler, M.J. (1985). Climate and seasonal reproduction in the Cayo

Santiago macaques. - Am. J. Primatol. 9, p. 87-99. - & - (1986). The history of the Cayo Santiago colony. - In: The Cayo Santiago

macaques: History, behavior and biology (R.G. Rawlins & M.J. Kessler, eds). State University of New York Press, Albany, p. 13-45.

Redican, W.K. (1975). Facial expressions in nonhuman primates. - In: Primate behavior: Development in field and laboratory research (L.A. Rosenblum, ed.). Academic Press, New York, p. 103-194.

Reynolds, V. (1976). The origins of a behavioral vocabulary: the case of the rhesus monkey. J. Theory Social Behav. 6, p. 105-142.

Rowe, C. (1999). Receiver psychology and the evolution of multicomponent signals. Anim. Behav. 58, p. 921-931.

- & Guilford, T. (1996). Hidden colour aversions in domestic chicks triggered by pyrazine odours of insect warning displays. - Nature 383, p. 520-522.

Rowell, T.E. (1962). Agonistic noises of the rhesus monkey (Macaca mulatta). - Symp. Zool. Soc. Lond. 8, p. 91-96.

- & Hinde, R.A. (1962). Vocal communication by the rhesus monkey (Macaca mulatta). - Proc. Zool. Soc. Lond. 138, p. 279-294.

Sade, D.S. (1973). An ethogram for rhesus monkeys: I. Antithetical contrasts in posture and movement. - Am. J. Phys. Anthrop. 38, p. 537-542.

de Waal, F.B.M., van Hooff, J.A.R.A.M. & Netto, W.J. (1976). An ethological analysis of types of agonistic interaction in a captive group of Java-monkeys (Macacafascicularis). - Primates 17, p. 257-290.

Zeller, A.C. (1986). Comparison of component patterns in threatening and friendly gestures in Macaca sylvanus of Gibraltar. - In: Current perspectives in primate social dynamics (D.M. Taub & F.A. King, eds). Van Nostrand Reinhold Co., New York, p. 487-504.


- - (1996). The inter-play of organisation and facial communication in the macaques. - In: Evolution and ecology of macaque societies (J.E. Fa & D.G. Lindburg, eds). Cambridge University Press, Cambridge.

single and multichannel signal composition: facial ... · single and multichannel signal...

Documents