GESPIN – GESTURE & SPEECH IN INTERACTION – Poznań, 24-26 September 2009

The semiotic notion of gesture in Israeli Sign Language Orit Fuks and Yishai Tobin

Ben-Gurion University of the Negev Be'er Sheva, Israel

ofuks@inter.net.il/yishai@bgu.ac.il

Abstract This study deals with the influence of gestures on the dynamic and static structure of Israeli Sign Language (ISR). In the past, much effort was invested by generative-oriented linguistic theories to show that sign language was composed of the same kinds of elements and functioned according to a similar set of rules and constraints as spoken language. As a consequence, the study of gestures was totally separated from research in sign language and any gestural expression in the formal structure of the language was totally rejected. Today, the legitimate status of sign language as a full-fledged human language is no longer held in doubt. This fact allows for research concentrating on the differences rather than the similarities between signed and spoken languages by linguists in general and by sign-oriented or semiotic-based linguists in particular (Tobin, 1990, 2009). All of the sign languages that have been studied have been defined as being natural but iconic languages which enhanced the "Imagistic thought" (McNeill,1992; Langacker, 1987) and facilitated the interpretation and expression of the formal linguistic units as well as the "less-linguistic" gestural elements of signed languages. The interaction between the gestural and the linguistic elements in the development and in the use of ISL will be the focus of this paper.

1 Introduction Since Stokoe's (1960) groundbreaking and seminal work, a large body of research has shown that sign languages are natural languages that have developed independently throughout the world and that each signed language has its own independent lexicon. Yet, the use of communicative body movement is not the unique or exclusive property of the deaf. Hearing speakers also use gestures in spoken discourse in order to enhance arguments that cannot be simultaneously encoded in speech (McNeill 1992). Kendon (2004) defined the term GESTURE as a hand, facial, or body movement that is used expressly for purposes of communication. In other words, a gesture is a purposeful act of communication. In his earlier works Kendon classified body gestures into five categories that were later placed by McNeill (1992) into a continuum which was subsequently labeled as "Kendon's Continuum". Linguists employ Kendon's Continuum as the basis for distinguishing between signed linguistic structures versus other communicative manual gestures. Kendon's Continuum presented in figure 1 reflects the connection of every kind of gesture to language (adapted from McNeill (1992:9):

Figure 1: Gestural Signs on a Most to Least Linguistic Continuum (Kendon's Continuum)

Most Linguistic Least Linguistic Sign Language Emblems Pointing Pantomimes Gesticulation

GESPIN proceedings, vol. I

2

Gesticulation relates to spontaneous and idiosyncratic body movements that appear in the process of speaking that can be grammatically merged in the structure of the sentence (e.g. movements that can complete a sentence by replacing one of its parts). Gesticulations do not appear alone, i.e. unaccompanied by speech, nor do they string together compositionally.

Pantomimes are iconic and idiosyncratic representations of everyday actions like riding a bicycle or kicking a ball, etc. Pantomimes do not necessarily appear with speech and they can be joined together in a way similar to a sentence. Yet, in a way similar to gesticulations, speakers do not have a fixed standard lexicon of pantomimes that allow them to be freely related to events or activities that occur at different locations and times in space.

McNeill (2000) places pointing between pantomimes and emblems because pointing has a conventional form in every given culture. Pointing is also placed in the middle of the continuum because in specific discourse contexts their simultaneous appearance with spoken language is necessary whereas in other contexts they can be understood without speech.

Emblems, in contrast with pantomimes, express a fixed and stable connection between form and meaning. Their meaning is conventional and they generally appear in the absence of spoken language. Nevertheless, speakers have an extremely limited lexicon of emblems as opposed to linguistic signs and they rarely combine together to form sentences.

If we summarize, progress in the continuum from left (the most linguistic) to right (the least linguistic) indicates that the obligatory presence of spoken language is reduced, as is the presence of language-like features, and fixed social conventional forms and constructions are replaced by more idiosyncratic gestures.

2 Methods and Materials In the last few years pointing has been the subject of much attention in the linguistic research of sign language. There has been much discussion in the literature concerning the question: Is the morphophonemic structure of part of the lexemes of sign language derived from both linguistic units and gestural elements together? Liddell (2003), in a revolutionary suggestion, claims that spatial verbs, pronouns, and agreeing verbs are composed in sign language from linguistic units (hand shapes and movements) and from gestural elements (locations/directionality and orientations). He claims that while the production of hand shapes and movements in spatial verbs, pronouns, and agreeing verbs are fixed and not discourse-dependent, the production and the meaning of the less linguistic elements, i.e. directionality is context-determined.

For example, the basic linguistic units of the ISL sign (as in other sign languages) such as the third person singular pronoun are the index hand shape + a short outward movement. The "phonological" gestural element in the structure of the sign is the direction of the movement. The direction of the movement of the pointing is not fixed but changes according to the discourse context. In the beginning of the discourse the signer determines a point in space (and sometimes the object) that is identified with what is "physically not present" and directs the pointing towards the same location every time the discourse deals with what is not physically present. Lidell's claim contradicts the widely-accepted generative claim that every location in space towards which the movement of pointing is directed is a morpheme.

Another topic that has been recently discussed deals with the question: Does the ability to simultaneously produce a linguistic sign and a gesture with the same articulator exist in sign language as it does in spoken language? Okrent (2002) relates to the semiotic aspect of McNeill's (2002) perception of gestures, and claims that the use of gesture is not dependent on the channel of communication. Gestures according to McNeill express of the imagistic side of thought in the speech process. Thus, according to McNeill, the appearance of gestures at the time of speaking motivated by the work of two kind of cognition during the language use: (a) global imagistic thinking that is reflected in gestures and (b) linear segmental thinking that is reflected in speech. Recent research has examined the operations of these types of thinking by means of the simultaneous use manual gestures and speech (Sandler, 1999). Yet Okrent claimed that imagistic thinking in terms of image can be expressed not only by manual gestures but also by verbal gestures that are created in the process of speech. An example of an iconic spoken gesture created while speaking according to Okrent is an exaggerated lengthening of vowels in order to reflect the temporal dimension: 'It happened a loooooong time ago'.

Orit Fuks and Yishai Tobin: The semiotic notion of gesture in Israeli Sign Language

3

3 Discussion In Israeli and other sign languages the simultaneous appearance of iconic gestures together with linguistic signs in the same act of communication is possible (Emmorey, 1999). In ISL signers often tend to produce changes in the basic structure of lexical signs during the discourse. These changes in the basic structure of lexical signs can only be understood in context, and they are created in order to produce images that cannot be encoded in linguistic units. For example, one of the consultants in our research produced in the context of discourse a basic change in the sign for the plain verb "to correct" in the space next to his temple instead of in the more neutral location next to his chest. Not only was the location of the production of the sign changed but he also changed the orientation of the palm of the passive hand: instead of facing towards the ceiling, the palm of the passive hand stood in the new location facing the direction of the other side of his body.

The iconic-gestural change in the location and direction of the articulator was meant to denote the imagistic message of the signer in the context of the discourse: "The psychologist "corrected"/put his HEAD in order" (with the sign 'fix'). The lateral placement of the palm of the hand reflected the patient, the location [next to the temple] reflected the head of the patient and the active hand that produced the sign 'fix' stood for the psychologist. In figures 2.1 and 2.2 we present the sign for the plain verb 'fix' and the gestural version of 'fix', respectively.

Figure 2.1: The plain verb: fix Figure 2.2: The gestural version of fix

The reciprocal relationship between gestures and signs in the process of signing and the research on the dynamic dimension of language have hardly been exhausted. Yet the idea that gestures directly express imagistic thinking influences in our opinion not only the dynamic structure of signs during the discourse, but also the static formation of the lexicon, more specifically the creation of lexical paradigms.

A stratified sample was randomly selected from the index of the Dictionary of ISL. Our random stratified sample yielded the following corpus of data: the category of nouns included 356 lexical signs, the category of action words included 96 lexical signs, the general category included 57 lexical signs and the category for features included 51 lexical signs. The total number of lexical signs in the random stratified sample in our research is 560. A content analysis that examined the distribution of the basic phonological unites in the lexicon of ISL showed that the signed Israeli lexicon was constructed on paradigmatic clusters of paradigms.

Each cluster revolves around an iconic phonological basic unit (hand shape, location, movement) that serves as a prototype or global image for the rest of the signs that are connected to the same content theme. For example: the iconic image associated with the hand shape is the grasping or holding of money (bills of currency). Therefore, as a consequence of this association, the hand shape is the prototype in the content area of money and appears in all the lexical signs associated with money. A similar phenomenon also is found in other sign languages such as ASL (Boyes-Braem, 1981), BSL (Brennan, 1990) and SLN (Van der Kooij, 2002).

GESPIN proceedings, vol. I

4

An interesting finding in our research is that some of the paradigms found in the lexicon of ISL developed around an image associated with a gestural sign that was prevalent in the use of the hearing community. Signs such as 'moment', 'strong', 'woe, alas, ah!', 'terrible', 'awful', 'beg/request', 'OK', '100%', are all examples of gestural signs that appear in the lexicon of ISL and are considered as accepted conventional gestures in the hearing community in Israel as well. Our research revealed that the image associated with some of the gestures of the sign apparently may be used as a source to expand the vocabulary in the lexicon of ISL. Johnson & Schembri (2007) reported that a similar phenomenon also exists in Australian Sign Language. The following are some examples of lexical paradigms in the ISL lexicon revolving around the image associated with gestures of the sign:

- The paradigm of "prohibition" ('nu!-nu!-nu': This paradigm exemplifies the use of the hand shape which appears in lexical signs associated with the prohibitive ('nu!-nu!-nu'! gesture: 'forbidden', 'annoy', and 'chastise/punish'. In figure 3 we present the sign for 'punishment'.

Figure 3: The sign: punishment

- The paradigm of 'anti': revolving around the hand shape: that is exploited by the hearing community to convey the message of discontent and opposition. In the lexicon of ISL it appears in signs such as: 'oppose/against', 'confrontation/competition', 'enemy', 'consider'. It is interesting to note that in the sign 'consider' the degree of bending of the active articulator is adapted by the message being conveyed by the sign: When the sign is produced in the context of a positive discourse, the base joint in its natural point of origin or in a position of being partially bent. When the signer wants to convey a negative message regarding the situation of a "reckoning" (euphemistically) the articulator is totally or fully bent the whole way. In figure 4 we have the sign for 'confrontation'.

Figure 4: The sign: confrontation

Orit Fuks and Yishai Tobin: The semiotic notion of gesture in Israeli Sign Language

5

-The paradigm of 'humour, facetiousness, mocking': uses the hand shape that appears in abstract lexical signs associated with the semantic domain of 'joke, jest, comedy, farce, witticism, mockery': 'joke', 'laughter', 'to fool someone', 'to lie', 'to deceive', 'sneer' / 'mock' / deride' / 'scoff'/ ridicule' / 'scorn' / 'laugh at' / 'make fun of'. The 'mocking' paradigm apparently developed from the derisive gestures of "making horns" behind the head of the person that the gesture is being made. In figure 5 we have the sign for 'to lie' and in figure 6 the sign for 'to mock':

Figure 5: The sign: lie Figure 6: The sign: to mock

- -The paradigm of 'OK – excellent': revolving around the hand shape that appears in the gestural sign 'OK – excellent'. Additional lexical signs using the hand shape an image for excellence are: 'special', 'excellent / excellence', 'superior' /'distinguished' / 'eminent', 'very good', 'cool / kool', 'laudable' / 'best' / 'choicest' 'fancy' / 'luxury'. In figures 7 and 8 we present the signs for 'excellent' and 'fancy' / 'luxury', respectively.

Figure 7: The sign: excellent Figure 8: The sign: fancy/luxury

To sum up: All of the examples presented above provide evidence that gestures play a key role in ISL both on the discourse level and in the development and formation of the lexicon. The fact that signed languages are produced in a visual rather than an auditory medium that is more accessible to iconicity, in our opinion, facilitates the realization and the expression of imagistic thinking through the use of gestural and linguistic means as well as the ability of images to influence and enhance processes associated with the formation and use of language. This research is related to and an extension of our previous work emphasizing the semiotic and iconic aspects of ISL (Fuks 2008, 2009, Fuks & Tobin 2008, 2009, and Tobin 2007a-b, 2008).

GESPIN proceedings, vol. I

6

Bibliography Brennan, M. 1990. Word formation in British Sign Language. Stockholm: University of Stockholm Press. Boyes-Bream, P. 1981. Distinctive features of the handshape in American Sign Language. Unpublished

doctoral dissertation, University of California, Berkeley. Emmorey, K. 1999. Do signers gesture? In. L. Messing & R. Campbell (Eds.), Gesture, speech, and sign (pp.

133-159). New-York: Oxford University Press. Fuks, O. 2008. Israeli Sign Language (ISL) according to according to the sign-oriented approach of the Columbia school and the theory of Phonology as Human Behavior. Ph.D, Ben-Gurion University of the Negev.

Fuks, O; Tobin, Y. 2008. The signs B and B-bent in Israeli Sign Language according to the theory of Phonology as Human Behavior. International Journal of Clinical and Phonetics and Linguistics 22 (4-5): 391-400.

Fuks, O. 2009. The status of '"movement"' in the semiotic phonology of Israeli Sign Language. Special Theme Issue: Phonology as Human Behavior. Asia-Pacific Journal of Speech, Language and Hearing 12(2): 201-211.

Fuks, O; Tobin, Y. 2009. Struggle and compromise between the striving for transparency and the tendency for ease of performance in the semiotic phonology of Israeli Sign Language. Special Theme Issue: Phonology as Human Behavior. Asia-Pacific Journal of Speech, Language and Hearing (12:2): 213-219.

Johnson, T.; A. Schembri. 2007. Australian Sign language: An introduction to sign language linguistics. Cambridge: Cambridge University Press.

Kendon, A. 2004. Gesture: Visible action as utterance. Cambridge: Cambridge University Press. Kooij, E. van der. 2002. Phonological categories in sign language of the Netherlands: The role of phonetic

implementation and iconicity. Utrecht, The Netherlands: LOT. Langacker,R. 1987. Foundations of cognitive grammar. Vol.1: Theoretical pre-requisites. Stanford. CA:

Stanford University Press. Liddell, S. K. 2002. Modality effects and conflicting agendas'. In D. F. Armstrong; M. A. Karchmer & J. van

Cleve, (eds.), The study of signed languages (pp. 53-81). Liddell, S. K. 2003. Grammar, gesture, and meaning in American Sign Language. Cambridge: Cambridge

University Press. McNeill, D. 1992. Hand and mind: What gestures reveal about thought. Chicago: Chicago University Press. McNeill, D. (ed.) 2000. Language and gesture. Cambridge: Cambridge University Press. Okrent, A. 2002. A modality-free notion of gesture and how it can help ss with the morpheme vs. gesture

question in sign language linguistics (or at least give us some criteria to work with). In R. P. Meier, K. A. Cormier & D. Quinto-Pozons (eds.), Modality and Structure in Signed and Spoken Language (pp. 175-198). Cambridge University Press.

Sandler, W. 1999. The medium is the message: Prosodic interpretation of linguistic content in sign language. Sign Language and Linguistics ( 2 /2): 187-216.

Stokoe, W. C. 1960. Sign language structure: An outline of the visual communication systems of the American deaf (Vol. 8). Silver Spring, MD: Linstok Press.

Tobin , Y. 1990. Semiotics and linguistics, London and New York: Longman. Tobin, Y. 2007a. A semiotic view of signed versus spoken language. In: Actas-1 del X Simposio

Internacional de Comunicacion Social, Santiago de Cuba 2007. 428-432. Tobin, Y. 2007b. Arbitrariness versus iconicity in signed and spoken languages. In: Memorias de la

Conferencia Linguistica Internacional 2007. El Instituto de Linguistics y Literatura, la Habana, Cuba. CD Rom, 25 pp.

Tobin, Y. 2008. Looking at sign language as a visual and gestural short hand. Poznan Studies in Contemporary Linguistics. 41(1): 103-119.

Tobin, Y. (ed.). 2009. Special Theme Issue: Phonology as Human Behavior. Asia Pacific Journal of Speech, Language and Hearing 12(2.) (March, 2009).