multimodal discourse analysis systemic functional perspectives

Multirnodal Discourse Analysis

Systemic-Functional Perspectives

Open Linguistics SeriesSeries EditorRobin Fawcett, Cardiff University

The series is 'open' in two related ways. First, it is not confined to works associated withany one school of linguistics. For almost two decades the series has played a significantrole in establishing and maintaining the present climate of 'openness' in linguistics, andwe intend to maintain this tradition. However, we particularly welcome works whichexplore the nature and use of language through modelling its potential for use in socialcontexts, or through a cognitive model of language - or indeed a combination of the two.

The series is also 'open' in the sense that it welcomes works that open out 'core'linguistics in various ways: to give a central place to the description of natural texts and theuse of corpora; to encompass discourse 'above the sentence'; to relate language to othersemiotic systems; to apply linguistics in fields such as education, language pathology andlaw; and to explore the areas that lie between linguistics and its neighbouring disciplinessuch as semiotics, psychology, sociology, philosophy, and cultural and literary studies.

Continuum also publishes a series that offers a forum for primarily functionaldescriptions of languages or parts of languages — Functional Descriptions of Language.Relations between linguistics and computing are covered in the Communication in ArtificialIntelligence series, two series, Advances in Applied Linguistics and Communication in Public Life,publish books in applied linguistics and the series Modern Pragmatics in Theory and Practicepublishes both social and cognitive perspectives on the making of meaning in languageuse. We also publish a range of introductory textbooks on topics in linguistics, semioticsand deaf studies.

Recent titles in this seriesClassroom Discourse Analysis: A Functional Perspective, Frances ChristieConstruing Experience through Meaning: A Language-based Approach to Cognition,

M. A. K. Halliday and Christian M. I. M. MatthiessenCulturally Speaking: Managing Rapport through Talk across Cultures, Helen Spencer-Oatey (ed.)Educating Eve: The 'Language Instinct' Debate, Geoffrey SampsonEmpirical Linguistics, Geoffrey SampsonGenre and Institutions: Social Processes in the Workplace and School, Frances Christie and

J. R. Martin (eds)The Intonation Systems of English, Paul TenchLanguage Policy in Britain and France: The Processes of Policy, Dennis AgerLanguage Relations across Bering Strait: Reappraising the Archaeological and Linguistic Evidence,

Michael FortescueLearning through Language in Early Childhood, Clare PainterPedagogy and the Shaping of Consciousness: Linguistic and Social Processes, Frances Christie (ed.)Register Analysis: Theory and Practice, Mohsen Ghadessy (ed.)Relations and Functions within and around Language, Peter H. Fries, Michael Cummings,

David Lockwood and William Spruiell (eds)Researching Language in Schools and Communities: Functional Linguistic Perspectives,

Len Unsworth (ed.)Summary Justice: Judges Address Juries, Paul RobertshawSyntactic Analysis and Description: A Constructional Approach, David G. LockwoodThematic Developments in English Texts, Mohsen Ghadessy (ed.)Ways of Saying: Ways of Meaning. Selected Papers of Ruqaiya Hasan. Carmen Cloran, David

Butt and Geoffrey Williams (eds)Words, Meaning and Vocabulary: An Introduction to Modern English Lexicology, Howard Jackson

and Etienne Zé AmvelaWorking with Discourse: Meaning beyond the Clause, J. R. Martin and David Rose

Multimodal Discourse Analysis

Systemic-Functional Perspectives

Edited by Kay L. O'Halloran

continuumL O N D O N N E W Y O R K

ContinuumThe Tower Building 15 East 26th Street11 York Road New YorkLondon SE1 7NX NY 10010

© Kay L. O'Halloran 2004

All rights reserved. No part of this publication may be reproduced or transmittedin any form or by any means, electronic or mechanical, including photocopying,recording, or any information storage or retrieval system, without prior permissionin writing from the publishers.

British Library Cataloguing-in-Publication DataA catalogue record for this book is available from the British Library.

ISBN: 0-8264-7256-7

Library of Congress Cataloguing-in-Publication DataA catalogue record for this book is available from the Library of Congress.

Typeset by RefineCatch Limited, Bungay, SuffolkPrinted and bound in Great Britain by MPG Books Ltd, Bodmin, Cornwall

Contents

Introduction 1Kay L. O'Hallomn

Part IThree-dimensional material objects in space

1 Opera Ludentes: the Sydney Opera House at work and play 11Michael O'Toole

2 Making history in From Colony to Nation: a multimodal analysisof a museum exhibition in Singapore 28Alfred Pang Kah Meng

3 A semiotic study of Singapore's Orchard Road and MarriottHotel 55Safeyaton Alias

Part IIElectronic media and film

4 Phase and transition, type and instance: patterns in media textsas seen through a multimodal concordancer 83Anthony P. Baldry

Kay L. O'Halloran

6 Multisemiotic mediation in hypertext 131Arthur Kok Kum Chiew

Part IIIPrint media

7 The construal of Ideational meaning in print advertisements 163Cheong Tin Yuen

5 Visual semiosis in film 109

vi CONTENTS

8 Multimodality in a biology textbook 196Libo Guo

9 Developing an integrative multi-semiotic model 220Victor Lim Fei

Index 247

This book is dedicated to my mother, Janet O'Halloran

This page intentionally left blank

Introduction

Kay L. O'Halloran

Multi-modal Discourse Analysis is a collection of research papers in the field ofmultimodality. These papers are concerned with developing the theory andpractice of the analysis of discourse and sites which make use of multiplesemiotic resources; for example, language, visual images, space and archi-tecture. New social semiotic frameworks are presented for the analysis of arange of discourse genres in print media, dynamic and static electronicmedia and three-dimensional objects in space. The theoretical approachinforming these research efforts is Michael Halliday's (1994) systemic-functional theory of language which is extended to other semiotic resources.These frameworks, many of which are inspired by Michael O'Toole's (1994)approach in The Language of Displayed Art, are also used to investigate mean-ing arising from the integrated use of semiotic resources.

The research presented here represents the early stages in a shift of focusin linguistic enquiry where language use is no longer theorized as an isolatedphenomenon (see, for example, Baldry, 2000; Kress, 2003; Kress and vanLeeuwen, 1996, 2001; ledema, 2003; Ventola et al., forthcoming). Theanalysis and interpretation of language use is contextualized in conjunctionwith other semiotic resources which are simultaneously used for the con-struction of meaning. For example, in addition to linguistic choices and theirtypographical instantiation on the printed page,1 multimodal analysis takesinto account the functions and meaning of the visual images, together withthe meaning arising from the integrated use of the two semiotic resources.To date, the majority of research endeavours in linguistics have tended toconcentrate solely on language while ignoring, or at least downplaying, thecontributions of other meaning-making resources. This has resulted inrather an impoverished view of functions and meaning of discourse.Language studies are thus undergoing a major shift to account fully formeaning-making practices as evidenced by recent research in multimodality(for example, Baldry, 2000; Callaghan and McDonald, 2002; ledema, 2001;Jewitt, 2002; Martin, forthcoming; Kress, 2000, 2003; Kress et al., 2001:Kress and van Leeuwen, 1996, 2001; Lemke, 1998, 2002, 2003; O'Halloran,1999a, 2000, 2003a, 2003b; Royce, 2002; Thibault, 2000; Unsworth, 2001;Ventola et al., forthcoming; Zammit and Callow, 1998).

Multimodal Discourse Analysis contains an invited paper by Michael

2 INTRODUCTION

O'Toole, a founding scholar in the extension of systemic-functional theoryto semiotic resources other than language. The collection also features aninvited contribution from Anthony Baldry, a forerunner in the use of inform-ation technology for the development of multimodal theory and practice.The remaining seven research papers have been completed by KayO'Halloran and her postgraduate students in the Semiotics Research Group(SRG) in the Department of English Language and Literature at theNational University of Singapore. The SRG has been actively involved inresearch in systemic-functional approaches to multimodality over theperiod 1999-2003.

The papers are organized into sections according to the medium of thediscourse: Part I which is concerned with three-dimensional material objectsin space, Part II which deals with electronic media and film and Part IIIwhich contains investigations into print media. The theoretical advancespresented in this volume are illustrated through the analysis of a range ofmultimodal discourses and sites, some of which are Singaporean. Thesecontributions represent a critical yet sensitive interpretation of everydaydiscourses in Singapore. Thus, like all discourse, they are grounded in localknowledge, but due to the universality of the semiotic model being used,they are applicable to similar texts in any culture. A brief synopsis of eachpaper in this collection is given below.

In Michael O'Toole's opening paper in Part I, 'Opera Ludentes: theSydney Opera House at work and play', a systemic-functional analysis ofarchitecture (O'Toole, 1990, 1994) is used to consider in turn the Experien-tial, Interpersonal and Textual functions ofJ0rn Utzon's (1957-73) SydneyOpera House and its parts, both internally and in relation to its physical andsocial context. In this paper, the usual definition of 'functionalism' in archi-tecture is significantly extended. Like language, the building embodies anExperiential function: its practical purposes, the 'lexical content' of its com-ponents (theatre, stage, seats, lights, and so forth) and the relations of whodoes what to whom, and when and where. It also embodies a 'stance' vis-a-vis the viewer and user (its facade, height, transparency, resemblance toother buildings or objects) which also reflects the power relations betweengroups of users. That is, it embodies an Interpersonal function like lan-guage. The Sydney Opera House also embodies a Textual function: its partsconnect with each other and combine to make a coherent 'text', and itrelates meaningfully to its surrounding context of streets, quays, harbour,nearby buildings and cityscape, and by 'meaningful' here we include delib-erate dramatic contrast as well as harmonious blending in. In the analysis,certain features are discovered to be multifunctional, marking 'hot spots' ofmeaning in the total building complex. In terms of all three functions, theOpera House emerges as a playful building: Opera Ludentes. Utzon's build-ing started its life as a focus of architectural and political controversy andmost discourses about the building are still preoccupied with the politics ofits conception, competition, controversies and completion by different archi-tects. A semiotic rereading of the building can relate its structure and design

INTRODUCTION 3

to the 'social semiotic' of both Sydney in the 1960s and to the internationalcommunity of its users today.

The museum is located as the next site for semiotic study in Alfred Pang's'Making history in From Colony to Nation: a multimodal analysis of a museumexhibition in Singapore'. Pang discusses how systemic-functional theory isproductive in fashioning an interpretative framework that facilitates a multi-modal analysis of a museum exhibition. The usefulness of this frameworkis exemplified in the critical analyses of particular displays in From Colony toNation, an exhibition at the Singapore History Museum (SHM) that displaysSingapore's political constitutional history. From this analysis, Pang explainshow the museum as a discursive site powerfully constitutes and maintainsparticular social structures through the primary composite medium of anexhibition. Of interest is the relationship between the museum, nation andhistory and how the multimodal representation of history in From Colony toNation ideologically positions the visitor to a particular style of imagining a'nation' (Anderson, 1991).

Safeyaton Alias investigates the semiotic makeup of the city in 'A semioticstudy of Singapore's Orchard Road and Marriott Hotel'. Like a written text,the city stores information and 'presents particular transformations andembeddings of a culture's knowledge of itself and of the world' (Preziosi,1984: 50-51). In this paper, a rank-scale framework for the functions andsystems in the three-dimensional multi-semiotic city is proposed. The focus inthis paper, however, is the analysis of the built forms of Orchard Road andthe Marriott Hotel. Safeyaton discusses how these built forms transmit mes-sages which are articulated through choices in a range of metafunctionallybased systems. This paper discusses the intertextuality and the discourses thatconstruct Singapore as a city that survives on consumerism and capitalism.

In Part II on electronic media and film, Anthony Baldry's opening paper,'Phase and transition, type and instance: patterns in media texts as seenthrough a multimodal concordancer', explores the use of computer tech-nology for capturing 'the slippery eel-like' (to quote Baldry) dynamics ofsemiosis. Baldry demonstrates that the online multimodal concordancer, theMultimodal Corpus Authoring (MCA) system, provides new possibilities forthe analysis and comparison of film and videotexts. This type of concord-ancing transcends in vitro approaches by preserving the dynamic text, insofaras this is ever possible, in its original form. The relational properties of themultimodal concordancer also allow a researcher to embark on a quest forpatterns and types. Taking the crucial semiotic units of phase and transitionas its starting point, Baldry shows that, when examining the semiotic andstructural units that make up a video, a multimodal concordancer far out-strips multimodal transcription in the quest for typical patterns.

Kay O'Halloran further explores the use of computer technology forthe semiotic analysis of dynamic images in 'Visual semiosis in film'. A sys-temic-functional model which incorporates the visual imagery and thesoundtrack for the analysis of film is introduced. Inspired by O'Toole's(1999) representation of systemic choices in paintings in the interactive

4 INTRODUCTION

CD-ROM Engaging with Art., O'Halloran uses video-editing software AdobePremiere 6.0 to discuss the analysis of the temporal unfolding of semioticchoices in the visual images for two short extracts from Roman Polanski's(1974) film Chinatown. While film narrative involves staged and directedbehaviour to achieve particular effects, the analysis of film is at least a firststep to understanding semiosis in everyday life. The analysis demonstratesthe difficulty of capturing and interpreting the complexity of dynamicsemiotic activity.

Attention turns to hypertext in Arthur Kok's 'Multisemiotic mediation inhypertext'. In this paper, Kok explores how hypertext (re)presents reality andengages the user, and how instantiations of different semiotic resources arearranged and co-deployed for this purpose. This paper formulates a workingdefinition and a theoretical model of hypertext which contains differentorders of abstraction. As with many papers in this collection, the semioticanalysis is employed through extending previously developed systemic-functional frameworks (Halliday, 1994; Kress and van Leeuwen, 1996;O'Toole, 1994). Via an examination of the semiotic choices made inSingapore's Ministry of Education (MOE) homepage, this analysis seeks tounderstand how the objectives of an institution become translated, trans-mitted and received through the hypertext medium. In the process, anaccount of the highly elusive process of intersemiosis, the interaction ofmeanings across different semiotic instantiations, is given.

In Part III on print media, in the first paper, 'The construal of ideationalmeaning in print advertisements', Cheong Yin Yuen proposes a genericstructure potential for print advertisements which incorporates visual andverbal components. Cheong also investigates lexicogrammatical strategiesfor the expansion of ideational meaning which occur through the inter-action of the linguistic text and visual images. Through the analysis of fiveadvertisements, Cheong develops a new vocabulary to discuss the strategieswhich account for semantic expansions of ideational meaning in these texts;namely, the Bi-directional Investment of Meaning, Contextual Propensity,Interpretative Space, Semantic Effervescence and Visual Metaphor.

Moving to the field of education, Guo Libo investigates the multi-semioticnature of introductory biology textbooks in 'Multimodality in a biologytextbook'. These books invariably contain words and visual images: forexample, diagrams, photographs, and mathematical and statistical graphs.Drawing upon the work of sociological studies of biology texts and followingO'Toole (1994), Lemke (1998) and O'Halloran (1999b), this paper proposessocial semiotic frameworks for the analysis of schematic drawings and math-ematical or statistical graphs in biology. The frameworks are used to analysehow the various semiotic resources interact with each other to make meaningin selected pages from the biology textbook Essential Cell Biology (Alberts et al.,1998). The article concludes by reiterating Johns's (1998: 194) claim that inteaching English for Academic Purposes to science and engineering stu-dents, due attention must be given to the visual as well as the linguisticmeaning in what is termed Visual/Textual interactivity' (ibid.: 186).

INTRODUCTION 5

Lastly, in order to further theorize the meaning made in texts containinglanguage and visual images, Victor Lim proposes a meta-model in 'Develop-ing an integrative multi-semiotic model'. This model allows for an integra-tive approach to the interpretation of texts where the simultaneousco-deployment of choices from various systems contextualize each other ateach instance of the meaning-making process. It takes into account theindependent meanings made by each semiotic resource and, further to this,theorizes a space of interaction and integration where inter-semiotic pro-cesses for the expansion of meaning (for example, 'homospatiality' and'semiotic metaphor') take place. The model also accounts for systems ofTypography and Graphics that operate on the Expression plane. Building onthe pioneering work done in this field (for example, Baldry, 2000; Baldry andThibault, forthcoming; Lemke, 1998; O'Halloran, 1999a; Thibault, 2000),as with each paper in this collection, the model is conceived in the traditionof the systemic-functional theory.

Michael Halliday has always been ready to extend and enrich his lin-guistic theory when particular types of text demanded it. The contributorsto this volume may be seen to be attempting to extend productively thesecategories for multimodal analysis.

Note

Regrettably it has not been possible to reproduce coloured plates in thispublication. However, as will become evident in what follows, the contribu-tors in this volume recognize that colour is a significant resource for mean-ing (see also Kress and van Leeuwen, 2002). While the papers have beensomewhat comprised by the black and white reproductions, every possibleeffort has been made to ensure that the analysis refers to the original colourof the texts.

References

Alberts, B., Bray, D., Johnson, A., Lewis, J., Raff, M., Roberts, K. and Walter,P. (1998) Essential Cell Biology: An Introduction to the Molecular Biology of the Cell.New York: Garland.

Anderson, B. (1991) Imagined Communities: Reflections on the Origin and Spread of National-ism (revised edn). London: Verso.

Baldry, A. P. (ed.) (2000) Multimodality and Multimediality in the Distance Learning Age.Gampobasso, Italy: Palladino Editore.

Baldry, A. P. and Thibault, P. (forthcoming) Multimodal Transcription and Text.London: Equinox.

Gallaghan, J. and McDonald, E. (2002) Expression, content and meaning in lan-guage and music: an integrated semiotic analysis. In P. McKevitt, S. O'Nuallainand C. Mulvihill (eds), Language, Vision and Music. Selected papers from the 8th Inter-national Workshop on the Cognitive Science of Natural Language Processing, Galway, Ireland,1999. Advances in Consciousness Research, Volume 35. Amsterdam: Benjamins,205-220.

6 INTRODUCTION

Halliday, M. A. K. (1994) An Introduction to Functional Grammar (2nd edn). London:Edward Arnold.

ledema, R. (2001) Analysing film and television: a social semiotic account of hos-pital: an unhealthy business. In T. van. Leeuwen and C. Jewitt (eds), Handbook ofVisual Analysis. London: Sage, 183—204.

ledema, R. (2003) Multimodality, resemioticization: extending the analysis of dis-course as a multi-semiotic practice. Visual Communication 2(1): 29—57.

Jewitt, C. (2002) The move from page to screen: the multimodal reshaping of schoolEnglish. Visual Communication 1(2): 171—195.

Johns, A. (1998) The visual and the verbal: a case study in macroeconomics. Englishfor Specific Purposes 17(2): 183-197.

Kress, G. (2000) Multimodality. In B. Cope and M. Kalantzis (eds), Multiliteracies:Literacy Learning and the Design of Social Futures. London: Routledge, 182—202.

Kress, G. (2003) Literacy in the New Media Age. London: Routledge.Kress, G, Jewitt, G., Ogborn, J. and Tsatsarelis, C. (2001) Multimodal Teaching and

Learning: The Rhetorics of the Science Classroom. London: Continuum.Kress, G. and van Leeuwen, T. (1996) Reading Images: The Grammar of Visual Design.

London: Routledge.Kress, G. and van Leeuwen, T. (2001) Multimodal Discourse: The Modes and Media of

Contemporary Communication. London: Arnold.Kress, G. and van Leeuwen, T. (2002) Colour as a semiotic mode: notes for a

grammar of colour. Visual Communication 1(3): 343-368.Lemke, J. L. (1998) Multiplying meaning: visual and verbal semiotics in scientific

text. InJ. R. Martin and R. Veel (eds), Reading Science: Critical and Functional Perspec-tives on Discourses of Science. London: Routledge, 87—113.

Lemke, J. L. (2002) Travels in hypermodality. Visual Communication 1(3): 299—325.Lemke, J. L. (2003) Mathematics in the middle: measure, picture, gesture, sign and

word. In M. Anderson, A. Saenz-Ludlow, S. Zellweger and V Cifarelli (eds),Educational Perspectives on Mathematics as Semiosis: From Thinking to Interpreting to Know-ing. Ottawa: Legas Publishing, 215-234.

Martin, J. R. (forthcoming) Sense and sensibility: texturing evaluation. InJ. Foley(ed.), Mew Perspectives on Education and Discourse. London: Continuum.

O'Halloran, K. L. (1999a) Interdependence, interaction and metaphor in multi-semiotic texts. Social Semiotics 9(3): 317—354.

O'Halloran, K. L. (1999b) Towards a systemic-functional analysis of multi-semioticmathematics texts. Semiotica (124-1/2): 1-29.

O'Halloran, K. L. (2000) Classroom discourse in mathematics: a multi-semioticanalysis. Linguistics and Education 10(3): 359—388.

O'Halloran, K. L. (2003a) Educational implications of mathematics as a multi-semiotic discourse. In M. Anderson, A. Saenz-Ludlow, S. Zellweger, and V VCifarelli (eds), Educational Perspectives on Mathematics as Semiosis: From Thinking toInterpreting to Knowing. Ottawa: Legas Publishing, 185-214

O'Halloran, K. L. (2003b) Intersemiosis in mathematics and science: grammaticalmetaphor and semiotic metaphor. In A.-M. Simon-Vandenbergen, M. Taverni-ers, and L. Ravelli (eds), Grammatical Metaphor: Views from Systemic Functional Lin-guistics. Amsterdam: John Benjamins, 337—365.

O'Toole, M. (1990) A systemic-functional semiotics of art. Semiotica (82—3/4):185-209.

O'Toole, M. (1994) The Language of Displayed Art. London: Leicester University Press.O'Toole, M. (1999) Engaging with Art [CD-ROM]. Perth: Murdoch University.

INTRODUCTION 7

Preziosi, D. (1984) Relations between environmental and linguistic structure. InR. P. Fawcett, M. A. K. Halliday, S. M. Lamb and A. Makkai (eds), The Semiotics ofCulture and Language Volume 2. Language and Other Semiotic Systems of Culture. Dover,NH: Frances Pinter, 47-67.

Royce, T. (2002) Multimodality in the TESOL classroom: exploring visual—verbalsynergy. TESOL Quarterly 36(2): 191-205.

Thibault, P. J. (2000) The multimodal transcription of a television advertisement:theory and practice. In A. P. Baldry (ed.), Multimodality and Multimediality in theDistance Learning Age. Gampobasso, Italy: Palladino Editore, 311—385.

Unsworth, L. (2001) Teaching Multiliteracies across the Curriculum: Changing Contexts ofText and Image in Classroom Practice. Buckingham, UK: Open University Press.

Ventola, E., Charles, C. and Kaltenbacher, M. (eds) (forthcoming) Perspectives onMultimodality. Amsterdam: John Benjamins.

Zammit, K. and Callow, J. (1998) Ideology and technology: visual and textualanalysis of two popular CD-ROM programs. Linguistics and Education 10(1):89-105.

Acknowledgements

The research presented here is only made possible through the foundationalwork of Michael Halliday and Michael O'Toole. I am also indebted to JayLemke for originally pointing me in this direction many years ago, and forhis continued support since that time. I also thank Joe Foley, Eija Ventola,Frances Christie and Anthony Baldry for their friendship, advice and activesupport over the years.

My special thanks also to Michael O'Toole for his invaluable reading ofthe first draft of the manuscript. His comments, corrections and suggestionshave contributed to the final form of this volume, although of course anyerrors of interpretation are mine. I am also most grateful to Guo Libo for hiscareful proof-reading and corrections to the manuscript.

My sincere thanks to my talented group of postgraduate research stu-dents for their enthusiasm, dedication and commitment to push the bound-aries of multimodal analysis. This volume would not be possible withouttheir contributions. And special thanks to my past and present colleagues inthe Department of English Language and Literature at the National Uni-versity of Singapore (NUS), especially Linda Thompson, Chris Stroud, EdMcDonald and Desmond Allison for their continued friendship andsupport.

I would also like to thank Anne Pakir and the Faculty Research Commit-tee (FRC) in the Faculty of Arts and Social Sciences at NUS for providingthe research grant (R-103-000-014-107/112) in 2000 to establish the Labora-tory for Research in Semiotics (LRS) in the Department of English Languageand Literature. The research grant has directly supported the researchpresented in this publication.

PartiThree-dimensional material objects in space

1 Opera Ludentes: the Sydney Opera House at workand play

Michael O'TooleMurdoch University, Western Australia

Here the trick was to get people up. When you go up the steps you see nobuildings. You see the sky and you get separated from being between houses. Ilike procession very much: sky — foyer — windows — sea. It takes you to anotherworld. That's what you want for an audience: to separate themselves from theirdaily life.

(J0rnUtzon, 1998)1

Clearly, for the architect of Sydney Opera House (Plate 1.1) 'Interpersonal'meanings are very important: the building's height and orientation to itsvisitors; the play of vistas as one approaches the entrance; the stress onarchitecture as theatre; constructing an audience; a working building at play.

In a systemic-functional semiotic model of architecture2 (O'Toole, 1994;Table 1.1) these kinds of meaning are analogous to the Interpersonalsemantic functions in language: Mood constructing the roles to be played ina verbal interaction; Modality constructing a hinge between the real and thehypothetical; Attitudinal Modifiers and Intensifiers expressing the speaker'sposition and influencing the response of the hearer.

If you look out here [at Utzon's home in Helebek, Denmark], you see a fieldwith flowers and a small bush and small trees and big trees. They all consist ofsmall elements. And if you take them up and put them on the table it's anumber of elements. Together they make this. In architecture you have a floor,your walls, you have windows, doors, and you have a lot of materials. And youselect them. You must have in mind that they make a whole or an expression ofsome kind.

(J0rnUtzon, 1998)3

Here Utzon's focus is on 'Textual' meanings: the way distinct architecturalcomponents are combined to make a coherent whole, that is to say, animportant dimension of the meaning ('an expression of some kind') is in thecomposition.4

As in language, the Collocational potential of architectural elements - theirConjunction in rooms and floors and buildings, their Reference to each otherand to their environment - is what makes them into coherent and usable'texts'.

Table 1.1 Functions and systems in architecture (reproduced from O'Toole 1994: 86)

Units/ ExperientialFunctions

Interpersonal Texture

Building

Floor

Practical function: Public/Private;Industrial/Commercial/Agricultural/Governmental/Educational/Medical/

Size Orientation to neighboursVerticality Orientation to roadChthonicity Orientation to entrant

Cultural/Religious/Residential; Domestic/ Fagade IntertextualityUtility

Orientation to lightOrientation to windOrientation to earthOrientation to service (water/sewage/power)

Gladding referenceColour mimicryModernity contrastExoticism

Relation to cityRelation to roadRelation to adjacent buildingsProportionsRhythms: contrasting shapes,anglesTextures: rough/smoothRoof/ wall relationReflectivityOpacity

Sub-functions: Access:WorkingSellingAdministrationStoringWakingSleepingParking

Height Sites of powerSpaciousness Separation of groupsAccessibilityOpenness of vistaViewHard/ soft textureColour

Relation to other floorsRelation to outer worldRelation to connectors; stairs/liftescalator (external cohesion)Relation of landing/corridor/foyer/room (internal cohesion)Degree of partitionPermanence of partition

Units/ ExperientialFunctions

Interpersonal Texture

Room Specific functions:AccessEntryLiving roomFamily roomKitchenBathroomBedroom

StudyToiletLaundryGamesroomRetreat

FoyerRestaurantKitchenBarBedroomEnsuiteServery

Comfort LightingModernity SoundOpulence WelcomeStyle: rustic, pioneer, colonial, suburban

'Dallas', working class, tenement,slum

Foregrounding of function

ScaleLightingSoundRelation to outsideRelation to other roomsConnectors: doors/windows/hatches/intercomFocus (e.g. hearth, dais, altar, desk)

Element Light: window, lamp, curtains,Air: window, fan, conditionerHeating: central, fire, stoveSound: carpet, rugs,partitions acoustic,treatment

functionSeating <{ table '

I comfort

blinds

"diningcoffeeoccasional

deskcomputerdrawing

RelevanceFunctionality: convention/surpriseTexture: rough/smoothNewnessDecorativeness'Stance'Stylistic coherenceProjection (e.g. TV)

TexturePositioning: to light/heat/otherelementsFinish

Plate 1.1 The Sydney Opera House as procession

THREE-DIMENSIONAL MATERIAL OBJECTS IN SPACE 15

It's a curious fact that in all the drama of constructing the building, not muchdetailed thought had gone into its specific uses. The competition entrants hadbeen asked to provide large and small halls, the larger to accommodate orchestralconcerts and opera as its chief forms of entertainment. At this point, seven yearsafter construction began, the Australian Broadcasting Commission decided that amulti-purpose venue wouldn't be good enough as the permanent home of theSydney Symphony Orchestra.

It was a difficult situation. To argue stubbornly in favour of the original multi-purpose concept for the major auditorium would mean accepting compromiseson both sides in terms of stage requirements and acoustics: orchestral musicversus opera. There were also practical considerations involved, such as a reduc-tion in the seating capacity for concerts, and the logistics of sharing the hall.Should it be devoted to performances of opera and ballet alone?

(Sykes, 1993: 45)

A great deal of the political controversy surrounding the design and con-struction of the Opera House focused on the 'Experiential' use-functions ofthe building and the competing claims of its corporate users. The brief forany commissioned architect or entrant to an architectural competitionnecessarily starts from the uses proposed for the building.

Like a clause in language, a building incorporates Types of Process andtheir Participants; its specific functions are Modified in terms of material,size, colour and texture; and its component elements are organized taxo-nomically like lexical items in the vocabulary of our language.

We clearly need to take account of the Experiential function of archi-tecture. Otherwise, our roof will leak, our rooms will be full of draughts, ourcupboards and desk will face the wall, and we will find ourselves cooking orworshipping or taking baths in the bedroom. But the obsession with Tunc-tionalism' in architecture by both its modernist proponents and its Post-modernist critics has taken it for granted either that the Experiential functionis the only function and that the design and evaluation of a building standsor falls by this criterion alone, or that the form of the building primarilyexpresses its practical use, which confuses functions, or modes of meaning,which should be kept distinct. A systemic-functional approach corrects suchblinkered approaches by proposing that there are three functions creatingmeaning in all buildings: an Experiential, an Interpersonal and a Textualfunction, and that these are all equally valid and equally necessary for abuilding to be meaningful and socially usable.

J0rn Utzon was probably naive in the early phase of designing and con-structing the Opera House in that his revolutionary designs foregroundedthe public image (Interpersonal) and sculptural coherence (Textual function)of the building, leaving many features of its use (Experiential) insufficientlyresolved. Given the political partisanship, the conflicting client requirementsand the media hype surrounding his design from the outset, this bias isunderstandable, but it meant that his successors had to focus in the firstinstance on the Experiential function:

16 MULTIMODAL DISCOURSE ANALYSIS

When Utzon resigned in 1966, the construction of the roof and its tile claddingwas well under way. But plans for Stage III were scarcely defined, and theyinvolved the elements which would turn the building from a magnificent sculp-ture to a working centre for the performing arts: the walls that would enclose theroof area, the performing venues within it, the stage equipment and the furnish-ing of foyer, backstage and administrative areas throughout.

The newly appointed triumvirate of architects (Peter Hall, Lionel Todd andDavid Littlemore) declared their intention to complete the building as closely aspossible to Utzon's intentions. But in the drawings that Utzon left behind, therewere no precise dimensions worked out for what would be more than a thousandrooms within the structure. [. . .]

The key to finalising the internal designs was to establish what their userswanted. Incredibly, in the Alice in Wonderland development of the construction,there had been no formal compilation of user organisations' expectationsin terms of performance characteristics and capacities, dressing room andrehearsal area backup, box office, administration, air conditioning and cateringrequirements.

(Sykes, 1993: 61-62)

As our chart of functions and systems in architecture (Table 1.1) shows, alarge number of Experiential functions are involved in a building complexlike the Sydney Opera House. Practical orientations to light (the sun, reflec-tions off the harbour), to wind (prevailing winds, strength of the highestpossible wind gusts), to the earth (the building up of Bennelong Point toform the massive podium, its projection out into the harbour), and theprovision of services such as water, sewage, power, scenery and food deliv-ery, car-parking, waste disposal, etc. had already been accounted for eitherby Utzon and his team of architects or by the consultant engineers, OveArup and Partners. But each functioning part requires separate specifica-tions: the concert hall with its open plan and relatively fixed fittings asopposed to the opera theatre with its proscenium arch and constantly chan-ging scenery, its stage tower, backstage, stage and auditorium; the dramatheatre (originally designed as a smaller experimental theatre) as opposed tothe playhouse (originally designed as a 'music room' for solo recitals andchamber music) or the Broadwalk Studio (originally conceived purely as arecording hall); the Bennelong Restaurant, serving high-quality inter-national cuisine for leisurely eating under its own miniature shell roof, asopposed to the more informal forecourt restaurant, the Cafe Mozart, theperformers' cafeteria, or the ad hoc catering arrangements in the foyers.

As I discovered in analyzing a church and even a suburban display homein The Language of Displayed Art (O'Toole, 1994), the rank of Floor on thechart may not always be valid as such. And yet even in the complex struc-tures of the Opera House, particular spaces below the rank of Building butabove the rank of Room, as likely to be separated horizontally as vertically,still need to be accounted for experientially. To the list of sub-functions listedat floor rank on the chart we could add Rehearsing, Recording, Cooking,


Eating, Scenery Construction, Maintenance, Security, and so on. And eventhe multifunctional (in the old sense) outdoor areas of the forecourt, broad-walk, arcade and steps involve specific but varying activities (Process-Participant relations).

The point is that a systemic-functional semiotics takes a rank-scale as oneof its starting-points, differentiating the options available at lower ranks inrelation to those available at higher ranks. In the Experiential function this ispartly a matter of common sense (i.e. the shape of a drinks bar or a boxoffice at Room rank requires different decisions from either the types ofElement (desk, chairs, equipment) with which they will be furnished or theshape, illumination, ventilation and accessibility or enclosability ofthe larger foyer (Floor) spaces in which they are to be found). It is also amatter of different design specialists, with whole firms and even industriesbeing responsible for particular Experiential sub-functions (cooking, drinks-serving, ticketing, public relations, etc.).

The heuristic value of the rank-scale becomes more obvious when werelate these Experiential distinctions to Interpersonal and Textual distinc-tions at the same ranks. To illustrate how architectural meanings are madethrough all three functions I want to start with one of the smallest, mostnumerous and most visible elements of the whole structure, the 1,056,000roof tiles.

Experientially, the roof covering had to be weatherproof to all climaticconditions and had to be self-cleaning, but curved roofs can be sprayed orsheeted in copper or bronze. As the architect, Harry Seidler relates:

I asked him in his office, 'Why do you want to cover a building like that with tiles?A curved surface, it could be sprayed.' And he looked surprised and said, 'Buttiles are the best.' And he'd looked all over the world at them, and he'd seen themin the Middle East and elsewhere, mosques covered in gleaming tiles. And he'dbeen to Japan and China, and he was very concerned with the quality that madethem up: what material they used, where they got the clay from and what mixesthey used in the clay, till it eventually satisfied him that it gave a slightly roughsurface. And this was the natural colour, the white, and over that surface was avery clear glaze, a very shiny glaze.5

The material quality and the rough surface, the texture of the built surfaceare primarily Interpersonal considerations. Like the shine and the gleamthey are part of the impact the Opera House shells have on spectators. Andthe intertextual references to mosques and Oriental architecture, visual simi-larities which may jog our cultural memory, are Interpersonal issues. Theimpact on the spectator is crucial to Utzon. For him his Opera House isalmost more than a sculpture; it has a human personality:

It tells a story, it's not a calm building, it's awake all the time. You cannot make asculpture better than something that's white or off-white. If you look at bronzesculptures in nature, they're difficult to read. If you had put a copper roof on thishouse, you wouldn't have benefitted from the light. You would have seen a green


marvellous colour. So this was my first and only idea for the roof. And Saarinensaid to me, 'Keep it white. Sydney harbour is dark.' And at that time the buildingswere dark. So it's the right answer.6

The older Finnish architect is as alert as his Danish colleague to the effect onthe spectator of the chiaroscuro of a white building against a dark groundand the quality of light in a city on the water like Sydney, Helsinki orCopenhagen.

Of course, the Interpersonal function at the rank of Element is not con-fined to the roof tiles. The concrete ribs of the shells have a primaryExperiential function of binding and supporting the roof, but as soon as onesteps inside, one becomes aware of the contrast between the raw, matt andunpatterned grey concrete of the ribs and the warm brown satin grain ofinternal balustrades and doors. In terms of its textures, the building (apartfrom the tiled surfaces) seems to start as rough, raw, grey and abrasive in itsouter layers and become progressively more smooth, polished, colourful andcomforting as we move to the core of the personal artistic experience in ourseat in any of the auditorial it speaks to us Interpersonally through its shine,colours, textures, the very warmth or coldness of the materials used. Thisplay of material qualities has even more impact on the spectator at thosepoints outside the building where the shells meet the metal struts and sheetsof glass of the windows in an exciting geometry of tiles, raw concrete, metaland glass (Plate 1.2). As we shall see, this involves an important interplay ofthe Interpersonal and Textual functions.

Interpersonal relevance is obviously a key criterion inside a theatricalbuilding. Audience seats and lighting and sound booths face performers'spaces; conductor's rostra face orchestras; prompt boxes face actors; bar-tenders face customers across bars, counters and tables (as Ervin Goflftnanshowed in the 1950s7 - and Fawlty Towers hyperbolized in the 1960s - res-taurants and hotels are highly dramatistic spaces). The public relationsmechanisms of display boards, information desks, ticket offices, mediainterview spaces and Opera House guide routes all have their structure asmini-theatres. And where 'projection' in the home may be confined to oneor two TV sets, in theatres it covers the gamut of possibilities from staging,rostra, lighting, sound projection, security video and telephones (fixed andmobile) and even the projection of performances to overflow audiences onclosed-circuit television. In all these aspects of a theatre or concert hall youmight say that the Interpersonal is Experiential - but we will argue that thereis still real heuristic value in keeping them separate.

Less obviously 'theatrical' choices at Room rank are involved in the Inter-personal systems of Comfort, Modernity, Opulence and Style. Patrons ofconcerts and operas are enveloped in a cocoon of almost perfect acousticsand seated on luxuriously upholstered seats (Plate 1.3). These seats inmoulded birch ply and contrasting scarlet upholstery carry a message ofScandinavian 'functionalism' of the 1960s and 1970s: like so much of thearchitecture here, they put their working functions on display. The steel


Plate 1.2 Texture and geometry

cable tensioning of the concrete columns, the moulded curves of internalbeams and columns, the glass curtain walls, even the acoustic baffles andplexiglass 'doughnuts' which hang in a ring from the roof of the ConcertHall 'show the works' - though less stridently than Richard Rogers's Pompi-dou Centre in Paris and its imitators. The stress here comes from a humanist'craft' tradition of high-quality but 'natural' materials (wool, varnished ply,grained parquet, shuttered concrete) with a modest unassertive finish.

The Interpersonal meaning of many types of building is carried by theplacing and styling of 'sites of power', that is, a building expresses thepolitical relations between its various users. A building primarily dedicatedto classical musical performance incorporates the power of the conductor'srostrum over the orchestra and the power of both over the audience. Hid-den control booths and 'Private' administration rooms mystify this powerfurther. The stage and orchestra in the Opera House and other theatricalspaces carry the same power relations.

We have pinpointed many of the systems realizing the Interpersonal func-tion at the ranks of Floor, Room and Element, but with the Sydney OperaHouse this function begins and ends at the rank of the whole buildingcomplex. Our very opening quotation of Utzon's own words shows thearchitect's concern with imposing Size and Verticality and Orientation to the

Plate 1.3 The Concert Hall


entrant (systems in the top central box of the Chart): the viewer is induced tolook up, beyond the steps, beyond the shells to the sky and to imaginethemselves into another world of the imagination, even before the officialperformance starts. Chthonicity is a particularly interesting system in thiscase, because the Opera House deliberately plays with conflicting options:on the one hand, there seem to be no solid walls embedded in the base. Theshells rear up skywards (anti-chthonically, away from the earth), to such adegree that their corners hardly seem to touch their footings, seeming tobalance on pinpoints. The smooth spherical curves induce a touch of ver-tigo and it is no wonder that so many of the photographs of the OperaHouse, whether by official agencies or casual tourists, accentuate theupward thrust of the shells. On the other hand, the podium is highlychthonic: it has turned Bennelong Point into a rock-like headland and, as weknow, incorporates many of the key functions of the working building. Thelight, dynamic, mobile and poetic structures above are embedded in thesolid and prosaic podium.

A building's orientation to its neighbours and the road by which it isapproached are important aspects of its Interpersonal function. Utzon andSaarinen were keen for the white curves of the sails to stand out against thepredominantly dark water of the harbour and the high-rise buildings ofSydney's rigidly rectangular central business district at that time. (Since1973 more of the neighbouring buildings have been constructed in lighterconcrete, marble or glass - perhaps in deference to Utzon's building as wellas in harmony with changing architectural fashions.) The multiple curves,however, offer visual echoes of Sydney Harbour Bridge (Plate 1.4), CircularQuay and the bays and headlands of the harbour. Of course, good archi-tectural as well as human relations can be spoiled when bad neighboursmove in. The Opera House's visual relationship with Circular Quay hasbeen obstructed and, more importantly, the easy natural pedestrian routefrom the ferry terminals to the entrance steps has been interrupted by therectangular complex of shops and apartments erected in 1997-8,unpopularly known as 'the East Circular Quay toaster'.

The final heading in the Interpersonal box at the rank of Building on thechart is 'Intertextuality'. This was a term coined by Mikhail Bakhtin, theRussian literary theorist and philosopher, to account for the deliberate refer-ences, allusions or echoes that a writer makes to other widely known texts.As with language texts, this would seem to carry primarily an Interpersonalfunction in architecture: the writer/architect is saying to the viewer 'Nudge-nudge . . . look at my clever reference here to Stonehenge, or Palladianvillas, or St Peter's in Rome, or the Pompidou Centre in Paris . . . It is up toyou to enrich the meaning further here by your knowledge of that building,its uses, its tradition, its local cultural significance, etc'. And to some extentwe as viewers interpret the allusion according to our range of references andour cultural preoccupations at the time. Virtually everyone seeing the OperaHouse sees the visual metaphor of sails; many see sharks' jaws or clamshells; Barry Humphries saw a drowning nun. Utzon claims that the curves

Plate 1.4 Visual echoes


of the sails were inspired by the segments of an orange; the relation betweenthe outer shell and the inner roof of the auditoria - by the snug fit betweenshell and kernel in a walnut; the structural relations between the construc-tion units and the whole building — variously by the leaves on a palm tree orby Meccano toy construction sets. But in terms of other built texts, we haveUtzon's word for it that he had in mind a relationship between water andbuilt forms at Kronborg Castle, Helsingor; the soaring vaults of Gothiccathedrals; and the shining segmentation of tiles on a mosque.

The tiles bring us at last to the Textual function (which does not have to bethe last function examined: the three functions are all equally meaningfuland may be considered in any order). At the lowest rank of Element thefinish of the tiles and the chevron patterning create the surface texture ofthe Opera House shells. This is texture as such - Textual meaning - asopposed to their practical (Experiential) function of keeping out the rain andtheir decorative or dramatic (Interpersonal) functions.

At the rank of Room, each auditorium, or foyer, or office, or restauranthas its own scale and proportions, it is lit or in shadow, and has its ownacoustic properties in contrast to other spaces around it. Its relation tooutside carries Textual meaning, so that our response to the isolated andinsulated worlds of the concert hall, opera theatre, drama theatre or cinemais quite different from how we feel in the foyers, where our gaze is delib-erately projected out to the harbour and city views - where we are no longerfully enclosed in the built text. At this rank we experience a Textual focus aswell as the power relation (Interpersonal) between the rostrum and theorchestra and the audience. This is facilitated by aisles and stairways withinthe auditoria, and all such 'connectors' as corridors, stairs, lifts, escalators,hatches and interconnecting windows throughout the building are primarilyTextual in function: they work like the cohesive devices of conjunction inlanguage.

Like cohesive devices in language, these connectors work across severalranks, since they also work to relate floors and the various auditoria andother internal spaces to each other. Doors and windows, of course, relate theinternal spaces to exterior parts of the built text: walkways, entrance stepsand terraces, and thence to the Broadwalk and approach road.

The most striking Textual systems of the Opera House at the rank ofBuilding are listed in the top right-hand box of the Chart. We will considerthem from the bottom up - as if we were moving from near the building tovantage points further away. Opacity/Reflectivity/Transparency is a systemof options that tends to have meaning when we are near a building. Theshells of the Opera House are opaque, but, being shiny and white or off-white, reflect the light, whereas the podium is opaque and comparativelymatt, giving a denser, less light-responsive texture. The windows, of demi-topaz coloured laminated glass, are highly transparent for the viewer frominside and for those outside when the interior is lit - after dark, when mostof the building's theatrical functions are at play. Unlike most glass facingwater and sky, they do not reflect much of their environment, except from


an aerial vantage-point; the three distinct surface planes of the northernfoyer windows draw our gaze in rather than reflecting us and the world westand in.

These windows also make a major contribution to other kinds of Textualmeaning such as the Roof-Wall relation, Rhythms and Proportions. As JillSykes explains, after Utzon's resignation it took his replacement architectsnearly four years:

to solve the design problems, at first working through trial proposals and thentackling tricky situations as they arose under construction [. . .]

Linking the curves of the sails to the rectangular lines of the podium required aconcept that combined the aesthetic with the pragmatic. Without a mathematicalrelationship between the shape of the shell and that of the podium to use as astarting point for a geometrical solution to devising the structure of the twolargest glass walls overlooking the Harbour, a new design element had to beintroduced. The result was a combination of three surface planes: vertical at thetop, coming down to a half-circle leaning outwards from the vertical, then pullingback in a cone shape [Plate 1.5].

This verandah-style approach provided the practical advantage of extending thearea within the building well beyond the feet of the shells, as well as offering non-reflective views over the Harbour through the inward-slanting glass that ended atfloor level.

(Sykes, 1993: 62-63)

Sykes is here describing the resolution of Experiential ('pragmatic') andInterpersonal ('aesthetic') problems through the Textual functions ('geom-etry') of shell-glass wall-podium relations and the contrasting shapes, anglesand proportions created by the windows. Even she has difficulty in articulat-ing the sheer visual excitement of this brilliant and unique interplay of theparallel lines of the vertical mullions, gradually diverging in the other twoplanes, with the stepped window spacers and the curve of the intersectionsof the planes and the curve of the front of the canopy creating an intricateharmony with the curve of the shells above: only a musical metaphor can dojustice to the Textual meaning of these mathematical relationships.

We have discussed the Opera House's relation to adjacent buildings andto the road already in terms of their Interpersonal tensions and mimicry, buta full account must also recognize the Textual relations created by theirshared geometry (partly discernible in Plate 1.5). The strong vertical flutingof some of the tower blocks, the proportions of the relations between glasscurtain and solid plane walls and the curves of some towers or roof featuresall give the Opera House a distinctive role in the urban texture of Sydney.

Its relation to the city as a whole is highly dynamic. Because of its prom-inent and open, uncluttered site, it is visible from many vantage points, bothnear and distant, low and high. From the foot of the podium steps or apassing ferry it rears up colossally, as Utzon intended, but from the ferry


Plate 1.5 Intersecting geometries


terminals or the far side of Circular Quay it has diminished to an imposingsculpture against the skyline, or another sailboat cutting through the waves,while from the Harbour Bridge or the roads descending to the Harbourthrough North Sydney it has become a tantalizing jewel on a darkvelvet cloth.

We have allowed ourselves a lyrical passage here out of deference to themany encomiums that have been written and spoken and filmed for this'Eighth Wonder of the Modern World'. So much has been written, indeed,that we must ask whether anything remains to be said about the SydneyOpera House. Can a systemic-functional analysis (with or without the lyri-cism) add anything to the mass of books, magazine, journal and newspaperarticles, architectural, historical or political speeches, interviews or discus-sions devoted to it?

Apart from attempting to extend the limits of systemic-functional semi-otic theory by applying it to a complex three-dimensional work of art (whichis one of the aims of this book), I believe that a functional approach allowsone to see certain features in a new light. In the first place, it counters thesimplistic tendency to interpret 'functionalism' as concerned only or primar-ily with the utilitarian, Experiential, functions of a building. While recogniz-ing that the practical functions may have a priority in all kinds of text aboutbuildings, from architects' briefs to security manuals or tourist brochures, itasserts - and tries to prove - that the Interpersonal and Textual functions arejust as important in the elucidation of what a building 'means', whether tothe individual viewer, the citizens of Sydney, contemporary society or pos-terity. It does this not by generalizing, but in detail, teasing out the systemsof choice which are available to the architects, engineers and builders atdifferent ranks of unit - Building Complex - Building - Floor - Room -Element in each of the three functions. This then enables us to pinpointthose features of the building where the meaning is 'hottest', where specificfunctional meanings overlap, interplay or conflict to produce more complex,sometimes contradictory interpretations. The process may well generatenew insights we can share with others in an agreed common language.

The chart of systems and functions becomes a kind of hypertext - a non-sequential tool for exploring the hypertext of the building itself: the user canstart with any system in any box of the chart, analyse that part of thebuilding and interpret it in terms either of higher or lower ranks in the samefunction or in terms of related systems in other functions. Like any goodmap, it will still help us know where we are - theoretically as well as practic-ally - at any stage of our exploration. Similarly, as I have tried to show, wemay stand on the Opera House podium looking at the tiles on the shellsrearing above and around us. An appreciation of their colour and shine maylead us to the imposing grandeur of each shell or the whole building com-plex against the harbour and sky (higher ranks in the Interpersonal func-tion), or the geometrical textures of the tiles and chevrons may draw ourattention to the complex interplay of materials and geometry in the win-dows which I discussed under the Textual function.


A further theoretically principled step may then be possible if the func-tional meanings in the 'text' of the building itself are projected onto themanifestations of the 'social semiotic', that aggregation of opinions,assumptions and prejudices about what should be built and how it shouldlook that prevails in a given culture (which might be the political right or theartistic avant-garde of Sydney in the 1960s, or world architectural opinionin the 1990s, or mass tourist culture in 2000, Sydney's Olympic Year).

On a more prosaic and technical level, the specification of distinct ranksof unit in the systemic-functional model allows one to discriminate thekinds of choices the architect has made and the kinds of construal weourselves make as viewers, visitors and users of the building. And the specifi-cation of the systems which make up the 'grammar of architecture' helps usto understand the nature of the choices the architect has made in relation tothe practical, aesthetic, social, political and financial constraints which arelaid on him — and his justification in calling it quits when those constraintsbecome unmanageable.

Notes

1 J0rn Utzon in an interview for the film The Edge of the Possible: J0rn Utzpn and theSydney Opera House, director: Daniel Dellora, ABC Television, 20.10.98.

2 Michael O'Toole, The Language of Displayed Art (1994), Chap. 3 'A Semiotics ofArchitecture', pp. 85-144.

3 J0rn Utzon in an interview for the film The Edge of the Possible.4 My versions of Halliday's model for the systemic-functional analysis of painting

and sculpture (O'Toole, 1994) use the term 'Compositional function' for this kindof meaning in those arts which are primarily for display. In the case of archi-tecture, which, like language, is of practical use as well as display, it seemsappropriate to retain Halliday's notion of the 'Textual function'.

5 Harry Seidler in an interview for the film The Edge of the Possible.6 J0rn Utzon in an interview for the film The Edge of the Possible.1 Ervin Goffman, The Presentation of Self in Everyday Life (1965).

References

Dellora, D. (1998) The Edge of the Possible: J0rn Utzon and the Sydney Opera House. ABCTelevision, 20.10.98.

GofTman, E. (1965) The Presentation of Self in Everyday Life. London: Penguin Books.O'Toole, M. (1994) The Language of Displayed Art. London: Leicester University Press.Sykes, J. (1993) Sydney Opera House from the Outside In. Sydney: Playbill Proprietary

Ltd/Sydney Opera House Trust.

2 Making history in From Colony to Nation: a multimodalanalysis of a museum exhibition in Singapore

Alfred Pang Kah MengNational University of Singapore

Introduction

This paper explores how systemic-functional (SF) theory may be extendedto a social semiotic analysis of the museum exhibition as a multimodal site.The museum exhibition is obviously multimodal in that different semioticresources, such as photographs, three-dimensional physical objects, spaceand language, are co-deployed in complex ways to construct meaning. Isketch here a preliminary SF framework for the multimodal analysis of amuseum exhibition and exemplify its usefulness in articulating the criticalconstruction of historical meaning by particular displays in From Colony toNation, an exhibition at the Singapore History Museum (SHM) that repre-sents the national history of Singapore. By critical, I mean understandinghow the communicative complexity of the exhibition connects with thediscursive institution of the museum as 'a dynamic power-play of compet-ing knowledges, intentions and interests' (Macdonald, 1998: 3). In particu-lar, I reflect on how the making of Singapore's national history in FromColony to Nation serves to (re)produce particular dominant imaginings ofSingapore as a 'nation'. The general point here is that making history isnever value-free; it is, rather, imbued with power-knowledge relations1

invested in the site of historical production.

From systemic-functional linguistics (SFL) tosystemic-functional semiotics

The project to extend SF theory into the analysis of multimodal terrainssuch as the museum exhibition entails, in the first place, an understandingof the theory. SF theory, as Halliday (1970, 1973, 1978, 1994) originallyformulates it, has principally centred on language as the object of analysis.Hence, the emergence of SFL as a method of linguistic analysis informed bythe theoretical conception of language as a social semiotic; that is, languageas meaning potential that evolves with the functions it has to serve in socialliving (HaUiday, 1973; HaUiday and Martin, 1993). As HaUiday (1973: 34)asserts, 'Language is as it is because of what it has to do'. From the


standpoint of SFL, then, language constitutes the social practice ofmeaning-making.

Recently, there has been much interest among some practitioners of SFtheory in the analysis of specific non-linguistic semiotic modes of meaning(e.g. visual images in Kress and van Leeuwen, 1996; displayed artin O'Toole, 1994; movement in Martinec, 1998), as well as in the co-articulation of meaning between them and language (e.g. Lemke, 1998;O'Halloran, 1999; RaveUi, 2000; Thibault, 2000). Such an interest makesexplicit the fact that conceiving language as meaning potential necessarilyentails a broadening of perspective that recognizes its co-deployment (andhence co-evolution) with other non-linguistic semiotic resources in meaning-making. As Thibault (1997: 342, emphasis original) argues, '[t]he linguisticsemiotic is strongly coupled with the various other semiotic modalities in socialsemiosis'. It follows, then, that language is as it is not only because of what ithas to do, but also what it does with and to other semiotic resources.

Implicit in the choice of SF theory to facilitate an understanding of whatmultimodal texts mean is the assumption that the theory has reached a pointof development where the descriptive tools elaborated for analyzing lan-guage can be useful in articulating the dynamic processes of meaning-making within and across various semiotic resources (Baldry, 2000). Howvalid is this assumption? That is, what are the spaces within SF theory thatrender viable (or not) its extrapolation from linguistic to general semiotictheory able to cope with the analysis of multimodal texts? Unfortunately,there is no space here to explore in-depth these questions.2 For the purposeof this paper, however, it suffices to recognize that the viability of extrapo-lating SF theory into the field of multimodal analysis may be claimed on thegrounds that the principles that underpin its description of language areconceptualized at a level of abstraction relevant to social meaning-makingin general (Kress et al., 1997) These principles are:

1. The generality of Halliday's three metafunctions of language (Ideational,Interpersonal and Textual) as abstract semiotic functions (see Kress andvan Leeuwen, 1996; Lemke, 1998; O'Toole, 1994).

2. The exotropic lens of SF theory, which conceives of the non-accidentalrelation between language and social context, potentially affords thefoundation for modelling contextual semiotics. The crucial implicationhere is that 'there are no contextless signs' (Harris, 2000: 81). That is, thelanguage system which powers various instances of text comes into mean-ingful existence only in their situation within social context. More thanjust the socio-cultural environment, the exotropic lens of SF theory, in thelight of multimodality, entails a refining focus on co-contextualizing rela-tions between language and other semiotic modalities.

Notwithstanding the two principles above, it is crucial to recognize whatLemke (1998: 110) has termed as the principle of incommensurability between signsystems. That is, every semiotic system embodies its own unique complexity


and the co-articulation between two or more semiotic systems in multimodaltexts is multiplicative of the relative specificity of each semiotic (Lemke, 1998).Hence, the descriptive tools elaborated for language in SFL cannot be directlyimported and applied to the analysis of other non-linguistic modalities. It maybe necessary to formulate SF descriptions of specific non-linguistic semioticresources (e.g. O'Toole, 1994). However, this defines only partially the prob-lematic of multimodality. The development of such specific descriptive toolsshould hopefully culminate in some means of (un)packing the processes ofintersemiosis, which Ravelli (2000: 508) defines as 'a co-ordination of semiosisacross different sign systems'. In sum, we need to cultivate an integrationalsemiolog/1 to better understand how multimodal texts work.

Towards the analysis of From Colony to Nation

In this section, I sketch a preliminary SF framework for the interpretativeanalysis of the museum exhibition as a multimodal text. It is important torecognize that the conceptualization of any framework to understand anysocial phenomenon is inherently a reductive abstraction from the dynamicworlds that we inhabit. As such, it is not my intention here to insist on a strictconformal fit between the proposed framework and the myriad exhibitionstyles that one encounters in social living. Rather, I aim to explore thosedimensions that can be useful in articulating and negotiating one's (dis)-agreement with others about how an exhibition means. I also develop theframework as far as it allows me to adequately unpack the ideological natureof particular displays in the exhibition, From Colony to Nation.

A semiological approach towards museum communication is not new.Delibasic (cited in Maroevic, 1997: 29), for example, has conceived of themuseum as 'filled with signs or systems of signs, which are at the disposal ofthose who know how to interpret them'. It is important to recognize,though, that museum communication is more than the exhibition. AsHooper-Greenhill (1999) observes, catalogues, books and souvenirs inmuseum shops, for example, also form a strategy through which museumscommunicate with the public. Nonetheless, the exhibition warrants primaryattention in museum communication as it is still 'a typical museum mediumfor expressing the museum message' (Maroevic 1997: 30).

Broadly speaking, at least two perspectives may be discerned from thedevelopment of various semiological approaches undertaken in museumstudies. The first tends to centre narrowly on the collection of materialobjects as the means par excellence of communication in a museum (e.g.Pearce, 1991, 1994). Noteworthy in such analyses is the conclusion that theartefactual significance of objects lies in the socio-cultural relations of theirproduction, circulation and use. However, it is crucial to recognize that thevalues of artefactual objects are as much mediated by the institutionalenvironment of their display in a museum. This leads us to the secondperspective, which emphasizes the (re)appropriation and (re)interpretationof artefactual objects in relation to the composite design of an exhibition as a


whole (e.g. Hooper-Greenhill, 1999; Kavanagh, 2000; Vergo, 1989). AsSmith (1989: 19) puts it:

artifacts do not exist in a space of their own, transmitting meaning to the specta-tor, but on the contrary, are susceptible to a multiform construction of meaningwhich is dependent on the design, the context of other objects, the visual andhistorical representation, the whole environment.

Such a perspective may be increasingly relevant now, given the prevailingtrend to democratize museums through the creation of audience-orientedexhibitions, where 'a shift in focus from individual objects to a "whole gal-lery experience"' (Martin, 1997: 36) is encouraged. Herein lies the pressingmotivation to conceive of the exhibition as a multimodal social semiotic,where objects are rarely left to 'speak for themselves' (Vergo, 1989: 49), butmean in collaboration with other semiotic modalities such as space, visualimages and language.

Multimodality in an exhibition implies the multi-tiered complexity ofmuseum messages. While this has been generally acknowledged in variousstudies on the museum exhibition (e.g. Belcher, 1991; Hall, 1987; Hooper-Greenhill, 1999), what remains insufficiently elucidated is the 'what' ofthese tiers that underlie the exhibitionary construction of meaning. In thisregard, Halliday's (1994) three metafunctions for language - Ideational,Interpersonal and Textual - provide a useful dimension to organize thismulti-tiered meaning potential of exhibitions '[as] pieces of functionaldesign with the purpose of doing a specific task' (Belcher, 1991: 41). Indeed,this tripartite organization of meaning seems latent in Bennett's (1995: 67,emphasis mine) conception of the exhibitionary complex., which is an 'ability toorganize and coordinate an order of things and to produce a. place for the people inrelation to that order1. The museum exhibition performs an Ideational functionin representing a cultural practice that construes social 'realities'. It realizesan Interpersonal function by powerfully addressing and shaping the inter-ests of visitors in particular ways. The Textual function orders the intercon-nected flow of both ideational and Interpersonal meanings to compose anexhibition as a coherent and cohesive whole.

The metafunctional organization of the meaning potential of a museumexhibition has been broadly conceived in Ravelli (1997, 2000). According toher, the exhibition is a site for intersemiosis, which is 'a co-ordination ofsemiosis across different sign systems' (Ravelli, 2000: 508). Rather than thespecific analyses of individual semiotic codes per se, Ravelli (2000)emphasizes the productivity of a macro-level analysis in unpacking theinteraction between them in an integrated way. The framework formulatedhere aims to abstract such macro features of meaning that emerge from thedynamic interplay of various semiotic modalities deployed in an exhibition.However, it does not (and perhaps should not) preclude the relevance ofmicro-level analyses of individual semiotic systems whenever possible.

To recall an earlier discussion, the nature and extent of their interaction


depends on the relative specificities of each semiotic resource co-deployed.As such, the interpretative framework I suggest is open to apply eclecticallyparticular SF descriptions conceptualized for specific semiotic codes(for example, language in Halliday, 1994; visual images in Kress and vanLeeuwen, 1996; displayed art - including sculptural and architectural texts -in O'Toole, 1994). The point of integrating such descriptions is directed todiscern that level of deep detailed analyses required for each sign system soas to explicate its interconnection with other semiotic modalities. It is thusmy view that analytical approaches to unpacking multimodal texts in gen-eral need to maintain a balance between micro- and macro-level perspec-tives of the range of semiotic resources coordinated. In practice, of course,this balance is also subject to the purposes of the research analyst.

Apart from the metafunctions, the logic of a rank-scale in SFL alsoprovides another dimension to conceptualize the multi-tiered complexity inan exhibition. In the case of a museum exhibition, it is possible, by analogy,to postulate a rank scale based on a hierarchical layering of spatial constitu-ents: Museum, Gallery, Area and Surface/Item. These rank units, which Iterm as Sites, are conceived as different environments wherein an exhibitioncan be viewed. Each environment presents a set of dimensions that orientsthe analytical 'eye' to interpret the (multi)semiotic space of an exhibitionfrom a particular angle.

Thus, as conceived in Table 2.1, the three metafunctions and the order ofsites may serve as two axes of a matrix of systemic components that charac-terize the meaning potential of a museum exhibition. There is, however, nospace here to explain in detail each systemic component in the matrix. It ishoped that the analysis in the section which follows will sufficiently illumin-ate some of the components in the matrix. At this juncture, it is worthstressing that the various components in the proposed functional semioticmodel are, in reality, more fluid than their discrete placing in the matrixsuggests; that is, 'certain features [can] either operate in more than onefunction or have consequences for other features from other systems, func-tions or ranks of unit' (O'Toole, 1999: 6).

I also explore how the co-patterning of these options from various semi-otic modalities may be organized by the co-evaluation of some phenomenaalong some foregrounded parameter. In this respect, I consider the possibil-ity of extending Appraisal Theory (Martin, 2000a) into the domain of multi-modal discourse analysis. The point here is that evaluation can serve as anintegrative principle organizing intersemiosis. The basis for this may belocated deeply in the question whether one can mean anything outsideevaluation. That is, when are humans not evaluating if the view is that the'worlds' which selves inhabit are always created in dynamic relation with., forand to others? According to Hernadi (1995: 116):

emotive awareness initiates the dialectical process through which the self and itsworld 'make' each other so that the former may begin to 'mean' and 'do' — bothcognize and act upon the latter.


He further suggests that 'if our evolution has enabled us to evaluate ingreater depth, our evaluations enable us to evolve at a far more unsettlingspeed than members of other species' (Hernadi, 1995: 135). That the actof evaluating is behaviour potential (Ravelli, 2000) that possibly co-extendswith making-meaning through some material-semiotic technology in theexosomatic evolution of the human species lends natural credence toevaluation as an integrative principle of intersemiosis. To put it anotherway, it is difficult, if not impossible, to mean something without also evalu-ating it. The co-evolution of language with other semiotic modalities isprobably marked with co-evaluation between them, which in turn inter-twines with the larger co-evolution of nature and cultures in the ecologyof being human. From this perspective, evaluation is multi-layeredand takes place at many levels in making-meaning. These various levelscould well afford different scales by which multiple semiotic modes arecombined.

It is critical to recognize that access to and selection of possible configur-ations of these components towards evaluation in an exhibition are as muchregulated by the communities of values and beliefs invested in the ideologicalspace of the museum (Hodge and D'Souza, 1999; Karp, 1992). As Hooper-Greenhill (1992: 214) cautions:

The total experience (in living history or interactive exhibits), the total immersion(in gallery workshops and events), can have the function, in the apparently dem-ocratized environment of the museum marketplace, of soothing, of silencing, ofquieting questions, of closing minds.

In other words, the current popular paradigm that pushes for the democra-tization of the museum does not equal the dissolution of power. Instead, itindexes the powerful capacity of the museum in strategically negotiating itsinstitutional authority to position the subjectivities of its audience in particu-lar ways. This ideological motivation of meanings construed and constru-able in an exhibition is taken seriously in an SF framework that emphasizesa dialectical relationship between social context and semiotic system(s).

Touring From Colony to Nation — 'Communist United Front'

From Colony to Nation is a permanent exhibition at the Singapore HistoryMuseum (SHM) and displays the national political history of Singapore.This exhibition, which opened on 19 July 1997, was motivated by the formu-lation of National Education (NE) by the ruling People's Action Party (PAP).The idea of NE was initiated by Prime Minister Goh Chok Tong at theTeachers' Day Rally on 8 September 1996 in response to a survey, whichfound students ignorant of Singapore's past, particularly '[t]he country'sstruggle against communism, and how it went about getting self-rule andindependence' (The Straits Times., 16 September 1996).4 According to Goh(cited in Wee, Business Times, 31 May 1998):

Table 2.1 Systemic functional framework for a museum exhibition

Site/ IdeationalFunction

Interpersonal Textual

Museum Museum type Disciplinary Field Target AudienceArchitectural Appeal

Public/PrivateExternal environment Relation to city/Relation toInternal environment adjacent buildingsRelation to PracticalFacilities

Gallery Narrative Design Interplay of GenresInterplay of Areas

Ideal VisitorCirculation Path

Setting (Mood)

Traffic Flow/FlowRateLighting, Colour,Size, Volume, Kindsof Object Props

Internal CohesionSequence of Areas Focal points

Rhythm Lighting, Colour, Scaleof Exhibits, DisplayDensity, Degree ofPartition

Information CompositionExternal Cohesion (e.g. relativeprominence in museum, relation toconnectors — corridors, stairways)

Area Sub-narrative ThemeInterplay of Surfaces (i.e. displays onwalls, floors and ceilings)

Circulation PathSetting

Traffic Flow/Flow RateLighting, Colour,Size, Volume, Kindsof Object Props

Rhythm Sequence of SurfacesRelative Prominence of Area

Site/ IdeationalFunction

Interpersonal Textual

Surface/ Topics (Sub-topics)Item

Relationship MapIntra-relationship of elements in an itemInter-relationship of elements across items

Image-Word-Object: Extra-VocalizationSemioticMetaphorObj ectificationMetonymy

Visual semiotic O'Toole (1994)/Kress andvan Leeuwen (1996)Linguistic semiotic: Halliday (1994)

Interactivity

Interpretive PathDirectional PathFocus (CVI)

PerspectiveViewing height

Gaze and other sensorymodes of attention

Interplay of modal andcompositional elements(e.g. Colour, Light,Shape, Size, Lines)

Display Style ClassificationArrangement

Visual Salience Balance:Flank/ SpiralAlignment

Information CompositionRelative Prominence of Surface/Item


National Education . . . is an exercise to develop instincts that become part of thepsyche of every child. It must engender a shared sense of nationhood, an under-standing of how our past is relevant to our present and future. It must appeal toboth heart and mind.

Deputy Prime Minister Lee Hsien Loong reiterated this position at theformal launch of NE on 19 May 1997, saying that it is 'a concerted effort toimbue the right values and instincts in the psyche of our young' throughteaching 'the Singapore Story - how Singapore succeeded against all oddsto become a nation'. Thus, From Colony to Nation, which is also referred to as'The story of Singapore' in the exhibition guide (see Plate 2.1),5 has a strongpedagogic purpose that is tightly circumscribed by the ideals of NE, namelyto underscore the constraints and vulnerabilities of Singapore. I discuss nowhow the intent of NE motivates a selective remembering of Singapore'srecent political past, with particular focus on an Area - the 'CommunistUnited Front' — that displays the Communist movement in Singapore afterthe Japanese Occupation.

It is worthwhile first to contextualize this Area concerning the Communistmovement in terms of the Narrative Design at the rank of Gallery. Typicallyreferred to as the 'storyline' among exhibition makers, the Narrative Designis abstracted as that overall thematic content of an exhibition that binds theparticular selection and arrangement of multiple semiotic systems. As Vergo(1989: 46) puts it:

in the case of most exhibitions at least, objects are brought together not simply forthe sake of their physical manifestation or juxtaposition, but because they arepart of a story one is trying to tell . . . Through being incorporated into anexhibition, they [objects] become not merely works of art or tokens of a certainculture or society, but elements of a narrative, forming part of a thread ofdiscourse which is itself one element in a more complex web of meanings.

The Narrative Design is, then, an 'interpretative strategy' (Dean, 1994: 103),within which the subject matter of an exhibition is formulated at severallevels of complexity. An aspect of this complexity lies in the Interplay ofGenres, which is worked through the social experience of a museum visit.An instance of this would be the experience of picking up and glancingthrough an exhibition/gallery guide before viewing the actual three-dimensional display. In From Colony to Nation, where no main introductorypanel is installed, the exhibition guide plays a marked role in providingvisitors with an overview of the content of the display. More significantly,the exhibition guide, in orientating the visitor to '[t]ake a walk throughhistory and understand why Singapore must prize her independence aboveall else', inflects the historical recount displayed as an exemplum. Anexemplum, according to Martin (2000b: 8), 'relate [s] a sequence of events inorder to make a moral point'. The moral point here is the obligation forSingaporeans to value positively and not take for granted the country'sindependence.

Plate 2.1 Exhibition guide to From Colony to Nation (layout plan)


This interplay between the guide and three-dimensional display may beconceived as a generic chain (Fairclough, 2001)6 across media, which co-evaluates Singapore as a vulnerable body politic. The vulnerability ofSingapore, therefore, forms an even more abstract theme that organizes theinterconnectivity between various semiotic resources in the exhibition. Thisstate of vulnerability is perceived within the Narrative Design through theerection of points that risk the status quo established by the PAP govern-ment. Communism is one such risky point.

Now, I move into the Area 'Communist United Front' (see Plate 2.2) anddiscuss how the co-deployment of various semiotic resources (primarilywritten language, visual images and space) serves mainly to discredit theMalayan Communist Party (MCP).

From the outset, the undesirability of the Communists is already indicatedby the thematic classification of this Area under 'Colony in Chaos' (see Table2.2, p. 40). In the exhibition on the left wall, this classification is indexed bythe use of a red board on which the linguistic text panel is mounted.

Linguistic text panel

Written language is used in the main text panel and in museum labels.Table 2.3 (see pp. 42-43) contains a linguistic analysis of the main text panelin terms of its schematic organization and the sub-system of Attitude inAppraisal Theory (Martin 2000a).

Attitudinal evaluations of the MCP and pro-Communists are mostlynegative Judgements on propriety. For example, Material Processes like'infiltrating' (clause 7), 'exploit/ed' (clauses 8 and 11) and 'incited' (clause21) dramatically construct a negative Judgement of (pro)-Communists asreactionary, unlawful, manipulative and perhaps even irrational. Note-worthy too is the accumulation of negative Judgement from clauses 3—10,which function to elaborate the Thesis. It is interesting to observe how theseries of non-finite in clauses 6—10 appears to 'quicken' this accumulationby allowing a jam-pack of New information, which refers back to 'It' (clause5) as thematized Actor. This 'It', in turn, anaphorically refers to the MCP.A cluster of attitudinal evaluations is thus rhetorically woven to intensify thenegative evaluative force on Communism.

Noteworthy in the analysis presented above is also the embedding of twohistorical recounts - the May 13th Incident in 1954 and the Hock Lee BusRiots in 1955 — as examples of Communist-instigated violence. Thisembedding has the effect of re-interpreting the historical recounts to the pointof the Thesis (clause 2), which generalizes via an intensive identifyingrelational process the use of violence as the primary strategy by which theMCP aimed to achieve power. Indeed, the negative propriety of the Com-munists is predicated on this use of violence. The point of this linguistic textis not to recover the specifics of the actual perpetrators and victims in theseacts of violence. Rather, within the genre of an exemplum, the social pro-cess here is to moralize violence as socially undesirable in order to discredit

Plate 2.2 Display area of 'Communist United Front'


Table 2.2 Classificatory scheme of From Colony to Nation

Colony in Chaos Tides ofTransition

Nation-Building

World War II &Southeast AsiaDivided PopulationThe Maria HertoghRiotsA Time of HardshipA Political Goal: Unionwith MalayaCommunist UnitedFront1955 General ElectionsSelf-Government

Mighty MalaysiaProposalHistoric PAP splitBattle for MergerReferendumConfrontationPolitical RivalryEconomic ProblemsRacial TensionRacial RiotsSingapore is Out!

On Our OwnWe Had to Accept RealityPolitical UnrestWho will Protect Us?The Struggle to LiveForeign RelationsDefending OurselvesEconomic GrowthCaring for our People (up to1970s)Passing the Baton (1984/1990)Our Presidents (1965-present)

Communism and Communalism Elaboration of 'national'interests in terms of what isneeded for Singapore to survive(Economic Pragmatism)Communitarian values

Division —> Unity in Diversity

the Communists. Any act of violence which might have been committed bythe police then is from the start tolerated and legitimized as control.

Moving into space

The spatialization of information is a central feature in the three-dimensional text of an exhibition. As Bennett (1995: 6) remarks, 'an exhibi-tionary space . . . is a place for "organized walking" in which an intendedmessage is communicated in the form of a (more or less) directed itinerary'.The framework here conceives this 'organized' walking as the system ofCirculation Path under the Interpersonal function. There are two aspects toCirculation Path: Traffic Flow which concerns the routing through a seriesof spaces within an exhibition, and Flow Rate which relates to how a visitoris paced along the circulation route throughout a gallery and within an areaof an exhibition. The system of Circulation Path is visually represented inFigure 2.1.

Apart from the application of Circulation Path, I also examine inthis section the operation of semiotic metaphor in the spatial re-representation of the meanings constructed in the linguistic text panel.

1945-50s 1960s 1965 present


Figure 2.1 System of Circulation Path (adapted from Royal Ontario Museum1999)

FoUowing O'Halloran (1996, 1999, 2003a, 2003b), semiotic metaphorrelates to the semantic shift that takes place inter-semiotically, duringwhich the function of an element may be receded and new functionalelements may be introduced in the movement from one semiotic resourceto another.

In her investigation on secondary school history, Coffin (1997: 202)notes the linguistic construal of external and internal time in organizingthe past. The linguistic text panel sets up a chronological template inwhich external time unfolds categorically through marked Circumstances(in bold):

(03) In 1948, it failed in an armed uprising during the emergency(19) On 13 May 1954, students and police clashed(20) In May 1955, the pro-communists incited students to join the Hock Lee

Bus workers in a strike.

Internal time is deployed to build up an explanation about the past and thisis linguistically construed in the text panel via logical links of Cause. Now, thespatial semiotic also affords the capacity to realize external and internaltime, but perhaps in ways less differentiated than language.

The three-dimensional spatialization of external time can be seen toinvolve parallel semiotic metaphor. The events dynamically recounted alonga chronological timeline of marked Circumstances in the linguistic text arephysically bounded in a more or less rectangular enclosure with exhibitsdisplayed along the two longer walls (see Plate 2.2). The left wall consists of


Table 2.3 Main text panel: schematic organization and attitude

Thesis

Elaboration

Summary

(01)(02)

(03)

(04)

(05)(06)

(07)

(08)

(09)

(10)

(11)

Exemplify IRecount of May13th Incident

(19)(20)

Communist United FrontThe Malayan Communist Party's (MCP) fundamental[Appreciation: -valuation] aim was the establishment of acommunist state in Malaya (including Singapore) by revolutionaryviolence [token, Affect: insecurity: disquiet —> token,Judgement on MCP: -propriety]

In 1948, it failed Judgement on MCP: -capacity] in anarmed uprising during the Emergency [token, Judgement onMCP: -propriety]and went underground [token, Judgement on MCP:-propriety].It then changed its tacticsto form a communist-controlled united front [token, Judgementon MCP: -veracity / propriety]by infiltrating Judgement on MCP: -propriety] into legalorganizations such as trade unions, students' unions,farmer's associations, Women's Federation, cultural groupsand political partiesto exploit Judgement on MCP: -propriety] grievances[Affect: unhappiness: antipathy],expand their influence [token, Judgement on MCP:-propriety]and eventually gain control of these organizations, [token,Judgement on MCP: -propriety]

The MCP through the communist united front exploitedJudgement on MCP: -propriety] anti-colonial feelings[Affect: unhappiness: antipathy], concern about Chineseeducation [Affect: insecurity: disquiet], feelings of social

frustration and economic injustice [Affect: dissatisfaction:displeasure].

When the British announcedthat 2,500 youths would be drafted under the NationalService Ordinance,the pro-communists fanned discontent [token, Judgement:-propriety]by claiming Judgement: -veracity]that locals were usedto further colonial rule [token, Judgement on the British:-propriety].Mass student protest demonstrations were staged [token,Judgement on student demonstrators: -propriety—> token, Judgement on pro-Communists (agentellipsed): -propriety]On 13 May 1954, students and police clashedand 48 students were arrested [token, Judgement onstudents: -propriety —> token, Judgement on pro-Communists: -capacity; token, Judgement onpolice: +capacity].

(12)(13)

(14)

(15)(16)(17)

(18)


Exemplify II (21) In May 1955, the pro-communists incited [Judgement:Recount of veracity] students to join the Hock Lee Bus workers in aHock Bus Riots strike [token, Judgement on pro-Communists:

-propriety].(22) Violence broke out [token, Affect: insecurity: disquiet —»

token, Judgement on pro-Communists: -propriety],(23) resulting in 4 people dead and 31 injured [token, Affect:

insecurity: disquiet —» token, Judgement on pro-communists: -capacity].

items that relate to the May 13th Incident in 1954, while the right wallexhibits items associated with the Hock Lee Bus Riots in 1955. Thereappears to be a shift from the linguistic construal of time as Circumstance toits spatial experience in the exhibition as a physical material Thing. It is thissemantic shift that enables the further compression of these events into aperiod, negatively appraised in its sub-thematic classification as 'Colony inChaos'.

The semantic shift is parallel in the sense that no new functional entitiesare introduced in this reconstrual although there is an overlay of meaningenabled by the system of Circulation Path in the spatial semiotic. The sightof space simultaneously invites its traversal. The continuous material pro-cess of'organized walking' (Bennett, 1995: 6) now topologically enacts thedynamic unfolding of time (external and internal) in space. The system thusactivated is that of the Circulation Path. From the perspective of Traffic Flow,this display on the Communists is situated relatively early in an Arterialpattern (see Plate 2.1) from left to right. This left-right directional flow isexplicitly insisted upon by the instruction on the Exit Door: 'Please enterexhibition via door on the left'. Interpersonally, the Arterial pattern pro-motes a didactic stance in that the visitor is given little choice in choosingher/his pathway through an exhibition. This textures the importance of thisdisplay since a visitor is made to walk through it anyhow.

Now, I focus on the Flow Rate, which is affected by the arrangement ofwalls. In this Area, the two longer walls run parallel to each other and areconjoined by a straight path through. Movement through this pathwayenacts a conjunctive relation in the Interplay of Walls. This conjunction isnot merely an additive of two external timeframes (referenced as 1954 and1955 from the linguistic text panel), but also expresses their internal relationas examples of Communist-instigated violence. More significantiy, this spa-tial design, by its relatively low Degree of Partition, affects a Flow Rate thattends not to be crowd stopping.

Furthermore, following Arnheim (1982: 61), the two longer walls of therectangular enclosure tend to emphasize an axial symmetry, which propelsthe Ideal Visitor to move forward and ahead of the Area, towards the portraitpainting of Lee Kuan Yew being sworn in as Prime Minister in 1959. Thiscoloured oil painting, enshrined in a gold frame, stands out in contrast to theblack walls and the black-and-white photographs used. According to Bal(1999: 176), the portrait is


[a] genre that bestows authority upon its subject. Its history is bound up with thatof capitalism, individualism, bourgeois culture . . . portraits are made to honorpower.

Thus, apart from visual contrast in the display design, the intertextual allu-sion to such generic conventions about the portrait marks the painting as afocal point, which indexes the starting-point within the Narrative Design ofhow the elected PAP Government (represented metonymically and authori-tatively by the figure of Lee) would overcome all odds to build Singaporeinto what it is today. From the perspective of Flow Rate, then, the relativeprominence of the Topic 'Communist United Front' is downplayed. It is notthat the Topic has become less important or significant. Rather, what seemsto be enacted by the continuous Flow Rate is perhaps a channelling of thatsignificance to an appropriateness of distancing oneself from Communismtowards the promise of social prosperity that the PAP Government hascome to stand for. This gesture of distancing is furthermore directed toreinforce the negative desirability of Communist activism in general.

Photographic images

I examine the collection of thirteen photographs placed immediately afterthe text panel along the left Wall (see Plate 2.3). What probably arrests avisitor's attention to this collection of photographs is the wired fence. Thesignificance of this wired fence, other than its role as a focal point that drawsa visitor's Gaze to the photographs, is discussed later in this section. For now,I concentrate my analysis on some of the photographic images displayed.For the specific analysis of the meanings constructed in each photograph, Iapply eclectically the SF interpretative frameworks formulated in O'Toole(1994) and Kress and van Leeuwen (1996). The analyst's situation is, how-ever, further complicated in the medium of a museum exhibition, wherehow any single photographic image can mean is as much mediated by itsdissemination alongside other photographs through display practices, twoof which are discussed here: museum labelling and setting.

In relation to the exposition set out by the linguistic text panel, thesephotographs serve as artefactual evidence that testify to the 'truth' of the May13th Incident recounted in clauses 12-20. Following O'Toole (1994), theRepresentational content expressed (at the rank of Work) in the thirteenphotographs consists of Scenes of police control and arrest, crowd dispersionand injury, all of which illustrate the non-productive consequences of theMay 13th Incident. In addition, photographs in black-and-white and par-ticularly sepia not only evoke a sense of the past, but also hark back to thetraditional genre of documentary. As Price (2000: 75) writes of documentaryphotography, one implicit claim that underlies its historical development isthat 'it offers us a disinterested and true picture of the world'. It is preciselythis naturalistic coding orientation (Kress and van Leeuwen, 1996; Thibault,2000) that underpins the evidential value of each photographic image.

Plate 2.3 Display of photographs on the May 13th Incident (left Wall)

image a


There is, in other words, the social assumption that photography real-istically captures 'an immediate and transparent identity between image andreferent' (Phillips, 1998: 155). However, as Ryan (1993, cited in Price, 2000:69) argues:

Despite claims for its accuracy and trustworthiness, however, photography didnot so much record the real as signify and construct it.

Tagg (1988: 187, emphasis mine) similarly reminds us that

[t]he photographer turns his or her camera on a world of objects already con-structed as a world of uses, values and meanings, though in the perceptual pro-cess these may not appear as such but only as qualities discerned in a 'natural'recognition of'what is there'.

Thus, rather than imputing an ontological status to realism, what is under-scored so far is its discursive constitution that invests the photographic imagewith an authority to authenticate. The photographs exhibited on this wallare themselves social semiotic constructions whose perceived naturalisticcoding orientation is worked through the genre of documentary to reify thefacticity of the linguistic recount of the May 13th Incident.

The photographs displayed are reproductions rather than 'originals'. Itfollows from this reproducibility that photographic images are 'transmutableobjects . . . involved in endless, complex acts of circulation and exchange'(Price, 2000: 111). That is to say, '[t]he photograph is not a magical "eman-ation" but a material product of a material apparatus set to work in specificcontexts, by specific forces, for more or less defined purposes' (Tagg, 1988:3). The key principle here is the recontextualization (Thibault, 2000: 364—365)of photography in relation to other social practices configured within par-ticular institutional spaces. In this display, the practice of museum labellingrecontextualizes photographs as archival knowledge. The photographs areminimally labelled as '1954 Student Riots; National Archives Singapore'.Such a form of labelling generalizes two reference points: first, it reductivelyidentifies what the photographs are about under a general classification'1954 Student Riots'; second, it specifies the source ('National ArchivesSingapore') from which these images are retrieved and reproduced.

Now, Smith (1989: 12) has argued that museum labelling 'conceals acomplex history' of artefacts on display. In this instance, however, labels donot simply hide but recreate the historical significance of the photographs asan archive. As Sekula observes of photographic archives, 'they heaptogether images of very different kinds and impose upon them a homo-geneity that is a product of their very existence within an archive' (Price andWells, 2000: 59). This thrust towards homogeneity is also directed strategic-ally to achieve particular social purposes. That is, archival knowledge isnever created for its own sake but for its appropriation to serve some (dom-inant) social impulse to recollect and review past times. The larger point


imprinted is that the significance of photographs as material artefacts is asmuch shaped within institutionalized relations of their select use.

I move on to probe deeper into the inter-semiotic mechanisms betweenlanguage and visual imagery in this display practice of labelling. It is inter-esting to observe how labelling photographs in terms of their source may beanalogous to the rhetorical technique of Attribution, where the content'cited' is now construed by the visual semiotic. Attribution here is also sig-nificant in enacting the 'network of intertextual connections' (Lemke, 1995:11) between SHM and the National Archives of Singapore (NAS). Furthercomplicating the discourse on heritage and history, then, is the collaborationof social practices between different memory institutions. In this instance,the institutional authority of NAS as a 'resource centre for the research anddissemination of information on the history of Singapore' (NHB 2000: 4) isevoked to authenticate the documentary value of the photographs exhib-ited. The institutional status of these photographs as official evidence in turndetermines the credibility of the propositions in the text panel. It is in thisrespect that the heteroglossic space of the discourse on Communism tendstowards constriction.

The Arrangement of photographs on this wall further positions the visitorto sympathize with the police as riot victim. Of the thirteen photographs,the only visual representation of injury is that of a policeman with a ban-daged head (see Image A). Image A, in its portrait formatting, seems visuallysalient as a pivotal centre balancing the flanked Arrangement of photo-graphs. Interpersonally, Image A is also prominent as the only photographthat directly addresses the visitor through the direct outward Gaze of thewounded policeman. The frontal angle of the shot, coupled with the nearcentral positioning of the injured policeman with his head slightly tilted,encourages viewer involvement and amplifies sympathy.

Noteworthy in Image A is the observation that the injured policeman isnon-Chinese (most probably Malay). In fact, most of the policemen cap-tured in the photographs are non-Chinese. The student demonstrators are,on the other hand, predominantly Chinese. To recall, historical research(for example, Lee, 1996; Wee, 1999) has reported how the Communistmovement in Singapore during the 1950s and 1960s developed in relationto its capacity to garner and mobilize support from the world of Chinese-speaking Chinese. As sociologist PuruShotam (1998: 55) also notes, 'Theequation according to which language equals culture equals race mirroredthe perceptions of students, supporters and sympathizers of the cause ofChinese education'. In this light, the Communists' alignment with theChinese-educated may be seen as provoking Chinese Communalism againstcolonialism. However, aside from the brief mention in the text panel of theMCP exploiting 'concern about Chinese education' (clause 11), the racialdimension of the Communist conflict in Singapore remains relativelyunelaborated in the exhibition. Racialization is also only covertly impliedthrough skin colour in the photographs. In sum, within the institutionalcontext of the museum, the photographs acquire a documentary value that


not only objectifies the negative appraisal of the Communists and theiractivities, but also layers it with the delicate complexity of race.

What probably arrests a visitor's attention to this collection of photo-graphs is the hapticity7 of the wired fence. The wired fence is used here asan object prop to 'fabricate' the Setting of a prison. It is within this Setting ofimprisonment that the photographs come to be interpreted as ideationaltokens of negative Judgement on riotous behaviour. The visitor walkingthrough the floor of this area is simultaneously locked in and out from theScenes captured in the photographs. This physical barrier serves as a 'safetynet' that 'protects' the visitor from acts of violence. In preventing the visitorfrom having any direct tactile contact with the photographs, the wired fenceenacts a form of metaphorical distancing from a riotous past. Even one'svisual interactivity with these images is 'intervened' by the criss-cross ofwire, as if dictating that these riots in the past should not be allowed torepeat themselves in present time. What may be implied in this constructionis the importance in preserving the 'safety net' that the PAP Governmenthas thus far spun for the peaceful progress of Singapore as a nation-state.

The wired fence thus amplifies the scale of the undesirability of theCommunist movement. In addition, the perceived risk of physical painevoked by the barbed wiring at the top disciplines the visitor into acceptingpolice control as a necessary and legitimate deterrent against Communismlest Singapore becomes a totalitarian state. For some, there may seem to be adash of irony here since police surveillance is as instrumental in enforcing asense of totalitarianism. Yet, any force wielded by police power remainshidden and naturalized behind a legalistic frame of social order presentlyarticulated to criminalize the Communist movement during the 1950s.

Ideological motivation

The exhibition, which displays a dominant 'progressivist national narrative'that stages 'a transition from a colonial society to a modern capitalist one'(Wee, 1999: 169, 172, emphasis original), suppresses any formative role theCommunists played in the 'nation-ising' of Singapore. The collective multi-modal definition of the Communists as a dangerous riotous Other is filteredthrough the dominant lens of communitarian ideology (Chua, 1995) pres-ently held by the PAP Government. Communitarian ideology is mostrecently articulated and instituted in the Government's 1991 White Paperon Shared Values.8

Two of these Shared Values are transmitted through this display. First, thenon-legitimate place of revolutionary violence emphasizes PAP's order ofpolitics, which is one founded on constitutional consensus rather than con-flict; this echoes the Shared Value Consensus instead of contention. Second, down-playing the racial script in this display also aligns the exhibition with theShared Value of Racial harmony. As Wee (1999: 170) writes of the delicateracial communal tension that underlay the mobilization of Communism inSingapore's 'stage of nationalist polities':


The problem was that if the Chinese-speaking of various linguistic stripes(Mandarin, Hokkien etc.) were the politicized masses who could be mobilized,they also represented a problem for the imagining of a multiethnic or multi-cultural 'national' community. In Singapore, a clear-cut national entity could notbe created, as there was not one single 'nation' on which to build the nation-state— a common enough problem for a former colonial state.

The representation of the 'Communist United Front', in precludingthe historical view that the PAP initially rode on the force of Chinesecommunalism as a source of anti-colonial resistance in nationalist politics,preserves the party's dominant ideal of multiracialism. This emphasis onmultiracialism is in line with the museum's mission of 'preserving and inter-preting the nation's history and material culture in the context of its multi-cultural origins' (NHB 2000: 5, emphasis mine).

'Closing' the tour

In this paper I have tried to exemplify how the SF theory may be extendedto articulate systematically the complex dimensions of meanings construedin From Colony to Nation. A major emphasis is the collaboration between thesedimensions in co-evaluating the spectacle of history in particular ways.Through the analysis I hope to have extended evaluation (or Appraisal the-ory in SF context) as a discursive end realized by the interaction of varioussemiotic systems. This extension of Appraisal theory into the multimodalterrain of the museum exhibition has also led us to appreciate evaluativedynamics as essentially multi-levelled. Cortazzi and Jin (2000: 119) haveactually conceived of this multi-level complexity from three perspectives:evaluation in, through and of narrative. These three perspectives may beextended to social semiotic practices in general. At the close of this paper,it might be worthwhile to tease out more clearly for the reader the operationof these three evaluative levels, which have remained implicit in my preced-ing analysis of the exhibition.

On the first level, there is evaluation in the exhibition's Narrative Designwhere the co-evaluative relations between multiple semiotic resources assessthe historical representations of Communist and communal unrest in spe-cific ways. It is worth emphasizing that, even at this level, evaluation isshown to be simultaneously implicated and complicated in the Interplay ofGenres configured within particular institutional formations.

As reflected in the analysis, the period of Communist insurgency ('Com-munist United Front') is evaluated within the Narrative Design as traumatic.Perhaps, as Antze and Lambek (1996: xii) have observed, 'memory worthtalking about — worth remembering — is memory of trauma'. More import-antly, foregrounding the traumatic nature of any incident here is also point-ing to its control. As Neal (1998: 5) conceives:

A national trauma involves sufficient damage to the social system that discoursethroughout the nation is directed toward the repair work that needs to be done.


This 'repair work' to recover from the trauma of Communist and com-munal violence allows the reinstatement of the communitarian ideologyespoused by the ruling PAP Government. Herein lies the second evaluativelevel, where through the Narrative Design, the national 'self of Singapore ispositioned as vulnerable; this vulnerability includes especially the delicateproblem of difference posed by race. In this light, communitarianism isposed as a form of social discipline cultivated to prevent a relapse into atraumatic past. This social discipline is hardly resisted primarily because ofits pragmatic effectiveness in sustaining Singapore's material progress. Thebody politic of Singapore thus risks trauma if there should be a lapse fromthis progress. Underscored in all these is also the discursive positioning ofthe museum (SHM) as a State apparatus that plays a political role in repro-ducing PAP's ideals of a Singapore citizenry. Such politicization residesprecisely in their capacity to structure knowledge.

Finally, on the third level, evaluation of the Narrative Design engages theresearcher's subjectivity in her/his analysis. That is, the interpretative analy-sis I present in this paper is as evaluative, positioning you to view theexhibition in a particular light. The interpretative stance I adopt towards theanalysis undertaken here aims to trace how From Colony to Nation naturalizesdominant conceptions of social 'reality'. It is necessary, though, to add thequalification that the point here is not to denounce the credibility of the pastrepresented in the exhibition. Indeed, the emphasis on history as an ideo-logical (re)construction throughout this paper does not mean that thosepast events recounted did not happen. Nor should it be easily conflated witha claim of historical falsity. In fact, if one takes the social constructivist viewof history seriously, notions of 'truth' and 'falsity' appear to be in flux sincethe crux of the matter now is how any single interpretation of the pastbecomes (de)legitimized, by whom and for what purposes. Further, it is the actof evaluating that is directive of one's sensibilities to the past. Herein lies thedisciplining act of history, whose representation in the museum is a form ofdirected remembering. The flipside of this selective remembering is, ofcourse, a disciplined forgetting motivated by the ideologies of the dominantin society.

Museums are then strategically placed in history making. The SFframework formulated here endeavours to be useful as some form of'meta-language' that enables visitors to 'talk' systematically about how theexhibition as a primary composite medium construes ideology. Yet, not just'talking' about, but also potentially 'talking' back to particular unequal repre-sentations displayed in exhibitions. In the final analysis, the museum repre-sents a heterogeneous zone that differentially engages multiple social playersin negotiating (or mutually disciplining) the discursive forces of socialchange. It is perhaps for this reason that the museum continues to stand as asite worth (re)visiting.


Notes

1 Foucault (1977, 1980) conceptualizes the mutual constitution of power andknowledge in social practices. As Foucault (1977: 194) has argued in Discipline andPunish: 'In fact, power . . . produces domains of objects and rituals of truth'.

2 For a more detailed consideration of the theoretical basis for extending SFtheory into the domain of multimodality, see Pang (2001: 38-54).

3 Harris (2000) first conceived the term integrational semiology to understand themultimodal character of writing. Under integrational semiology, Harris (2000:69, emphasis original) explains that 'signs . . . are not invariants: their semiologicalvalue depends on the circumstances and activities in which, in any particularinstance, they fulfil an integrational function'. Though insightful, Harris remainsvague on the what and how of this integrational function. This paper suggeststhat: (1) the metafunctional hypothesis and (2) the realizational dialectic between text andsocial context in SF theory help elucidate more concretely the shape of this inte-grational semiology.

4 Results of the survey are also reported in The Straits Times, 16 September 1996.For a sample of some of the questions asked in this survey, see The Straits Times,15 September 1996.

5 I refer to the guide here, not for an exhaustive multimodal analysis of it, but todistil the exhibition's classificatory scheme (see Table 2.2).

6 According to Fairclough (2001): 'An (interaction may involve a "chain" ofdifferent, interconnected texts which manifest a chain of different genres'.

7 Following O'Toole (1994: 35), hapticity refers to that three-dimensional qualityin sculpture which 'engages our whole body in an identification with [its] massand rhythms'.

8 For a detailed discussion on the promulgation of Shared Values as a NationalIdeology, see Hill and Lian (1995: 210—219). There are principally five com-ponents in this National Ideology: (1) nation before community and society aboveself; (2) family as the basic unit of society; (3) regard and community support forthe individual; (4) consensus instead of contention; and (5) racial and religiousharmony.

Acknowledgements

Plates 2.1, 2.2 and 2.3 are reproduced by courtesy of the Singapore HistoryMuseum, National Heritage Board, Singapore.

References

Antze, P. and Lambek, M. (eds) (1996) Tense Past: Cultural Essays in Trauma andMemory. London: Routledge.

Arnheim, R. (1982) The Power of the Center: A Study of Composition in the Visual Arts.Berkeley: University of California Press.

Bal, M. (1999) Memories in the museum: preposterous histories for today. In M. Bal,J. Crewe and L. Spitzer (eds), Acts of Memory: Cultural Recall in the Present. London:University Press of New England, 171-190.

Baldry, A. P. (ed.) (2000) Multimodality and Multimediality in the Distance Learning Age.Campobasso, Italy: Palladino Editore.


Belcher, M. (1991) Exhibitions in Museums. London: Leicester University Press.Bennett, T. (1995) The Birth qftheMuseum —History, Theory, Politics, London: Routledge.Chua, B. H. (1995) Communitarian Ideology and Democracy in Singapore. London:

Routledge.Coffin, C. (1997) Constructing and giving value to the past: an investigation into

secondary school history. In E Christie andj. R. Martin (eds), Genre and Institutions:Social Processes in the Workplace and School. London: Cassell, 196-230.

Cortazzi, M. and Jin, L. (2000) Evaluating evaluation in narrative. In S. Hunstonand G. Thompson (eds), Evaluation in Text: Authorial Stance and the Construction ofDiscourse. Oxford: Oxford University Press, 102—120.

Dean, D. (1994) Museum Exhibition - Theory and Practice. London: Routledge.Fairclough, N. (2001) Genre in Critical Discourse Analysis: Researching Language in New

Capitalism. Handout to keynote address at conference: Genres and Discourses inEducation, Work and Cultural Life: Encounters of Academic Disciplines onTheories and Practices, 13-16 May 2001, Oslo, Norway,

Foucault, M. (1977) Discipline and Punish. London: Penguin.Foucault, M. (1980) Power/Knowledge: Selected Interviews and other Writings 1972—1977.

London: Harvester Press.Hall, M. (1987) On Display: A Design Grammar for Museum Exhibitions. London: Lund

Humphries.Halliday M. A. K. (1970) Language structure and language function. In J. Lyons

(ed.), New Horizons in Linguistics. Harmondsworth: Penguin Books, 140—165.Halliday, M. A. K. (1973) Explorations in the Functions of Language. London: Edward

Arnold.Halliday, M. A. K. (1978) Language as a Social Semiotic: The Social Interpretation of

Language and Meaning. London: Edward Arnold.Halliday, M. A. K. (1994) An Introduction to Functional Grammar (2nd edition). London:

Edward Arnold.Halliday, M. A. K. and Martin, J. R. (eds) (1993) Writing Science: Literacy and Discursive

Power. London: The Falmer Press.Harris, R. (2000) Rethinking Writing. London: Athlone Press.Hernadi, P. (1995) Cultural Transactions: Nature, Self, Society. Ithaca: Cornell University

Press.Hill, M. and Lian, K. F. (1995) The Politics of Nation Building and Citizenship in Singapore.

London: Routledge.Hodge, R. and D'Souza, W. (1999) The museum as a communicator: a semiotic

analysis of the Western Australian Museum Aboriginal Gallery, Perth. InE. Hooper-Greenhill (ed.), The Educational Role of the Museum (2nd edition). Lon-don and New York: Routledge, 53-63. (First appeared in Museum 31(4) (1979).)

Hooper-Greenhill, E. (1992) Museums and the Shaping of Knowledge. London: Routledge.Hooper-Greenhill, E. (1999) Education, communication and interpretation:

towards a critical pedagogy in museums. In E. Hooper-Greenhill (ed.), The Edu-cational Role of the Museum (2nd edition). London: Routledge, 3-27.

Karp, I. (1992) Introduction: museums and communities: the politics of publicculture. In I. Karp, C. M. Kreamer and S. D. Lavine (eds), Museums and Com-munities. Washington: Smithsonian Institution Press, 1—17.

Kavanagh, G. (2000) Dream Spaces: Memory and the Museum. London: LeicesterUniversity Press.

Kress, G. and van Leeuwen, T. (1996) Reading Images: The Grammar of Visual Design.London: Routledge.


Kress, G., Leite-Garcia, R. and van Leeuwen, T. (1997) Discourse semiotics. In T. A.van Dijk (ed.), Discourse as Structure and Process. London: Sage Publications, 257-287.

Lee, T. H. (1996) The Open United Front: The Communist Struggle in Singapore 1954—1966.Singapore: South Seas Society.

Lemke, J. L. (1995) Textual Politics: Discourse and Social Dynamics. London: Taylor &Francis.

Lemke, J. L. (1998) Multiplying meaning: visual and verbal semiotics in scientifictext. In J. R. Martin and R. Veel (eds), Reading Science: Critical and Functional Perspec-tives on Discourses of Science. London: Routiedge, 87—113.

Macdonald, S. (1998) Exhibitions of power and powers of exhibition: an introduc-tion to the politics of display. In S. Macdonald (ed.), The Politics of Display.London: Routiedge, 1-24.

Maroevic, I. (1997) The museum message: between the document and information.In E. Hooper-Greenhill (ed.), Museum, Media, Message. London: Routiedge, 24—36.

Martin, D. (ed.) (1997) Museum Practice. Issue 5(2/2): 36-38.Martin, J. R. (2000a) Beyond exchange: Appraisal systems in English. In S. Hunston

and G. Thompson (eds), Evaluation in Text: Authorial Stance and the Construction ofDiscourse. Oxford: Oxford University Press, 142-175.

Martin, J. R. (2000b) Grammar meets Genre: Reflections on the 'Sydney School'.Inaugural Lecture at Sydney University Arts Association.

Martinec, R. (1998) Cohesion in action. Semiotica 120(1/2): 168-180.National Heritage Board (2000) National Heritage Board (NHB) 1998/1999 Annual

Report.Neal, A. G. (1998) National Trauma and Collective Memory: Major Events in the American

Century. London: M.E. Sharpe.O'Halloran, K. L. (1996) The discourses of secondary school mathematics,

unpublished Ph.D. thesis. Murdoch University, Western Australia.O'Halloran, K. L. (1999) Interdependence, interaction and metaphor in multi-

semiotic texts. Social Semiotics 9(3): 317-354.O'Halloran, K. L. (2003a). Educational implications of mathematics as a

multi-semiotic discourse. In M. Anderson, A. Saenz-Ludlow, S. Zellweger, andV V Cifarelli (eds), Educational Perspectives on Mathematics as Semiosis: From Thinking toInterpreting to Knowing. Ottawa: Legas Publishing, 185—214.

O'Halloran, K. L. (2003b). Intersemiosis in mathematics and science: grammaticalmetaphor and semiotic metaphor. In A.-M. Simon-Vandenbergen, M. Taverni-ers, and L. Ravelli (eds), Grammatical Metaphor: Views from Systemic Linguistics.Amsterdam: John Benjamins, 337—365.

O'Toole, M. (1994) The Language of Displayed Art. London: Leicester University Press.O'Toole, M. (1999) From systems to hypertext: navigating semiotic space in the

visual arts. Plenary paper presented at the 6th Congress of the InternationalAssociation for Semiotics Studies, Guadalajara, Mexico on Semiotics Bridging Natureand Culture.

Pang, K. M. A. (2001) Disciplining history — a multimodal analysis of the museumexhibition 'From Colony to Nation1., unpublished MA thesis. National University ofSingapore.

Pearce, S. M. (1991) Objects in structures. In S. M. Pearce (ed.), Museum Studies inMaterial Culture. Washington, DC: Smithsonian Institution Press, 47—59.

Pearce, S. M. (ed.) 1994. Interpreting Objects and Collections. London: Routiedge.Phillips, D. (1998) Photo-Logos: photography and deconstruction. In M. A.

Cheetham, M. A. Holly and K. Moxey (eds), The Subject of Art History: Historical


Objects in Contemporary Perspectives. Cambridge, UK: Cambridge University Press,155-179.

Price, D. (2000) Surveyors and surveyed: photography out and about. In L. Wells(ed.), Photography: A Critical Introduction. London: Routledge, 65-115.

Price, D. and Wells, L. (2000) Thinking about photography: debates, historically andnow. In L. Wells (ed.), Photography: A Critical Introduction. London: Routledge, 9—63.

PuruShotam, N. S. (1998) Negotiating Language, Constructing Race: Disciplining Differencein Singapore. Berlin: Mouton de Gruyter.

Ravelli, L. J. (1997) Making meaning: how, what and why? Paper presented at theconference Museum Making Meanings — Communication by Design?, AustralianNational Maritime Museum.

Ravelli, L. J. (2000) Beyond shopping: constructing the Sydney Olympics in three-dimensional text. Text 20(4): 489-515.

Royal Ontario Museum (1999) Spatial considerations. In E. Hooper-Greenhill (ed.),The Educational Role of the Museum (2nd edition). London: Routledge, 178-190.(First appeared in Communicating with the Museum Visitor (1976).)

Smith, C. S. (1989) Museums, artefacts and meanings. In P. Vergo (ed.), The NewMuseology. London: Routledge, 6-21.

Tagg, J. (1988) The Burden of Representation: Essays on Photographies and Histories.London: Macmillan.

The Straits Times (15 September 1996) Many students ignorant of Singapore history.The Straits Times (16 September 1996) Students know little of Singapore history:

survey.Thibault, P. J. (1997) Re-reading Saussure: The Dynamics of Signs in Social Life. London:

Routledge.Thibault, P. J. (2000) The multimodal transcription of a television advertisement:

theory and practice. In A. P. Baldry (ed.), Multimodality and Multimediality in theDistance Learning Age. Campobasso, Italy: Palladino Editore, 311—385.

Vergo, P. (1989) The reticent object. In P. Vergo (ed.), The New Museology. London:Reaktion Books, 41-59.

Wee, C. J. W-L. (1998) The need for National Education in Singapore. In BusinessTimes, 30-31 May 1998.

Wee, C. J. W-L. (1999) The vanquished: Lim Chin Siong and a progressivistnational narrative. In Lam Peng Er and Kelvin YL Tan (eds), Lee's Lieutenants:Singapore's Old Guard. St Leonards, NSW: Allen & Unwin, 169-244.

3 A semiotic study of Singapore's Orchard Road andMarriott Hotel

Safeyaton AliasNational University of Singapore

Introduction

Cities are more than a place to live, to work or to play in. As people observethe city while they move through it (Lynch, 1996), the city serves as apolitical and social statement, and in some cases, symbolizes andencompasses the achievement and political prowess of the country's rulingelite. This is especially true in the case of Singapore where the city becomesa showcase of what has been politically and economically achieved by thePeople's Action Party (PAP) over the years since independence in 1965.Within a span of thirty-five years, for instance, the country has achieved oneof the highest living standards in Asia, which has led some economists toproclaim it a modern miracle. Lacking in natural resources and having torely on its human resources, it was suggested that for Singapore 'the capital-ist road was [perhaps] the only one open' (Chua, 1995: 59). The number ofbuildings and shops in Orchard Road stands as testimony to the realizationof Raffles's vision of a 'bustling emporium' (Jayapal, 1992: 67). A city istherefore 'man's single most impressive and visible achievement' (Pike,1996: 243) while remaining nonetheless a 'social institution' (Mumford,1996: 184).

A city or a 'built world, like a written text, stores information' and 'pres-ents particular transformations and embeddings of a culture's knowledge ofitself and of the world' (Preziosi, 1984: 50-51). The built world is an exhibitof the culture of a given society, which in some ways reflects the ideologiesthat operate within that society. Buildings, for example, 'are not just func-tional machines; they have signs of their practical functions written all overthem: they signify their function as use' (O'Toole, 1994: 85); that is, 'buildingsare designed to mean something' (Stern, 1994: 47). Architecture is part of asociety's culture which affirms and re-establishes its values and ideals; it isthe representation of power (Betsky, 1994; Stern, 1994) and, whether posi-tive or negative, the city or the built world is the image of the community(Pike, 1996).

This paper therefore sets out to investigate the nature and manifestationof the prevailing ideologies within the society of Singapore. To achieve thispurpose Singapore is treated as a text and indeed, it is a discourse worth


investigating, analyzing and interpreting. Like a text, Singapore has beenstructurally organized but with a difference: the country is three-dimensional and multimodal. Like a text, it too leaves itself open to inter-pretation but how it is interpreted depends on one's theoretical perspective.The interpretation of a text requires the application of theory and, in thecase of a city, involves the interpretation of the integration of the variousmeaning-making resources. Although part of a larger research project(Safeyaton, 2001), due to space constraints the focus of the analysis anddiscussion in this paper is restricted to Orchard Road, that significant partof Singapore popularly known as the 'town'. It is here, and more specificallythe Marriott Hotel, where it is commonly believed that East meets West.The theoretical approach underlying these analyses is Michael Halliday's(1994) social semiotic theory of language, which has been extended to visualimages and architecture (O'Toole, 1994). From this perspective, language,visual images and architecture are viewed as social semiotic resources whichare metafunctional; that is, they simultaneously realize Textual, Interpersonaland Experiential meaning. The systemic-functional frameworks used in theanalyses are discussed in more detail below.

The semiotics of Singapore

As a city, Singapore is not static; it grows and reinvents itself according tothe changing needs and demands of its society (Tan, 1999). The physicalfeatures of a city such as Singapore, where there is constant developmentand redevelopment, are therefore not permanent. Changes to Singapore aredeemed necessary as it continues to aspire to be a 'model city', that is, a citythat is livable, attractive, business-friendly and accessible (URA Annual Report1998/1999). While national objectives must be met, the planning of a cityneeds also to consider the needs of the people who must be assured thathousing is 'affordable and comfortable', that there are 'enough public spacesto provide [them with] urban relief, that there are the 'necessary telecom-munications and fiscal infrastructure' and that there is an efficient andaffordable public transport (ibid.: 31). People should be able to move easilyfrom one designated area to another, for purposes of work or recreation.This freedom to move about permits the city dwellers to be in touch with theenvironment. As a result, a person develops a relationship with his or hersurroundings and that relationship is physical, emotional, mental, culturaland even religious.

The making and planning of modern-day Singapore, however, has beenan intensive and prolonged enterprise. Its urbanization planning began withthe formulation of the Island Concept Plan, also known as the Singapore ConceptPlan and later as the Master Plan, on 1 January 1952. The Master Plan wasaimed at regulating the development of land through plot zoning and plot/density controls. Since its implementation, and as required by legislation,the Master Plan has undergone several reviews involving various additionsand alterations. After Singapore's separation from Malaysia in 1965, the


plan was reviewed and renamed the Concept Plan in 1971 by an appointedlocal authority. The Central Area Plans came to fruition between 1974—1989only to be renamed the Revised Concept Plan in 1991. In 1998 the latestrevision of the Concept Plan was presented (Dale, 1993; Fong, 1973; MasterPlan Written Statement, 1993; Tan, J. H., 1972; Tan, S., 1999; URA AnnualReport, 1997/98).

Part of the Concept Plan's objectives is to meet what the authorities per-ceive as 'the new wants and needs of [the] people' (Dale, 1993: 42). This isaccomplished by improving the living environment and by offering, orrather prescribing, a better quality of life. This includes a policy ofdecentralizing commercial activities to avoid overcrowding in any one area,specifically the Central Area of which Orchard Road is a part (Dale, 1993).But the ideas and the benefits outlined in the Concept Plan can only besuccessfully implemented with a healthy economic growth. This becomes aplatform through which the authorities can justify their actions anddecisions both politically and in terms of the development practices. On thebusiness front, for example, one aim of the Concept Plan is to provide 12,000hectares of land for industrial needs (Keung, 1991). In addition, 'judicious'investments in the leisure industry will be welcomed because such invest-ments mean 'good business' and will 'enhance [the Singaporeans'] qualityof life' (Liu, 1991: 4). In an effort to add 'life and character' to the streets aswell as making them 'more exciting and lively' (URA Annual Report, 1997/98:28), the regulations for the setting up of outdoor refreshment areas andoutdoor kiosks along the pedestrian malls in the city were relaxed in July1996. Previously, these outlets could occupy only 10 per cent of the totalbuilding length but this is now 25 per cent, resulting in more outlets beingset up along pedestrian malls, especially along Orchard Road. These outletsbring in additional income for the authority in the form of 'payment ofdevelopment charges or different premiums' (ibid.}. Hence, every metre ofunoccupied space in, around, below and above Singapore has potential forextra revenue. This provides a boost to the economy with the Singaporeansthemselves helping to sustain that economy; the system and the peopledepend on each other.

A visitor to Singapore, however, is likely to have little knowledge of howthe country has been transformed historically although he or she may haveseen where the locals live, how they travel, where they eat, work, play, shopor seek medical attention. The visitor sees how the country 'operates' buthe or she may not be able to explain how this is possible because, morelikely than not, the visitor is not equipped with the knowledge or the toolsto explain what he or she sees or feels. For the uninitiated, Singapore'explains' its operations very well because every part of the country, be it adesignated area, its roads, the open spaces or the buildings, transmitsexplicit messages. Each of these 'speaks' to or 'addresses' the visitor dir-ectly. While part of the Singapore city has its specific functions or pur-poses, linguistically there is also a physical and Textual representationwhich transmits messages.

Table 3.1 Functions and systems in Singapore

Units/Functions

Experiential Interpersonal Textual

Area (Rank 1) Zone: north, north-east,central, west, southDistrict: CBD, non-CBDLocation: mainland, offshoreislandsTheme: business, cultural,educational, entertainment,medical, recreational,religious, residentialPortrayals: cultural, social,religiousInterplay between each themeFocal: business, cultural, social,religiousSelf-containment

SizeOrientation to generalamenitiesOrientation to members of thepublic: accessibility,affordabilityCharacterization: Oriental,occidentalSites of powerMessage

Relation to outer areasRelation to MRT stations, busterminalsRhythms: contrasting themes,building shapesRelation to prestigeCoherence and cohesion:repetition of themes, new andold buildings (preserved andconserved)

Expressways (ERP/non-ERP)RoadsFlyoversTunnelsMRT tracks (aboveground/underground)

Travelling hours: peak/non-peak periodSize and spaciousness: mainroad, minor road, slip road,one-way/two-way traffic,two/three lanesOrientation to entrant: userfriendliness, accessibility topublic transport (bus lanes),general public, fire-fighters,paramedics

Relation to other roads, MRTstationsRelation to other areasRelation to buildingsRelation to safetyExternal cohesion: relation toconnectors, escalators

Roads/MRT(Rank 2.1)

Specific functions:

Units/Functions


Orientation to buildingsCharacterization: MRTstations, bus-stops, street lights,road names, road signs,signboardsLightingOpennessSoft/hard texture: concrete,asphalt, dirt track

Open Space(Rank 2.2)

Specific functions:Road dividers, islandsRoad shouldersPavements/FootpathsParking spaceGrass verge/Green beltOpen field: recreational,businessBurial groundsPublic spacePrivate space

SpaciousnessOpennessOrientation to entrant:accessibilityViewRelevanceComfort: sheltered/unsheltered walk-ways,shades, benchesLighting: natural, artificialHard/soft textures: concrete,asphalt, grassColour

Relation to bus-stops, taxistands, roads, MRT stationsRelation to area/themeRelation to buildingsRelation to safetyRelation to power and prestigeDegree of visibilityDegree of partitionExternal cohesion: relation toconnectors, stairs, overheadbridges, pedestrian crossings,underground passagePermanence of open spacePermanence of partition


To help 'explain' Singapore, that is, to analyse and interpret the city,which is three-dimensional and multi-semiotic, a framework featuring arank-scale for the functions and systems is proposed (Table 3.1). The multi-plicity of the framework means that the city can be read 'backwards orforwards, upwards or downwards, and inside to outside' (Preziosi, 1984: 55).The framework may be used to analyse from the whole to the smallest unitin the city. This means that the semiotic analysis of the city of Singaporebegins with the unit Area at Rank 1, followed by the units Roads/MRT atRank 2.1 and the unit Open Space at Rank 2.2 (Table 3.1). The analysis ofthe smallest unit in a city, that is, Elements contained in a room or on a floorin a Building at Rank 2.3 in Table 3.2, completes the semiotic analysis.Alternatively, because the city is three-dimensional and multimodal, it ispossible to perform the analysis from the lowest rank to the highest, that is,from the unit Element in a Building (Rank 2.3) upwards to the unit Area(Rank 1). Although beyond the scope of this paper, Singapore could beconceived as the total sum of these Areas.

As buildings constitute an essential part of a city, O'Toole's (1994: 86)chart for architecture has been incorporated into Table 3.2. Although thechart has been amended to suit the Singapore context because 'the existenceof built form is not universal in all cultures' (Preziosi, 1984: 52), the changeis minimal. Elements such as the characterization of a building, that is,whether it is occidental or Oriental, for example, or how it is orientedtowards the MRT station, have been incorporated into the framework. Asmost buildings in Singapore are designed to be either self-contained (forexample, a hotel) or interdependent (for example, a market), they are treatedas individual episodes that help to contribute to or to complement the designof the whole area. In other words, there is interaction or 'interplay' betweenthese Episodes.

Functions and systems in Singapore

The built world has functions that are wittingly or unwittingly designated orprescribed. In land-scarce Singapore, the 'spatial products' or 'the builtforms' are likely to be designed to be multi-functional, that is, the practicalfunctions of a product very often overlap (Preziosi, 1984). A 'rank' or a 'unit'links each of these built forms to the other. Major roads link one area or a'unit' to another (see Table 3.1). Within an area, there are roads and openspaces, which will eventually lead to buildings where there may be differentlevels or storeys, with different rooms for different functions. Depending onits function, each room may have a different layout or decor. While thepractical functions of an area or a building are considered to realizeExperiential meaning, the relationship between these practical functions andits design or planning are Textual. The consistency and the repetition of aspecific theme in a particular area in Singapore means that textually it hasbeen designed to 'blend' and to 'fit' and construct the culture of the people.Each unit operates or functions in relation to another, usually a neighbour-


ing one, and to its general surroundings or environment. At the same time inthe built world, the 'built forms' (Preziosi, 1984) will direcdy or indirecdycommand the people's involvement and interaction; our senses respond inspecific ways to our natural environment (Kress, 2000). The framework inTable 3.1 lists the systems which function Interpersonally to engage us withour environment. In what follows, I analyse and interpret the built forms ofthe Orchard Road and the Marriott Hotel. These analyses reveal howspecific ideologies are manifested in the city of Singapore.

A semiotic analysis of Orchard Road

Commercial developments in Orchard Road began in the early 1900s whenstores were established to provide residents with fresh produce and foodsupplies. In the 1950s, when the late C. K. Tang opened a department store,it marked the beginning of rapid commercial development within the area.Entertainment centres and hotels were soon built to cater to the demands ofthe locals as well as the buoyant tourist industry. The construction ofOrchard and Somerset MRT stations and their locations within the CentralBusiness District reaffirmed Orchard Road's importance and its status as a'dynamic activity corridor' (Orchard Planning Area, 1994: 9).

Orchard Road, a seven-lane and one-way-traffic road, stretches fromDelfi Orchard to Plaza Singapura and is an area where open parking spacesfor cars are limited. Most of these parking spaces are found within theconfines of hotels or shopping malls where parking fees are high. Textually,this demonstrates the area's 'relation to prestige' and Orchard Road isindeed a prestigious area where public Housing Development Board (HDB)flats are not found. Building heights reach their maximum in the vicinity ofOrchard MRT station where buildings reach thirty storeys high. The heighttapers to ten storeys towards the Singaporean Presidential Palace or Istanaand twenty storeys towards the Tanglin zone (Orchard Planning Area, 1994)(Plate 3.1). We may note that the Size and Verticality of a building is anindication of its importance and status (O'Toole, 1994). Metaphorically, thevalue of commercial or cultural activities reaches their peak at the junctionof Scotts Road and Orchard Road where the Marriott Hotel is located.Suffice to say the Electronic Road Pricing (ERP) begins at this junction.Indeed, the lack of parking spaces means people are encouraged to use 'theefficient and affordable' public transport (URA Annual Report, 1998/1999:31) to reduce traffic congestion and to solve parking problems.

On both sides of the road, Open Spaces such as the wide pedestrian walk-ways facilitate smooth pedestrian flow but people are not induced to visit orto walk along these Open Spaces if there is no place to sit (Whyte, 1996).Therefore practical Interpersonal elements such as benches, which are madeof durable and maintenance-free concrete or granite or wrought iron, areselectively provided. People are socially engineered 'to get into new habits'(Whyte, 1996: 111) such as walking rather than driving because of the lackand excess of particular types of Open Spaces along Orchard Road. The

Table 3.2 Functions and systems in a building (adapted from O'Toole 1994: 86)

Units/Functions


Building(Rank 2.3)

Floor

Practical Function: Business,Cultural, Educational,Entertainment, Governmental,Medical, Private/Public,Recreational, Religious,ResidentialOrientation to lightOrientation to windOrientation to earthOrientation to service (water,power)Episode: self-contained,interdependentInterplay of episodes

Size (relation to area andsetting)Orientation to neighbours,adjacent buildingsOrientation to road, MRTtracks and stationsOrientation to entrantFacadeModernityColourCladdingCharacterizationColourIntertextuality: reference,mimicry, colourExoticism

Proportion (height/breadth/length)Relation to external areaRelation to road/MRT stationRelation to adjacent buildingsRelation to permanence: old,new, preservation,conservationRhythms: contrasting shapes,angles, coloursTextures: rough/smoothRoof/wall relationOpacityReflectivityCohesion: interplay of episodes

Sub-functions:AccessWorkingSellingAdministrationStoringWakingSleepingParking

HeightSpaciousnessAccessibilityOpennessViewHard/soft textureColourSites of powerSeparation of groups

Relation to other floorsRelation to outer worldRelation to connectors, stairs,lifts, escalators (externalcohesion)Relation of landing/corridor/room/foyer/room (internalcohesion)Degree of partitionPermanance of partition

Units/Functions


Room Specific functions:AccessEntryLobbyDiningBedroom suitesBathroom/toiletFitness centre /gamesroomRestaurants, bar, loungeKitchenEnsuiteServeryFoyerLaundryRetreat

ComfortLightingModernitySoundOpulenceWelcomeStyle: rustic, pioneer, colonial,suburban, 'Dallas', workingclass, tenement, slumForegrounding of functions

ScaleLightingSoundRelation to outside worldRelation to other roomsConnectors: doors, windows,hatches, intercomFocus (e.g. hearth, dais, altar,desk)

Element Lighting: windows, lamps,curtains, blindsAir: window, fan, conditionerSound: carpet, rugs, partitions,acoustic, treatmentSeating: function, comfortTable: buffet, dining, coffee,computerCounter: cash, reception, bar

RelevanceFunctionality: convention/surpriseTexture: rough/smoothNewnessDecorativenessStanceStylistic coherenceProjection

TexturePositioning: to light, otherelementsFinish

Plate 3.1 The shopping map from This Week Singapore


area appears to rely on the concept of'supply creates demand' (ibid.); that is,the restricted nature of parking spaces and the open spaces create thedemand for public transport.

The built forms in Orchard Road place a great emphasis on Interpersonalmetafunction. Commercial developments along the pedestrian routes areencouraged to 'have activity-generating uses on the [ground floor]' (OrchardPlanning Area, 1994: 20) and as a result, shops and restaurants open directlyto the mall, beckoning pedestrians as well as supporting street activities. Theopen spaces are planned so that people instinctively walk into the air-conditioned interiors of the shopping malls to escape the humidity of theoutdoors. The 'progression from street to interior is critical' (Whyte, 1996:117) and Orchard Road has been planned such that it is hard to tell whenone transition ends and when the other begins. Pedestrians also have visualaccess to the products on sale at the ground floor shops, which are encasedbehind glass panels. Window displays are usually used to attract the attentionof the female pedestrians and cater first to what are perceived as the primaryneeds of women: cosmetics, fine jewellery, clothing as well as their coordin-ated accessories while condoms at the Lucky Plaza shops are arranged toresemble a bouquet of flowers. A major part of the business strategy is tocapture the female eye first. Seen textually, sex implicitly becomes the sellingpoint in Orchard Road in what largely remains a patriarchal society.

The presence of overseas investors in Orchard Road is ubiquitous andthus there is a reinforcement of the culture of consumerism. For example, atthe time of writing, twenty-five outdoor refreshment outlets (OROs) arelocated along Orchard Road. Located on both sides of the road, theseoutlets serve coffee and tea and food such as burgers and fries; that is,foreign imports from the West. It is common to see several oudets promotingthe same items but under different trade names. Patronizing these oudetshas become a way of life. These OROs have built a 'new constituency'(Whyte, 1996: 111) where people are subconsciously trained to adopt newhabits such as having alfresco lunches. These outlets also act as an avenuefor the people to see and be seen and this has given rise to a new streetculture that is readily embraced. As competition among the various inves-tors intensifies, 'campaigns' are launched to remind consumers, particularlythe young, of the products' existence, which are readily accessible and avail-able to them. Hence, these OROs are located a few hundred metres awayfrom one another. While these outlets operate textually because they con-tribute to the thematic 'consistency' of Orchard Road, they have whatO'Toole describes as 'powerful [and serious] Interpersonal implications'(O'Toole, 1994: 103). The 'repetition of themes' ensures that people wouldnot miss or forget these products. To invest in the young and impressionableis therefore to invest in the future of Singapore. Such investments guaranteethe survival of these products and the continuous Western presence. Equallyimportant, these OROs continue to draw revenues for the authorities.

Ironically though, while these OROs are located at strategic and primelocations, that is, they are in the Open Space and visible from the road, outlets


serving local Singaporean fare are usually confined within a building, oftenat the basement or the back of the building or at a side road and away fromthe main road. Although the nature of Asian cooking is highly suitable forthe outdoors, it does not or rather is not allowed to fit into the contextof Orchard Road. Textually, a conscious effort has been made to ensureOrchard Road projects and reinforces the sophisticatedly developed cleanand green image that has become synonymous with the image of Singapore.The fact that ideas for these outlets were imported from overseas (URAAnnual Report, 1998/99) and are expected to 'make our streets more excitingand lively' while 'adding life and character to our streetscape' (URA AnnualReport., 1997/98: 28) suggests the relative value of Asian culture. The hotelsmay not necessarily cater solely to European tourists, but nevertheless, aforeign culture is foregrounded while its Asian counterpart is backgrounded.The message is clear: anything foreign, imported and specifically Westernexcites and sells readily.

Unlike Geylang or Serangoon Road in Singapore, there is a conspicuousabsence of religious symbols along Orchard Road even though a prayer hallfor the Muslims is located off the main street in Bideford Road. The vicinityis thus constructed to be secular, but not necessarily apolitical. CaliforniaFitness Centres and Planet Hollywood have made their presence felt inOrchard Road along with Singtel and the Safra Town Club. Unlike thesevibrant institutions whose open concept invites pedestrians to browse, theThai Embassy appears inaccessible behind its iron gates and thick foliage. Asthe lowest building there, the embassy does not fit into the concept ofOrchard Road because it does not generate sales or draw in the crowds. Itmars the overall outiook and thematic concept of Orchard Road and weinterpret it as 'failing' textually. In contrast, the Singaporean PresidentialPalace or the Istana, situated at the end of Orchard Road and not visiblefrom the main road, is designed to attract attention. The changing-of-the-guards ceremony has found favour with both the locals and the visitors. Thisappeal can be translated as a desirable Interpersonal relation. Officiallyclosed to the public throughout the year, however, the grounds are openedon designated public holidays.

Streetscapes such as road signs, street lights and bus shelters appear to beneutral, but closer inspection reveals a different scenario. While the streetsigns are in English, the ethnic group whose presence is strongly representedis Chinese. The architecture of the Marriott Hotel is an example of howthat presence is reinforced and preserved. Such buildings serve to remindSingaporeans of their cultural heritage. The one reminder of a multi-racialsociety is the mural wall located next to the entrance to Orchard MRTstation where foreigners, especially the Filipinos, congregate on Sundays.This mural wall depicts the cultural activities and the various landmarksassociated with the four main ethnic groups in Singapore. Discotheques andpubs are discreetly placed in various corners of buildings and roads, awayfrom the public eye during the day. However, at night, these entertainmentcentres spring to life, while out in the street the action continues. Orchard


Road has fulfilled the expectations and has realized the vision of the author-ities to create 'a modern and vibrant commercial corridor alive with day andnight activities' (Orchard Planning Area, 1994: 14).

Hence, it is immaterial whether the people are indoors or outdoors. InOrchard Road, people are constantly on the move and wherever they maybe, there are ATM machines for them to withdraw their money and at thesame time, an outlet for them to spend it. Every visitor to Orchard Road is apotential customer. Regardless of the time of day, one can be assured thatthere are cash transactions in Orchard Road. Textually then, the retailindustry has been successfully turned into one of the cultures of OrchardRoad. Except for Ngee Ann City, no other shopping mall has been promin-ently featured in postcards, the one form of communication that 'person-ally' connects Singapore and its visitors to the other parts of the world.Examination of the postcards available in the local shops reveals that apostcard of Orchard Road often includes Tang Plaza and the SingaporeMarriott Hotel, usually photographed from various angles and at differenttimes of day. This inevitably enhances the hotel's status but most import-antly, it transforms the hotel into the landmark of Orchard Road.

A semiotic analysis of the Tang Plaza

A landmark is observed externally and serves as a point of reference or aclue of identity. The choice of a landmark, according to Lynch (1996: 102),is 'more easily identifiable' if it is 'significant', has 'a clear form', 'contrast[s]with the background' and 'if there is some prominence of spatial location'.Hence, I have chosen to investigate the Tang Plaza/Marriott Hotel (hence-forth known as 'the complex') as a landmark.

Built in 1982, the 33-storey Singapore Marriott Hotel was formerlyknown as the Dynasty Hotel. Acquired by the Marriott Group in 1995, itunderwent extensive interior redecoration and renovation and has sinceoperated under its present name (The Straits Times, 4 September 2000: 42).Its strategic location ensures that every vehicle or commuter travellingdown from Tanglin, Scotts and Paterson Roads passes by it. It is situated ata location where the ERP begins and where vehicles stop at the trafficlights, the first of the four traffic junctions along Orchard Road. An under-ground passageway links the Plaza to the Orchard MRT station and theother buildings across from the hotel, namely Shaw House and WheelockPlace. Hence, no matter what mode of transportation one uses, the com-plex is highly accessible and visible to the public. Its prominent location,that is, its Orientation to the people and its Relation to the road andMRT station, which are Interpersonal and Textual functions respectively(Table 3.2 above), have been translated into a form of visual and massiveadvertisement that gives the complex an exposure not accorded to anyother shopping centre or hotel along the road. Additionally, its Size andVerticality acts as a 'clear indication of [its] status' in the vicinity (O'Toole,1994: 102).


What make the complex more significant are its colours and its pagoda-like architecture. Using the framework for architecture in Table 3.2, choicesfrom systems for Interpersonal meanings feature strongly in the design ofthe hotel. For example, as an illustration of the functions of the units Mod-ernity and Colour, its former owners had deliberately chosen its presentdesign to reflect their racial and cultural heritage and, although only eight-een years old, its design is representative of the days of ancient China. As faras Colour is concerned, the dominant colours in the vicinity of OrchardRoad are blue and brown, as in the Forum the Shopping Mall, Wisma Atriaand Ngee Ann City, but, at the complex, the traditional Chinese colours,green and red, dominate both the roof tiles and columns of the building.Unlike the other hotels, which were designed to resemble vertical rect-angular blocks that occupy extra lateral space, the Marriott Hotel isoctagonal in shape and is a tall and lean building with a distinctive Fa£ade,which is a conical top and upturned roof-ends that point towards the sky (seeFigure 3.2). The contrast in building shape and colour is grouped under thecategory of the unit Rhythms that operates textually In addition, theappearance of the complex has been likened to 'a decent Oriental gentle-man' and conferred as 'a trustworthy place' (Gwee, 1991: 62—63). We notethat the number 'eight' and the colour 'red' are considered lucky and sym-bolize prosperity within the Chinese community. Such beliefs or practicesare related to a community's social semiotic, which operates Interpersonally.

However, because of its pagoda-like structure and octagonal shape, thedesign of Tang Plaza and Marriott Hotel is not consistent with the overallenvironmental and architectural structure of Orchard Road. In otherwords, the complex does not 'exhibit some kind of "fit" with their neigh-bours and neighbourhood' (O'Toole, 1994: 87). Although this is apparentlydeliberate, textually the inconsistency could be said to 'fail' or be 'undesir-able'. This Textual failing means, of course, that Interpersonally the buildingattracts attention. The shape of the Marriott Hotel is only prominent froman aerial view (see Figure 3.2). At eye-level, due to its orientation, distance toand accessibility from the main road, the Tang Plaza is more distinct (seeFigure 3.1). This disparity may be partly due to proportionality in Size, asystem that operates interpersonally. The hotel seems to be sitting on a basethat is too broad for it (see Figure 3.2) and unlike the Tang Plaza, theMarriott Hotel is backgrounded. The hotel proper is built in the centre ofthe Plaza, which means that it is actually distanced from the main road.From the environmental point of view, and both interpersonally and text-ually, this location acts as a buffer to the noise generated by the traffic.Nevertheless, the hotel draws attention to itself due to its unique roofdesign. One needs to raise one's head to view the hotel, and what is first seenat ground level is the red and green roof (see Figure 3.1). In sum, theIntertextuality or the difference in overall design, mismatch in size and thecolour scheme gives the building its Oriental character, one that providesthat significant 'contrast with the background' (Lynch, 1996:102) which isOrchard Road. These differences have naturally proven to be advantageous


Figure 3.1 Front view of the Marriott Hotel


Figure 3.2 Three-dimensional view of Tang Plaza

because these are the features that are constandy highlighted in variouspostcards and travelling brochures.

The complex is a building with a hotel and the four-storey Tangs Super-store built as a whole unit. Experientially, there are two Episodes operatingsimultaneously at the Tang Plaza. One Episode is that of a hotel and theother, a shopping centre. Each is a different entity but one which has beenintegrated and superimposed over the other. Each Episode serves its ownfunction: the hotel provides lodging, food and entertainment, whilethe shopping centre is part of an industry that is responsible for shapingSingapore into the commonly perceived shoppers' paradise. Both cater tothe needs of the foreigners as well as the locals and fit into the concept of'under one roof; that is, shopping, dining, entertainment and lodging withinthe same building. This provides the Textual Cohesion in the Episodes. This


Cohesion is also responsible for the great interplay and interaction betweenthe two Episodes because, seen experientially, for the uninitiated at least, it ishard to predict where the shopping centre ends and where the hotel begins.Textually, the foregrounding and the prominence given to Tangs Superstoreensure that the complex fits into the overall thematic concept of OrchardRoad. Unlike the thematic Malay Village in Singapore, which was designedto promote the Malay culture as a form of tourist attraction, the Tangcomplex has proven to be a successful social, cultural and economic venture.

Even though the complex is located at the junction of Scotts and OrchardRoads, vehicle access or the Interpersonal salient Orientation to entrant tothe complex is only from Scotts Road. A slip road branching out from ScottsRoad leads to the hotel main entrance and subsequently to the mainentrance of Tangs Superstore. For those using public transport, a bus stopand an underpass to Orchard MRT station are conveniently located oppos-ite the entrance of Tangs Superstore giving commuters, who are also pro-spective customers, direct access to the shopping centre. The whole complexis slightly elevated from the main road, which metaphorically puts it in aposition of power or superiority. The protruding roof of the Tang complexprovides a much-needed shelter from both sun and rain while its red col-umns act as advertisement boards. The width of the pedestrian walkwayskirting the complex indicates that a heavy human traffic flow is anticipated.Therefore, the open spaces around the complex are put to efficient use.Benches are provided while OROs, such as Mrs Fields' and Juice & Java,provide quick snacks and drinks. Textually, unlike in most parts of Singapore,there is a sloping ramp that caters to the needs of the physically handi-capped or those who are wheelchair-bound. And in case pedestrians forgetthat the hotel is an octagonal-shaped building, this has been permanentlyimprinted on the non-slip tiles of the walkway skirting the complex, while anoctagon circumscribes each column of the complex on the roof. Like thebuilt form of Orchard Road, there is an overwhelming emphasis on theInterpersonal function at the complex.

Textually, in keeping with the green image of the area, low-lying shrubsand palm trees signifying 'a tropical island' line the perimeter of the com-plex. The hotel entrance, however, has the thickest shrubs. Interpersonally,other than complementing the colours of the hotel and enhancing its land-scape, these plants shield the hotel guests from the main road, providing alittle privacy. The names of the complex's main tenants, that is, 'Marriott'flanked by 'Tang' on either side, are mounted on the wall facing Scotts Road,giving the impression that each is vying for the attention of the onlookers. Ifone were to miss the hotel's name, the situation has been rectified through aconcrete signboard. This signboard 'announces' its presence in the vicinityas it is erected directly opposite the hotel entrance and thus faces towardsthe junction of Scotts/Orchard/Paterson Roads. Such a signboard, onethat is not part of a hotel proper and located in an open space, is the onlyone found in the area. Others, if available, are usually located within thehotel's premises.


A semiotic analysis of the Singapore Marriott Hotel

Upon arrival at the steps of the hotel entrance, what first attracts the atten-tion of a guest are the open-air sidewalk cafes on the left and on the rightand the side entrance to Tangs Superstore (see Figure 3.3). In other words,the hotel's outdoors scenery and the activities generated from and around itfunction to distract the guest from his or her intended destination. This canbe attributed to two factors. First, at the unit of Floor - colour for Inter-personal function, yellowish marble tiles are used for the floor at the hotelentrance and black and grey tiles for the walls. While the tiles may add atouch of class or sophistication and facilitate maintenance, the colours paleagainst the eye-catching colours of the sidewalk cafes and the OROs wherered dominates. Second, and more importantly, there is a considerable dis-tance between the complex's driveway and the hotel main entrance.Although it is directly facing the driveway, the entrance or the Access to thehotel appears hidden from the view of the guest.

Flanking the passageway leading to the hotel's main entrance are small-scale water fountains, which extend into the hotel. Fengshui, the art ofcomprehending how the natural energy of life affects us in our daily life(Gwee, 1991; Noble, 1994), seems to have played a role in the layout of thehotel's ground floor. However, depending on one's social and cultural back-ground, it could also be argued that these fountains are for aesthetic reasons.Though O'Toole (1994: 90) has expressed uncertainty of its actual placewithin the framework of his chart for architecture, fengshui, I feel, should becategorized as Interpersonal. As O'Toole has clearly stated, however, itdepends greatly on one's social semiotic. Noting this, water, being one of thefive elements of nature, the others being earth, wood, fire and metal, sym-bolizes wealth in the Chinese culture and its employment is intended topromote fortune (Gwee, 1991; Noble, 1994). Contextually, however, thefountains could mean different things to different people. For children, theyare a source of entertainment because they may find pleasure in dippingtheir hands into the water. To the hotel, the stream of water brings the hopethat success and prosperity continue to flow towards it. This is reinforced bythe motifs on its floor tiles. Here, the two unbroken concentric circles, whichcould signify smoothness in perhaps business dealings and continuous pros-perity, are divided into four segments, presumably representing the fourcorners of the world where the Marriott Group operates. The circles, how-ever, could also be a representation of the Luo Pan Compass that is used infengshui to determine the siting and building dynamics (Gwee, 1991; Noble,1994). This circular pattern is also repeated on the false ceiling. Seenexperientially and at the rank of Element, this false ceiling and the fountainconceal the light bulbs at the entrance where the lighting remains soft andwarm. Interpersonally, this provides Warmth and Comfort to the guests.Finally, orchids are used to brighten the passageway and to offset thedullness of the walls and floor. On the whole therefore, the entrance ofthe hotel, which acts as the introduction to the hotel, employs various


Figure 3.3 Floor plan of ground floor of Marriott Hotel


Interpersonal features or strategies to provide the guests with what are per-ceived as the necessary comforts. Guests are expected to respond visually,auditorily, mentally as well as emotionally to their immediate surroundings(Kress, 2000), and, in this case, to the soft lighting, to the orchids and to thesoothing sounds of the flowing water from the fountains.

The layout of the Foyer of the Marriott Hotel is unique because it doesnot conform to the standards adopted by the various hotels in the vicinity. Atthe rank of Element and for the Experiential function, for instance, the Foyeropens to the sky and, as described in its promotional brochure, is 'illumin-ated by a three-storey skylight' thereby reducing the reliance on artificiallightings while at the same time giving the air-conditioned lobby an airyatmosphere and good ventilation. No chandeliers are needed, just wall-mounted lamps and table lamps placed at strategic locations. The warm andsoft lightings are easy on the eyes. The walls and floor are bare as carpetsand ornaments or decorations such as paintings are kept to a minimum.Instead both the floor and the walls are fully tiled and of similar shades.Though this means easy maintenance, as the cleaning and mopping processis easier, the Foyer exudes coldness and appears businesslike. Interpersonalfunctions or considerations such as Warmth and Comfort appear to havebeen backgrounded.

At the Foyer, guests are not greeted by the traditional Sites of power,that function interpersonally at the rank of Floor. This is usually the recep-tion counter where the initial scrutiny of a guest takes place. Instead guestsare 'greeted' by an escalator or a 'connector' leading to the second floor ofthe hotel where the banquet rooms and restaurants are located. Sign-boards displaying the names of the banquet rooms and restaurants areplaced at the foot of the escalator. Thus, guests need not seek directions,thereby alleviating labour costs. Inevitably this reduces the human inter-action between guests and hotel staff. For an establishment that deals withthe service industry, Interpersonally, this is interpreted as another setback.The distance between guests and hotel is further widened by the locationof the reception counter, which is located at the far end of the lobby andsandwiched between its side entrance and its emergency exit. Guests eitherapproach the counter by walking across the Foyer (Path 'A' in Figure 3.3)or by passing the jewellery, pastry and cigar shops along the passagewayon the right (Path 'B' in Figure 3.3). Initially the location of the counter,which is part of the hotel's welcoming team and the human face of theestablishment, appears inconvenient to the guests but this apparentinconvenience is negated because of the close proximity of the lifts thatwould eventually lead the guests to their rooms. What can be deducedhere is that at the Foyer, the foregrounding of Textual functions such as theRelation of the lifts to the reception counter and the Relation of escalator tosignboards, far outweighs the Interpersonal functions such as Comfort, Wel-come, and human contact with hotel staff. This also functions to makesurveillance and the official scrutiny of the guests implicit rather thanexplicit.


The vertical as well as the horizontal space or the Spaciousness of theFoyer, which is a system for Interpersonal meaning, is striking and theabsence of physical or permanent Partition in this area suggests an open-concept approach to business. This openness allows guests to move andinteract freely around the Foyer. Guests arriving on the first day, forexample, are free to view the food and beverage menus at the cafe's reserva-tion counter. Simultaneously though, these guests become easy targets ofscrutiny by the hotel's security personnel or even by other guests. If therewere partitions on the ground floor of the hotel, this would not so readilyoccur. Except for the Marriott Cafe, which is slightly elevated, the Separ-ation of groups is achieved by utilizing the square or circular columnsaround the perimeter of the Foyer. These columns help to distinguishbetween one group of specific activity from another, such as the bar counterfrom the Lobby Lounge and the Lobby Lounge from the reception lobby.Seen textually, the only Permanent Partition at the Foyer is the hotel lifts,which are hidden behind four square columns. These columns and lifts actas markers to indicate the end of general public activities, such as drinkingand dining. They shield one's view from the most unpleasant sights or spaceson the ground floor but the ones that cater to a guest's more personal needs,that is, the passage to the parking lots and the restrooms. Understandably,there is a need for the hotel to put forward its best 'face' to the public andthis has been done overtly. There is a distinct separation between public andprivate needs or spaces with the former usually foregrounded. Judging fromthe location of the Business Centre, a guest's professional needs seem to fallwithin his or her private domain; it is located next to the emergency exit andadjacent to the lifts. On the priority scale, the size of the Centre suggeststhat 'business' or 'work' should constitute a major part of a guest's privatelife but at the Marriott Hotel, it retreats to the background.

The main physical attraction or distraction at the Foyer (depending onhow one chooses to see it) is the nine preserved palm trees placed almost inthe middle of the Foyer. As a Textual element, these trees serve as the 'focus'in the area known as the Lobby Lounge. The theme of a 'tropical island' isthus brought into the interior of the hotel. Hence there is continuity or aconstant repetition of themes in and around the hotel. The preserved palmtrees in the Lobby Lounge are encircled by four semi-circular flower troughswhere, once again, orchids are the choice plants. An artificial garden city orisland is thus created within the hotel's premises. An aerial view of the LobbyLounge suggests that it is the physical representation or the built form of themotifs found on the floor tiles at the hotel entrance. The open spaces underthe palm trees and around the flower troughs are also fully utilized. Theseare used as a dining area where diners are served drinks from the barcounter, food from the Marriott Cafe and cakes and pastries from the PastryShop. In fact, there is an overwhelming emphasis on the food and beveragebusiness at the hotel. The consumption of alcohol also seems to be widelypromoted and encouraged. There are bar counters located at the Foyer,at the sidewalk cafe and at the underground pub, Bar None. Guests are


continuously confronted and surrounded by food and drinks and if that isnot enough, the open-concept kitchen at the Marriott Cafe gives them a viewof how food is prepared for consumption. At the same time, the glass panelsat the Cafe allow patrons to view what the other diners at the Crossroad Cafeare having and vice versa.

The change in hotel management in 1995 did not affect the MarriottHotel physically or structurally, because it remains culturally Chinese. Theformer Dynasty Hotel, as its name suggested, had created an image of theexistence of Chinese imperialism and a dynasty of Chinese culture andtradition within the Orchard Road vicinity. This is the image that is capturedin postcards and promotional brochures to represent, ironically, a multi-racial and a multicultural society, a fact that has often been stressed by thegovernment. The interior of the hotel, however, does not reflect its culturalheritage and dominance, as it is more occidental than Oriental. The layoutof its ground floor unwittingly reveals the hotel's business philosophy anddemonstrates how it chooses to construct itself in the context of Singapore.Initially, there is seemingly a lack of customer service in the hotel because,like the reception counter, the concierge and tour desks are pushed towardsthe recesses in the walls of the Foyer. This reduces obstruction and creates aclear passageway but at the same time it pushes customer service to thebackground. But 'reducing obstruction' or eliminating 'tripping hazards'such as carpets is a 'service' in itself and this aspect of service belongs to theall-important department in every industry, that is, safety. UnfavourableInterpersonal functions such as the Orientation to entrant, which is the dis-tance between the hotel entrance and the reception counter, are oftennegated by other Experiential and Textual functions such as the Relation toconnectors, that is, the close proximity between the reception counter andthe lifts.

From a popular point of view, a big part of the success of the hotelappears to be its understanding and its knowledge of what the public wantsand by giving and capitalizing on those wants. There is an impression thatthere is something for everyone. There is day and night entertainment,jewellery and chocolates for the women, cigars for the men and food foreveryone. The Lobby Lounge in particular has captured the constructedspirit and image of Singapore and what is being projected is an image of anideal tropical island where palm trees flourish under the warm sun. Singa-pore is constructed as a city where food is in abundance and seen as apassion, and where eating is perceived to be a favourite pastime. But becauseof its open concept and the lack of permanent partition, another side of thehotel becomes invisible to the public eye. Women especially are vulnerableto the male gaze. While soliciting is an offence that is punishable by law andis an activity usually and wrongfully associated with women, the locations ofthe hotel side entrance, its emergency exit and the corridor that leads to thelifts provide the discreet routes (Path 'B' in Figure 3.3) for the guests whowish to bring in additional company. Similarly, the escalator to the hotel'ssecond floor does not only connect guests to the restaurants and banquet


rooms but also to the lifts on that level. In the same manner too, the cigarshop and the underground pub are sites for discreet soliciting. These are,however, located away from public viewing. What is further implied is thatseeking pleasures and entertainment is the prerogative of the male. Theopen concept in the hotel, which symbolizes one's public image, reflects aclosure to reality or to one's private life; it demands discretion because therestill remains the Asian obsession with the subject efface'.

Conclusion

While the analysis of Orchard Road entails the construction of a frameworkthat features a rank-scale with the functions and systems through whichSingapore is constructed, the analysis of the Marriott Hotel requires theapplication of O'Toole's (1994: 86) framework for architecture. Throughthe integration of both frameworks, the analyses of both Orchard Road andMarriott Hotel reveal how spaces in and around Singapore are carefullyorganized to meet the sociopolitical and socio-economic demands of theauthorities. Every available space is found to be potentially economicallyviable. In general, the analyses reveal how Singapore is constructed as ashopper's paradise, a tropical island and a food haven. What is presented,however, is a constructed image of a country and a hotel that both theauthorities and the management want the public and the world to see and tobelieve. How this is done requires, to a certain degree, the use of women ascommodities. The general perception in Orchard Road and the MarriottHotel is that sex sells, thus reflecting the values of a patriarchal society.

Orchard Road demonstrates how foreign cultures, specifically those fromthe West, are foregrounded and how the cultures of Singapore's multi-racialsocieties are backgrounded. This is perhaps part of a strategy to cater to theinflux of 'foreign talents' and tourists to the country. Business concepts suchas the outdoor refreshment areas, for example, are imported from overseasin an effort to make the streets 'more exciting and lively' (URA Annual Report,1997/98: 28). The concepts of excitement and liveliness are thereforedenned by the authorities and Singaporeans are socially engineered to sub-scribe to these prescribed concepts. These oudets as well as the abundanceof shopping centres in the vicinity are in reality revenue-generatingmachines. Profit-making is the key word; the culture of consumerism dom-inates the area and capitalism is seen as the answer for a land reliant onhuman resources. The presence of the Marriott Hotel, however, serves toremind the people, whether locals or foreigners, of Singapore's culturalheritage. The building is a potent social and cultural symbol and a reminderof the prominence of the Chinese community in the country. Amidst thechaotic cultural scene in Orchard Road, Singaporeans must be reminded oftheir cultural heritage and to meet those expectations, a system is imple-mented and a lifestyle prescribed. The system and the people depend onone another.


Acknowledgements

The map (Plate 3.1) is provided courtesy of This Week Singapore.

References

Betsky, A. (1994) James Gamble Rogers and the pragmatics of architectural repre-sentation. In W. J. Lillyman, M. E Moriarty and D. J. Neuman (eds), CriticalArchitecture and Contemporary Culture. New York: Oxford University Press, 64—84.

Chua, B. H. (1995) Communitarian Ideology and Democracy in Singapore. London:Routledge.

Dale, O. J. (1993) The Singapore Concept Plan: historical context/current assess-ment. PLANEWS. Journal of the Singapore Institute of Planners 14(1): 41-46.Singapore: Straits Printers Pte. Ltd.

Fong, T W. (1973) Industrial complexes and the garden city — can they co-exist? InChua Peng Chye (ed.), Planning In Singapore - Selected Aspects and Issues. Singapore:Chopmen Enterprises, 16-21.

Gwee, P. K. W. (1991) Fengshui: The Geomancy and Economy of Singapore. Singapore:Shing Lee Publishers Pte Ltd.

Halliday M. A. K. (1994) An Introduction to Functional Grammar (2nd edition). London:Arnold.

Jayapal, M. (1992) Old Singapore. New York: Oxford University Press.Keung, J. (1991) Overview on the Concept Plan. Living the Next Lap —Blueprintsfor Business.

Singapore: Urban Redevelopment Authority.Kress, G. (2000) Multimodality. In B. Cope and M. Kalantzis (eds), Multiliteracies:

Literacy Learning and the Design of Social Futures. South Yarra: Macmillan PublishersAustralia Pty Ltd, 182-202.

Liu, T. K. (1991) Press Release on Living the Next Lap — Blueprints for Business. Singapore:Urban Redevelopment Authority, 1—5.

Lynch, K. (1996) The city image and its elements (first published 1960). In R. T.LeGates and E Stout (eds), The City Reader. London: Routledge, 98-102.

Master Plan. Report of Survey Volume 1. (1955) Singapore: E S. Horslin, GovernmentPrinter.

Master Plan Written Statement 1993. The Planning Act (Cap 232, revised edn 1990).Republic of Singapore. Singapore: Ministry of National Development.

Mumford, L. (1996) What is a city? (first published 1937). In R. T. LeGates andE Stout (eds), The City Reader. London: Routiedge, 183-188.

Noble, S. (1994) Feng Shui in Singapore. Singapore: Graham Brash (Pte) Ltd.Orchard Planning Area: Planning Report 1994. Singapore: Urban Redevelopment

Authority.O'Toole, M. (1994) The Language of Displayed Art. London: Leicester University Press.Pike, B. (1996) The city as image (first published 1981). In R. T. LeGates and

E Stout (eds), The City Reader. London: Routledge, 242-249.Preziosi, D. (1984) Relations between environmental and linguistic structure. In

R. P. Fawcett, M. A. K. Halliday, S. M. Lamb and A. Makkai (eds), The Semiotics ofCulture and Language Volume 2. Language and Other Semiotic Systems of Culture. Dover,New Hampshire: Frances Pinter, 47-67.

Safeyaton, A. (2001) The Lion City as a text - a semiotic study of Singapore'sOrchard Road and Marriott Hotel. Unpublished MA dissertation. The NationalUniversity of Singapore.


Stern, R. A. M. (1994) The postmodern continuum. In W. J. Lillyman, M. EMoriarty and D. J. Neuman (eds), Critical Architecture and Contemporary Culture. NewYork: Oxford University Press, 46—63.

Straits Times, The, 4 September 2000, 42.Tan, J. H. (1972) Urbanization Planning and National Development Planning in Singapore.

SEADAG Papers On Problems of Development in Southeast Asia. New York: the AsiaSociety-SEADAG.

Tan, S. (1999) Home. Work. Play. Singapore: Urban Redevelopment Authority.URA Annual Report 1997/98. Singapore: Urban Redevelopment Authority of

Singapore.URA Annual Report 1998/99. Singapore: Urban Redevelopment Authority of

Singapore.Week Singapore, This, 18-24 December 1999. Singapore: Miller Freeman Pte. Ltd.Whyte, W. (1996) The design of spaces (first published 1988). In R. T LeGates and

E Stout (eds), The City Reader. London: Routledge, 109-117.

Part IIElectronic media and film

4 Phase and transition, type and instance: patternsin media texts as seen through a multimodalconcordancer

Anthony P. BaldryUniversity of Pavia

Introduction

How can we go about analyzing a TV advertisement? Despite the longtradition of analysis of printed advertisements, the prevailing view, untilquite recently, has been that it is impossible, for technical reasons, to analyseTV adverts in such a way that the interplay of visual and verbal resourcescan be reconstructed. Cook (1992: 37-38, see also 2001: 42-44), forexample, states that:

Any analysis of the language of adverts immediately encounters the paradox thatit both must and cannot take the musical and pictorial modes into account as well[. . .] This problem is more serious with tv than with printed ads, for on paperpictures stand still (and can even be reproduced), and there is no sound [. . .] Inconsidering tv ads, where pictures move, music plays, and language comes inchanging combinations of speech, song and writing, reproduction is virtuallyimpossible, and a video, to be watched while reading, would transform a writtenanalysis even more than companion illustrations. Many analyses of advertisingsolve this problem by ignoring it.

Cook's statement is, in fact, a testimony to the revolution that has takenplace in a decade vis-a-vis film texts and their analysis, often providingsolutions to the concerns he raises, particularly those relating to reproduc-tion: the videocassette can be easily digitalized using an appropriate PC cardand the resulting digital film can be manipulated in many ways, including,for example, the addition of explanatory captions; the Web, unknown tenyears ago, has spawned new forms of advertising which increasingly includestreaming video capturable through special software programs such asCamtasia; postproduction software such as Adobe Premiere has made itpossible to convert a film into a sequence of stills and hence into a printableformat.

These technological innovations have given rise to new descriptive prac-tices including: (a) the multimodal transcription (Baldry, 2000b: 81-85;Thibault, 2000: 374-385) and (b) the construction of PC-based multimodal


corpora accessible by structured queries (Baldry, 2000c: 31). The formerallows a TV advert to be reconstructed in terms of a Table containinga chronological sequence of frames, a technique that goes a long wayto resolving the difficulties of taking linguistic, musical and pictorialmodes into account. Figure 4.1 shows how, in a TV car advert, the inter-section between the Columns and Rows in a Table characterizes theinterplay between resources, not just those mentioned by Cook -speech, song and writing - but also others such as ambient sounds, gazeand gesture.

In keeping with the systemic-functional tradition of multimodality(Baldry, 2000a; Kress and van Leeuwen, 1996; O'Toole, 1994), a multi-modal transcription will also need to show how meaning is built up as aseries of functional units - typically, subphases, phases, but also potentiallymacrophases, minigenres and genres. An early example of this work waspresented by the author in the 25th International Systemic-FunctionalCongress in 1998 (Taylor and Baldry, 200la, 200Ib) which analysed thephasal (Gregory, 1995, 2002; Gregory and Malcolm, 1981) and metafunc-tional (Halliday, 1994) organization of a car advert in such a way as toshow the advert's interleaving of American and British cultural values.Subsequently, Thibault (2000: 311-385) devised an annotational system,partly reproduced in Figure 4.1, which provides a systematic descriptionof the interplay between resources in an Australian bank advert and whichillustrates how a typically Australian identity comes to be created.Thibault's model, which allows the phasal and metafunctional organiza-tion of a text to be described in great detail, has become a reference pointwhen extending the approach, for example, to subtitling for language-learning purposes (Baldry and Taylor, in press) and to the description ofgenres relating to the political arena (see the news report and interviewdescribed in Lombardo, 2001, and party political broadcasts described inVasta, 2001: 99-128). At the very least, this work has succeeded in estab-lishing how national identities and values are constandy expressed andmanipulated in many forms of advertising. In the process of this applica-tive work, the multimodal transcription has begun to change its function,increasingly being identified with the typical interplays that occur in manytexts; for example, Baldry and Thibault have devised a multimodal tran-scription which, by incorporating a multimodal tagging system, promotesan understanding of the notion of type that lies behind a specific instance(Baldry and Thibault, 2001: 94-98).

But is the multimodal transcription the answer to researchers' needs?This paper suggests that, at the very least, it needs to be backed up byother tools that help us understand the workings of multimodal genres.Indeed this paper describes the initial stages of research into multimodalconcordancing and development of appropriate software that will allowthe relationship between phase and transition to be viewed in terms of typerather than instance. In so doing it questions some aspects of the traditionalview of the relationship between phase and transition. But the main purpose

ELECTRONIC MEDIA AND FILM 85

of this paper is, however, to introduce the new field of multimodal con-cordancing as a means of examining text and text types in relation to theircontext of situation and context of culture (Halliday, 1978; Halliday andHasan, 1985). Multimodal concordancing thus builds on the foundationslaid by the multimodal transcription and on systemic-functionalapproaches to language-only concordancing such as the Systemics Coderdeveloped by O'Donnell (2002). In so doing it raises questions about howthe study of multimodal discourse might be undertaken in the language-learning classroom (Baldry, 1999, in press; Pavesi and Baldry, 2000) andmore generally how multimodal concordancing might develop in thefuture.

The multimodal transcription as an expression of instance or oftype in phasal organization?

What role do phases and transitions play in TV adverts? And crucially howfar are multimodal transcriptions limited to giving details of specificinstances of phases and how far can they, instead, give information abouttypes of phases? Figure 4.1 is a small and highly abridged sample of a

Figure 4.1 A classic multimodal transcription (Phase 1 of The Fan)


'classic' multimodal transcription based on the Table and (with a fewadditions and modifications and many abridgements) on, in particular,Thibault's annotational scheme (see Thibault, 2000: 374—385 for a com-plete example of a multimodal transcription). It relates to an advert entitledThe Fan in which a male driver, whose car has become overheated, hitches alift from a lady driver in an Audi automatic.

The transcription in Figure 4.1 is concerned with instance rather than withtype and may be read from left to right in three major blocks that constitute aprogression from a description of Textual data to one relating to the specifictext's organization into semiotic units: (a) the first two columns relate to theway in which frames are selected with a periodic regularity, in this case oneframe per second (see Table 4.1 for an explanation of the abbreviations

Table 4.1 List of abbreviations

Time: TS = time in seconds;

Phases: Ph — Phase or 11 ; SP= Subphase or |; CD = Car Drive; CS= CarStationary;

Metqfunctions: EXP= Experiential; £NT= Interpersonal; TEX= Textual;

Visual image: CP= Camera Position; VS = Visual salience; SH= Shot; CLS =Close Shot; D = Distance; MCS= Medium Close Shot; SV=Side view; FV= Front View P; /C= Inside Car; OC= Outside Car;ICLO — Inside Car Looking Out; OCLI= Outside Car LookingIn; WS = written slogan;

Participants: P= Participant, D = Male Driver; F= Female driver; M=Mascot;

Soundtrack: ST= Soundtrack; AS = Ambient sounds; MIS — Music andsinging or JJ + %* with words in italics representing wordsspoken or sung;

Transitions: T= Transition; \ <> \ = transition lasting a subphase; > || atransition crossing a phasal boundary;

Resource J high integration of resources (in particular between bodymovements and music);

Combinations (RC): L low integration of resources (in particular between bodymovements and music);

K medium integration of resources (in particular between bodymovements and music);

Other symbols: Movt. = Movement; •!• or —> = the same semiotic selections holdtrue as compared with the previous frame on the left (i.e. samephase or subphase but some changes have occurred); a doublearrow as in —> P: —>+ F (finger) means that the configuration isas before but there is now a new Participant added to thosepreviously present: the Female driver's finger.


used); (b) the subsequent columns provide descriptions of the individualsemiotic resources, including a description of the soundtrack in terms ofboth music and song and ambient sounds; (c) the final columns relate tosemiotically motivated interpretations of how resources combine to formmeaning-making units.

As Thibault (2000: 321) points out, multimodal text analysis does notaccept the notion that the meaning of the text can be divided into a numberof separate semiotic 'channels' or 'codes': the meaning of a multimodal textis instead the composite product/process of the ways in which differentresources are co-deployed and in which the phase is taken as an enactmentof'locally foregrounded selections of options'.

Figure 4.2 is a very different kind of multimodal transcription, organizednot so much as a finite Table with its page-based verticality, but more like amusical score. It unfolds from left to right in a manner that potentiallyextends well beyond the confines of the page. Such a transcription, remainshowever, in keeping with the overall goals of multimodal text analysiswhich is to specify both the selections made from the various semioticmodalities and the combinations used to produce a given (phase-specific)meaning (Thibault, 2000: 321). Vis-a-vis Figure 4.1, it uses a few moreabbreviations.

It should be noted that the car, although not included in the list ofabbreviations, is still to be considered a Participant (in the technical senseof a Participant in a ParticipantAProcess relationship, see Halliday, 1994:107-109), its parts (gear, window, etc.) being spelled out in full: thus a tran-scription of the type P: D: Arm; Car: Gear means that the major Participantsin the construction of meaning are the driver's arm and the car gearstick.Unlike the 'classic' multimodal transcription described in Figure 4.1, a tran-scription of the type presented in Figure 4.2 is concerned as much with typeas with instance. It dispenses, for example, with a precise reference to thetext's unfolding in time: in fact the total duration of this text is 35 secondsand the interval between the individual frames is, as in Figure 4.1, still onesecond.

The transcription in Figure 4.2 is in some respects slighdy less detailedthan the instantial conception of the multimodal transcription exemplified inFigure 4.1 since one of its functions is to summarize the major character-istics of the entire text in a concise way, thus demonstrating its greater poten-tial for compression in description. This kind of multimodal transcriptionrecords information about type in the top section, i.e. the Row above theindividual frames, and information about specific instance in the bottom part.In particular, the Top Row suggests the change that has taken place vis-a-visthe previous subphase: the text in question relates to the male driver jivingto the sounds of an Elvis Presley-type song as he drives along, while thedetails of what actually happens to the driver and the way this relates to thesong are given in the Bottom Row. The Top Row thus suggests the semioticdevelopment of the text in terms of its phasal structure and the mainchanges in its deployment of resources - the Top Row is thus oriented

Figure 4.2 A multimodal transcription incorporating structure/type of phase/subphase(Top Row) and co-deployments and resource selections (Bottom Row)


towards the constant shifts in the selection of options in keeping with Gregory'sprinciple that phase and transition can 'be used to capture the dynamicinstantiation of micro-registerial choices in a particular discourse' (Gregory,2002: 323); the Bottom Row (with its focus on the content of each shot)describes, on the other hand, the film's unfolding in time, and, though notexcluding the principle of selection from options, is thus oriented moretowards sequential development and specific realizations. Though notshown here, a multimodal transcription of this type also allows Textualelements from various texts to be aligned in such a way as to compare theirphasal organization (see Baldry, 2000b: 68-69 for the development of thecomparative multimodal transcription).

Though unusually involving two drivers and two car-drive phases, inmany other ways the text in question illustrates many typical features ofcar adverts, in particular the expression of the very strong relationshipbetween the driver's and the car's identity. As Figure 4.2 indicates,although other criteria might have been invoked, a good starting pointwhen defining the division into phases in this text (and we may add the60 adverts in the current car advert corpus) relates not to the humanparticipants but instead to the type of representation of the car: in thiscase (and in many other cases) whether the car is present and, if so,whether it is moving or stationary. At the start of the first phase of thistext, there is a typical car-drive phase [+CD], in the second, an essen-tially car-stationary phase [+CS] (though the second subphase containsthe idea of a car stopping and starting - hence the [+CS, +CD] tag); thethird phase is again a car-drive phase [+CD], while the fourth phase, theend phase, typically relates to the car abstractly in terms of its make andmanufacturer, and presents all the typical ingredients of one type of endphase where the car itself is (physically) excluded [—CD, —CS] and whereinstead the focus is on oral and written slogans and the manufacturer'slogo.

The correlation between driver and car is, of course, a major goal ofthe car advert genre, reflected in the genre's phasal organization, whichcharacterizes the way the car advert unfolds in time. The car is verymuch a Participant — by definition, at least an equal partner in thehuman/non-human participant relationship (and more often than not asuperior). This emerges quite clearly in the type-oriented multimodal tran-scription of Figure 4.2, which explicitly defines the constant shifts in localforegrounding in the Top Row, e.g. whether it is the car, the driver, themascot or the countryside that is the salient Participant in a particularphase or subphase.

In fact, the most salient phases in this text are the first and third, withthe first phase being conjunctive-disjunctive in nature and the third, con-versely, of the disjunctive-conjunctive type. This reflects the text's fore-grounding of potentially conflictual Interpersonal relationships betweenthe two drivers: the first, an outlandish jive-as-you-drive dude, the second,a suave, sophisticated female. 'Conjunctive' and 'disjunctive' are here


respectively used to describe specifically the synchronization and non-synchronization of movements among Participants (see Thibault, 2000:342), which, of course, include the cars and the mascot as well as thedrivers. It is frequently the case in film texts (for example, documentaries)that the visual and the verbal are out of step, with the visual anticipatingthe verbal (see Baldry, 2000b: 74), but what is striking in this text is whetheror not the movements of the participants are synchronized in relation to eachother and to the music. An attempt has been made to track the types ofshift that take place in this respect using symbols that relate to ResourceCombinations (RC).

Resources can thus be deployed, as in this text, in such a way that theirinitial synchronization is lost, thereby creating two sets of meanings whichare potentially in conflict. In the first phase, all the resources - bodymovement, music and song and the sequence of visual frames - are inunison but, by the end of the first phase, visual image, kinetic action andsoundtrack are out of step: neither driver nor mascot are swinging in timewith the music and song (which continues); instead with the sudden brak-ing of the car, they remain quite rigid and motionless, an indication that asecond and rather ironical series of rhythms is at work, which, togetherwith the smoke coming from the gearbox, signals the fact that the dude'scar is on the point of breaking down. Conversely, in the third phase,desynchronized resources become synchronized: the man and womanseem, initially, to be in conflict with each other, with gaze significantiycontributing to this meaning - the woman glares reproachfully at the manwho dares to stick his mascot on her impeccably clean windscreen whilethe poor man 'defends' himself by looking blankly straight ahead, out ofthe windscreen, his facial expression and body position, having becomeutterly rigid, in complete contrast to what happens in the first three sub-phases of the first phase. Resources such as gaze, spatial disposition, bodymovement and facial expression are deployed in this text in such a way asto be deliberately out of step with cultural expectations: two people sittingnext to each other in the confined space of a private car and who havenever met before and who are the car's only occupants will normally lookat and talk to each other, which is precisely what does not happen. Grad-ually, harmony between the two sets in, as the interplay of semioticresources underpinning the first phase is restored: music and song are thefirst to be reintroduced when the man hands over his cassette, followed bythe restored rhythms of the mascot, which when prodded by the now-smiling lady, once again swings in keeping with the music; finally, the man'sgood humour also returns: he lifts his head up, smiles and starts to chewgum again, all signs that his unrestrained jiving-'n'-driving is in the processof being rehabilitated. Important in this process is the focus on the mascot,which is seen being prodded by the lady who, though not present in theman's car, seems intuitively to understand that this gesture will cause themascot to wriggle and writhe to the music thereby restoring her passenger'sgood humour.


These two phases are separated by a very brief second phase in whichboth cars are essentially motionless and where, vis-a-vis the metafunctions,rather than Interpersonal elements, Textual and Experiential elements areprominent: in this short phase, the New is introduced in the form of a newcar, a new driver and an Internet address (note that, in fact, the Internetaddress has been 'carried across' from the previous phase, illustrating thesignificance of extended transitions and overlaps in phasal organization, amatter discussed in detail below). The main meaning created is that themale driver successfully hitches a lift, so that it is important to glimpse oneof the cars stopping - hence the [+CS +CD] tag for this phase. The absenceof salient Interpersonal elements in this phase is striking: the song, forexample, ceases, in contrast to the previous and subsequent phases,indirectly underscoring the fact that, in many car ads, song is a crucial sourceof meaning, often acting as the functional equivalent of a narrator, linkingthe viewer to the events at hand and, in part, defining the viewer's expectedresponse to actions and events.

This advert is no exception in this respect: the final refrain 'you own myheart', cements the identity between the viewer, the car drivers and the car.It also coincides with the written slogan - multitronic©: II cambio automa-tico a variazione continua da Audi [i.e. Audi's gearbox with continuouslyvariable automatic transmission] further building on the text's basic the-matics, namely that the discordant contrast between the smooth, sophisti-cated lady and the dude's jive-as-you-drive lifestyle will be resolved in aharmonious fashion by a relaxing ride in the right car, namely an Audiautomatic.

Significantly, many contemporary car adverts present the car as aspace where social conflicts, potential or real, may be resolved, whetherwithin the family or, for example, between loving couples. The car can thusbe a sexual space as in the Citroen car advert in the author's corpuswhere, in the hyperreal coding orientation (for coding orientation seeBernstein, 1971; for a multimodal perspective of coding orientationsee Kress and van Leeuwen, 1996: 168—171), the car rolls over andover as the couple make love; alternatively, it may be a place of protectionwhere a kissing couple in a lonely lane can successfully fend off anattack from Zombies. More mundanely, it can also be a space wherechildren and animals can be safely transported and sometimes even aspace in which members of a sports team can start throwing aball around, de facto transcending the narrow confines of the car'sinterior.

In this advert, song contributes significantly to this particular meaning ofconflict resolution, built up gradually and multimodally, throughout theadvert with its highly sensual suggestion that the relationship between theman and the woman will outlive the lift and that it will be the womanwho will provide the initiative in this respect: a prod, as it were, is asgood as a wink. Notice, indeed, how, rather than stopping at the end ofthe third phase, the song extends beyond into the final phase, with the


result that the latter, as well as giving the usual information about theparticular model and the manufacturer, also underscores the entire text'smeaning, by suggesting that the conflict between the man and the womancan be, and indeed, has been resolved by virtue of the car's designcharacteristics.

This meaning-making is the result of the artful juxtaposition and over-lapping of different types of phases that carry out different functions. Thevery notion of phase presupposes that there is some transition between onephase and another and, to a lesser extent, between the various subphasesthat constitute a phase. Moreover, following on from what has been statedabove, as well as phase types, we can also expect various types of transition to bepresent in film texts. For example, Thibault (2000: 320-321) suggests thatthe points of transition between phases have their own special features thatplay an important role in the ways in which observers or viewers recognizethe shift from one phase to the next and that, generally speaking, transitionpoints are perceptually more salient in relation to the phases themselves.Thus viewers of texts have no difficulty in perceiving particular Textualphases thanks to their ability to recognize the transition points or theboundaries between phases.

However, the notion of transition should not necessarily be associatedwith the idea that there is a precise boundary or point at which a transitionoccurs. In many cases, this vision of boundaries in the organization ofphases and transitions will work very successfully. But this is not always thecase. As Thibault (2000: 326-327) points out:

Perceptually speaking, transitions between phases are not always clear-cut [. . .]Thus, the transition point may be characterized by a gradual merging of featuresfrom the two phases in question as one phase decays or fades out and the otherconies into being [. . .] The transitions between subphases are not always sostraightforward. At times, there is an almost imperceptible overlap between sub-phases.

In this text, for example, each of the two main phases (the first and the third)contains a series of pivotal transitional points between the various sub-phases that mark the step-like progression from conjunctive to disjunctive (asdefined above) and vice versa: these are movements relating to the gearbox,the cassette, the mascot, the drivers and the cars (e.g. braking). In the firstphase, the malfunctioning of the gearbox (we hear an ominous crunchingnoise) and sudden braking and cessation of the mascot's movements signalthat disruption is to follow. The driver's jiving also comes to a stop. In thethird phase, the reverse is true: for the second time the camera carefullyfocuses on the gearbox, which, in keeping with the demands of the targetedaudience, and quite unlike the first gearbox, is an automatic gearbox fordrivers who like a smooth ride. Significantly, the transition points in this, andmany other adverts, are linked in a chain to form a crescendo which


contributes to the overall coherence of the text. One way in which thissalience is achieved is by changing the camera focus: thus the out-of-focusmascot suddenly comes into focus. Another is the type of shot used: twomajor subphasal transition points in the first phase and a third in the thirdphase coincide with the only three shots in which we view the mascot bylooking out of the car through the windscreen: in each case this selection of themascot contributes to the underlying conjunctive/disjunctive 'stop-start'flow of the text: the mascot is shown, in an alternating way, as either static,carrying with it a negative connotation (a 'stop') that things are wrong, or,when it sways in all directions, with a positive connotation (a 'continuation'or a 'restart' after a 'stop'). A similar chain is described in Thibault (2000:328-329), in terms of:

covariate semantic ties in the visual thematics [. . .] that are progressively dennedin the unfolding text as cohesive chains extending over the entire text. Forexample, the foregrounded co-patternings of items deriving from the interactingcohesive chains of 'smiling', 'rolling the sleeves', and 'moving forward' functionto create global coherence in the text.

In Thibault's example the meaning implied relates to the characterizationof different activities as being fundamentally analogous (the participants, eachin a different context, roll up their sleeves, smile and get on with theirdifferent jobs). But the transition chains in this text carry out a very differentfunction: they realize step-like crescendos relating to the creation of discordand the subsequent return to harmony.

That the notion of transition does not necessarily entail the notion of asingle point or a single boundary in any particular phase also emerges in otherways. Mergings and overlaps between phases are also typical in many filmtexts, adverts included. In this particular text, transitions are prominent inboth the second and the fourth phases of this text, where the transitionfrom Phase 1 to Phase 2 is prolonged over a few seconds in such a way as toconstruct the meaning that the male driver is in the process of changingcars. Thus, in the second phase, phase and transition are partly co-terminousinsofar as an entire subphase (SP2) is taken up with an (albeit rare) splitshot, in which the two different cars are simultaneously foregrounded andbackgrounded, the result of postproduction techniques (but also clevercamera work), whose purpose is to effect the transition from the Given (thefirst car) to the Mew (the second car) in a salient and lingering way, therebyunderscoring the fact that a major change in the events described in the textis taking place. Moreover, as we have already seen, the end phase, which incar adverts are typically associated with slogans for the particular car modeland car manufacturer, is merged with the previous phase, thanks to theprecise synchronization between the oral slogan (the final part of the song),and the written slogan.

Thus, transitions are not necessarily equated with the cutting from oneshot to another, nor indeed with what is happening in the visual. While


transitions will often be related to what is happening in the visual, thiswill not always be the case; while phases (see Gregory, 1995, 2002;Gregory and Malcolm, 1981) relate to 'stretches of text in which there is asignificant measure of consistency and congruity' (Gregory, 2002: 322)transitions, as Thibault (2000: 320) and Gregory (2002: 323) have pointedout, essentially relate to changes in the metafunctional organization of thetext and as such may very well be related to changes in the soundtrackand not just to what happens in the visual. One of the clues to the factthat, in this text, the second phase really is a separate phase is the factthat after the noise and commotion of the first phase, this phase uses'quiet' sounds: no song, no music - just the sound of car tyres and abarely audible wind. Indeed the Top Row of Figure 4.2 attempts torecord the constantly changing interplay between the types of resource inthe soundtrack: ambient sounds, music and music and song (but nevercomplete silence).

Transitions, as well as being structural in nature, are thus inherentlyand predominantly semiotic, contributing to the entire text's meaningthrough their typical organization into chains. They are thus not just partof the local foregrounding of semiotic selections. Indeed, precisely becausethey are salient, transitions are frequently linked to the advert's ultimatemessage. Transitions are ultimately bound up with the expectations thatthe viewer has about the text and often guide the viewer vis-a-vis theseexpectations to the right conclusion. Transitions thus have to do with theconstant interplay between the expected and unexpected in film texts. Inthis text, we expect song and music to be restored, which is precisely whathappens.

These expectations are inherently multimodal, the result of the interplaybetween many resources. Zago (2002: 62—70) reports an interesting case inwhich a drinks advert uses an animated cartoon to represent the transitionsfrom one experience to another in a sequence of hallucinations each repre-sented as a warping of the face of the protagonist, an exhausted cyclist, andin the buildings he cycles past. He finally reaches a place where he can drinka cool pint of Guinness, and thus bring a halt to the spiral of fever-likeexperiences that include blue penguins and deformed rubber-like walls.Here, too, it is the chain of transitions that is important, the text's meaningbeing built around a spiralling escalation, interrupted only by the act of'murdering' a cool pint. I have also reported a similar chain of transitions atwork in Benigni's 1997 film La vita e bella (Life is beautiful} (Baldry, 2002)suggesting that the viewer's expectations are that the chain of transitionsfrom one phase to another will be linked to the final climax in the film.Transition chains make their meaning by being typically multimodal. Inmany film genres, they will be visual and musical as well as linguistic, thecase, time and again, in the world of advertising. In La vita e bella all threeelements intertwine, with music playing a very significant role: the catchymusic starts off as background music but gradually as the film proceedsbecomes foregrounded and thematized as the only means of communication


in a concentration camp. In saying this, we are suggesting that it is thetransition., rather than the phase, that is the most significant element in thephasal organization of a text and that a focus on type of transition in multimo-dal analysis will help clarify that what is salient is ultimately what is mostmeaningful.

So far we have posited phase types applicable to this advert as beingdescribable as: [+GD], [-CD], [+CS] or [-CS] or a combination thereof(but see also the discussion below for their extension to many other caradverts) and, from a slightly different perspective, we have also posited theexistence of phases that can be characterized in terms of the conjunctiveand/or disjunctive deployment of resources. But what transition types arethere? In this paper, given that multimodal concordancing is taking its veryfirst steps, we can do little more than posit their existence.

Indeed, precisely because of its static nature, the multimodal transcrip-tion, which seems so far to have been the major research tool used in themultimodal analysis of film text, is inappropriate when identifying anddescribing what is quintessentially dynamic in nature: namely the transition.All this is reflected in a second type of multimodal transcription presented inFigure 4.2, concerned as much with multimodal type as with multimodalinstance. Still under development (e.g. as a method of reporting the findingsof multimodal concordancing), this type of transcription tries to highlighttypes of cut, types of shot, types of phases and types of transition. To give justone example, the symbol > has been used to suggest a transition overlap, thatis, points at which there is no clean break between one phase and anotherbut where instead one or more resources get carried across what otherwiseappears to be a phasal boundary. Thus, the symbol >ll J3 + %* means a typeof transition in which music and song are carried across from one phase toanother. Conversely, the symbol | <> | means a type of transition that lastsfor the entire length of a subphase.

Looking at types of phase and transition through a multimodalconcordancer

In describing The Fan advert, we are beginning to move away from themultimodal transcription as an expression of instance towards the multimo-dal transcription as an expression of type, which inevitably raises a wholeseries of questions. Are certain types of transitions likely to be found morefrequently in specific genres? Is the absence of speech, or indeed totalsilence, one of the typical markers of a transition from one phase to anotherin a feature film, but which, because of the need for maximum compression ofmeaning in a very short space of time, is unlikely to be present in suchgenres as the car advert? Or, on the other hand, do transitions function insuch a way as to introduce a new item of information that builds, sometimesin a repetitive chain, onto what has previously been constructed in the text?As viewers we are capable of recognizing phases and transitions; as tran-scribers, we can reconstruct where they occur. But this does not amount to


the same thing as characterizing the typical ways in which transitions come tobe the salient element in phasal organization.

A multimodal transcription is limited in the amount of information it cangive about types of semiotic units that are found in film texts and cannotprovide anything like the information we need in order to provide motivatedanswers to these questions. If we are to pursue our understanding of the co-deployment of semiotic resources more thoroughly we need to understandhow a large number of dynamic texts typically unfold in time.

And in order to be able to identify characteristic patterns, the researchprocess requires us to build corpora that can be analysed in terms of variousTextual phenomena, including, in particular, a study of the typical phasalorganization of a specific genre which ensures that a film's unfolding intime, in which the transition, as we have seen, is so significant, can becaptured by in vivo multimodal analysis. Such a requirement dictates theneed to build software programs that are capable of analyzing corpora andnot just individual texts.

What then are the characteristics of an online XML-based multimodalconcordancer such as the Multimodal Corpus Authoring (MCA) system,which has been designed by the author specifically to identify recurrentpatterns in films?

First, as an authoring tool, it enables researchers, however imperfectly, toview short pieces of film and simultaneously to write multimodal descrip-tions of them in terms of various parameters, for example, those relating toa text's metafunctional and phasal organization. Using MCA's editing tool,researchers can segment a particular film into functional units and, whileviewing these units, type out detailed annotations relating both to the semi-otic resources they deploy and the functions they perform within that film.Indeed, MCA approximates to the researcher's dream of simultaneouslyviewing and writing a description of a film in real time (see Baldry andTaylor, in press).

Second, like a linguistic concordancer, a multimodal concordancer canalso establish patterns that relate to a series of texts, rather than to specificinstances, to a much greater degree than is possible with a multimodaltranscription, even where the latter is oriented towards type rather thaninstance. For example, it is possible, using MCA, to determine the ratio offemale to male drivers, or to identify those texts relating to cars that are notbeing driven, and hence have no drivers, and those relating to cars which areinstead being driven but where the driver is 'implied' and not actually seen.It is also possible to identify special cases that involve two drivers, typicallyone male and one female, or non-human drivers, typically robots. As withany corpus approach using information technology, this information can beobtained within a few seconds. However, unlike many lemma-basedapproaches, the researcher must first carry out the work of description-cum-transcription of the texts in the corpus. Not surprisingly, the software designis such to incorporate an analytical framework that simplifies this task asmuch as possible.


MCA's incorporated relational database allows researchers to search thecorpora created and identify patterns in them, all of which leads to a furtherround of hypothesis formulation, segmentation, description and com-parison of results. Table 4.2 gives the results of multimodal concordancingin relation to 60 car adverts and shows that there are, in fact, many caseswhere there is either no driver, because the car is stationary (16/60), orwhere an unseen driver is driving the car (19/60). There are in fact a total of24 male drivers (though in 3 cases we assume that the driver is a male fromwhat goes before and after). There are only six woman drivers and two ofthese appear, as in the case of The Fan, in adverts where a man also drives.Importantly, half the adverts are careful not to show the driver's identity.

Moreover, the relationship between men and women takes on a differentperspective when we look at different participants in the structure of anadvert. When we examine, for example, the ratio between male and femalevoiceovers (whose function is usually to act as 'narrators' or 'storytellers'), wenotice that the imbalance begins to redress itself for there are various casesin the corpus where, vis-a-vis a male or an unidentified driver, a female

Table 4.2 Driver types in 60 TV car adverts


voiceover predominates. As Table 4.3 shows, the search query in this case isno longer formed by a single parameter (driver) but is a relational search thatlinks two disparate parameters: driver and storyteller.

Thus, unlike many lemma-based linguistic concordancers such as OCPor WordSmith, but in keeping with the approach adopted by O'Halloranand Judd (2002), a multimodal concordancer needs to be built around thenotion of the relationship between resources, events and participants. In thisrespect, any form of transcription is a hard task, often undertaken by aresearcher without knowing whether the effort will be worth the candle. Intheory, the results described in Table 4.2 could be acquired by watching avideocassette and marking down the various features using pen and paper.Though in principle feasible, it would be a time-consuming process. Evenusing MCA, which greatly reduces the time taken to provide a description, itis still a time-consuming process. A much harder task, however, is to relate theparameter DRIVER with other parameters such as STORYTELLER and ORALSLOGAN. This is virtually impossible to achieve using traditional pen-and-paper and cassette methods. A multimodal concordancer, such as MCA,which is based on these relational principles, can easily identify such pat-terns through relational searches as Table 4.3 indicates.

Third, a multimodal concordancer, even more than a linguistic concord-ancer, needs to be built around functional parameters such as those we havementioned above, namely Halliday's notion of metafunctions (Halliday,1994) and Gregory's notion of phase and transition (Gregory, 1995, 2002).In this respect, one significant step in the development of a corpus relates tothe work of tagging. In their paper on the development of a tagging system,Baldry and Thibault (2001: 94-98) proposed the use of an annotationalsystem that defined gesture and language in terms of Halliday's notion of

Table 4.3 A relational search in MCA


Experiential metafunction: thus a tag of the L-MENT:PROJ and G-MENT:PROJtype means that the text being described contains an instantiation in whichlanguage and gesture are being used together to express mental projection.MCA will support this type of tagging without any difficulty. However, giventhat, as mentioned above, annotational systems in multimodal concordanc-ers are still in their infancy, the system adopted so far has been oriented to abinary presence/absence distinction of the various descriptive parameters,which, as described elsewhere (Baldry and Taylor, in press), may be definedat will by the corpus author.

But how does all this contribute, for example, to our understanding of thephasal organization of texts? Though ultimately more sophisticated map-pings of the relationship between phases and metafunctions should bepossible, in the current stage of development, this relationship has beencharacterized only in terms of a very preliminary step, namely the analysisof the major Experiential 'category' in the car-drive phase(s) of 60 caradverts: the activity of driving and what precedes and follows it.

As Table 4.4 illustrates, this activity has been characterized in terms ofthe sub-components associated with the material process of driving, whereSP stands (as indicated in Table 4.1) for a subphase.

Using MCA, this information can be retrieved from the corpus with aquery of the form: SP2: contains YES or SP2: contains NO or even SP2:contains YES and NO in cases where the matter is not quite so clear (for

Table 4.4 Division of the activity of driving into subphases

SP1: INDICATES INTENTION: e.g. picks up keys (partly a mentalprocess, partly a material process);

SP2: APPROACHES: The driver a) approaches the car and b) unlocksthe driver's door;

SP3: GETS IN: The driver a) opens the door, b) gets in and c) closesdoor;

SP4: STARTS UP/DEPARTS: The driver a) puts the key in the ignition,b) starts the engine, c) indicates intention to move off and d) pullsaway;

SP5: CAR-DRIVE: The driver drives along the road (in town/country,by day/by night, in summer/winter, on/off road) towards his/her destination;

SPG: STOPS/SLOWS: The driver (a) stops and (b) slows down atINTERMEDIATE POINTS (e.g. traffic lights, junction, negotiatesbend, has accident, calls in a shop, changes cars);

SP7: ARRIVES/PARKS: The driver (a) slows down, (b) stops onreaching destination, (c) parks and (d) switches off engine;

SP8: EXITS: The driver (a) gets out of the car and (b) closes door;

SP9: WALKS OFF: separates himself/herself physically from the car.


example, when a driver opens the door and puts an object or person in thecar rather than himself/herself). Equally, it is possible, with a single search, toidentify all the cases where we see the car being driven and the driver gettinginto and out of the car, in this case a query of the type: SP3: contains YES +SP5: contains YES + SP8 contains YES. Thus 60 adverts were 'tagged' interms of the subphases of the car-drive phase (the first subphase has beenexcluded on the grounds that it is only partly a material process), in such away that the corpus could be searched for the absence or presence of aparticular subphase.

As Table 4.5 shows, there is in fact only one advert (n. 21) which comesanywhere close to instantiating all the possible subphases and even in thiscase one subphase is missing and another is doubtful - hence the YES/NO tagrepresented as a bracketed tick: this is a case where the driver is seen gettinginto the car but only to put his young son in the back seat (see Figure 4.3below). In all these adverts, visual/verbal ellipsis is constantly at work vis-a-vis the instantiation of the driving experience: there is normally no need tosee all the phases at work, since our own experience of driving allows us to'fill in the gaps'. With the exception of advert n. 21, in 60 adverts we neversee the driver getting into and out of a car.

Table 4.5 suggests that car adverts do, in fact, fall into three types, whichmay be tabulated as follows:

1 Car-drive adverts: The car is seen moving in a glorified way that attempts togo beyond the daily grind of the ordinary world. The car is in an idealworld. More often than not the number of participants is limited to one or two peopleand in many cases no human participant is foregrounded; the participants never talkabout the car and never talk to each other and only exceptionally to the audience. Inthese adverts only subphase 5 is apparent (17 cases);

2 Car-stationary adverts: The car is motionless, a statue to be 'worshipped'and is typically related to some inconsistency or oddity in the behaviourof the people surrounding the car who typically talk about the car. In theseadverts, none of the subphases listed in Table 4.4 is present (11 casesrepresented in Table 4.5 as grey-shaded columns) or alternatively sub-phases in which the car is seen moving are absent (a further 6 cases);

3 Hybrid storytelling adverts: where both car-drive and car-stationary elementsare present and where either other genres are exploited to meet theadvert's own ends (e.g. spoofs on cinema and TV genres) or some attemptis made to define the car in relation to daily activities and (usually) itsenhancement of these. These types include talk but never in the car-drive phase orsubphase. A good example of this is where the car-drive element is notshown - hence the bracketed tick notation - but is instead realized,through talk, as a mental and oral fantasy (projection) about the car'sdrive potential by the car driver while the car is actually stopped (say atthe traffic lights). This is by far the largest category (26 cases), although itshould be noted that the majority (15) instantiate CD subphases before CSsubphases (The Fan being a rather special case).


Table 4.5 Distribution of subphases (material process) in the car-drive phase

A fourth important characteristic of a multimodal concordancer is that itcomes close to functioning as a 'Mark II' multimodal transcription repre-sented in Figure 4.2 incorporating the notion of type in that it can 'printout' all the characteristics of a specific car advert in terms of a set of YES/NO presence of descriptive parameters. Thus Table 4.6 gives the 'print-out' (actually a screen illustration) for The Fan advert we have analysedabove.

102 MULTIMODAL DISCOURSE ANALYSIS '

Table 4.6 Screen illustration of a multimodal transcription generated by MCA

Table 4.7 A multimodal transcription generated by MCA using relationalparameters

Finally, an important function of the multimodal concordancer, closelylinked to its capacity to relate the characteristics of a specific car advert togeneral trends, lies in its ability to pick the 'odd man' out. Thus, for example,getting into a car is a comparatively rare event found in only 5 out of 60adverts. Though by no means the rarest of subphases, its relative absence issurprising. Moreover, there are only two cases (10, 21) where SP2+SP3+SP4all occur together. As Table 4.7 shows, they are both marked cases where, as isvery frequently the case in car adverts, the abnormality and unpredictabilityof humans (in this case, as Figure 4.3 shows, the stereotypical forgetfulnessof a male driver) is compared to the scientific reliability of cars.

Notice the player symbols on the left-hand side of Tables 4.3, 4.6 and 4.7.Once we have identified a particularly striking result, we can mouse-clickthese symbols and gain immediate access to the advert in question, all ofwhich allows us to view the precise context and to 'explain' the exception tothe predicted pattern in the manner indicated in Figure 4.3. A multimodalconcordancer is, after all, concerned with giving the researcher immediateaccess to phases in film that require careful scrutiny.

Analyses of the results of queries such as Table 4.4, together with excep-tions such as those suggested in Table 4.7 and Figure 4.3, all confirmThibault's (2000: 343) hypothesis that:

In movement, simultaneity and spatiality rather than linear succession in timeand particulateness (constituency) are important in the realization of Experientialevent and action configurations.


Figure 4.3 Unusual events dictate the need for an extended pre-drive subphase

This might at first seem surprising: undoubtedly, the posited sequence of8 subphases in the material process of driving might at first be seen asimplying a linear succession in time. However, as we have seen, most of thesubphases are implied rather than actually seen: only 5 out of 60 advertsexplicitly represent more than 4 subphases. Most are more like The Fan,concerned with the car as a social space rather than as a moving object.The camera focuses on the spatial location of the car (on a country road)and on a body or body part (where body = e.g. driver, mascot or car) whichperforms a movement as an instigator or a reactor. Nevertheless, car advertsmay, in general, be divided into three main blocks that can be tabulatedas follows:

an initial block consisting of a single phase focusing on a single, individualentity: a specific car, a person or a place in time;a main block consisting of one or more phases or subphases contextual-izing the initial focus through the specification of the relations of theselected entity with the 'missing' parameters;an end block consisting of a single phase: featuring the car logo, name,manufacturer and, in many cases, some kind of EVALUATING synthesis thatmay be used to project beyond the small world of individual entitiesshown in the advert to a larger, more complex world (and which, ofcourse, functions to persuade you, the viewer, by overcoming your resist-ance to the product).

This phasal organization seems to fit The Fan and many other adverts in thecorpus very well. However, more work using MCA is required to establishthe validity of this suggested typical phasal organization and the divisionof advert types into three types. The [+/-CD] and [+/—GS] taggingsystem will not, of course, always be distributed as in the current case as:+CD(P1)A+CSA+CS/+CD(P2)A+GD(P3)A-GS/-GD(P4). There are cases,for example, in which the distribution is essentially the reverse, with the car'sphysical presence being confined exclusively to the end phase. But this doesnot affect the hypothesis that three basic subtypes exist.


If they do exist, then it may well be that the predominating human figurein the car advert will turn out to be generically correlated with one of thespecific subtypes mentioned above: the DRIVER (the car-drive only advert),the INSPECTOR (the car-stationary advert) and the RACONTEUR/STORYTELLER(the hybrid type alternating car-drive and car-stationary phases and includ-ing the subtype which includes an off-screen narrator). A further predictionis that other roles will be involved definable, however, in relation to the car (asopposed to other participants, whether family, colleagues or strangers). Thatis, it may prove to be the case that (despite many overlaps between thecategories) the car may be defined in terms of first, second and third personrelationships. The general distribution might well be: (a) car-drive adverts:driver with his/her car [first person: mine: car and me, driver are the same thing];(b) car-stationary adverts: inspector with somebody else's car, not mine [third per-son: otherness: not mine/notyours\; (c) storytelling adverts: raconteur and his/herdream car for you [second person: yours, likely to include some kind of appealof the type: You should be driving it. . .].

Table 4.5 reconstructs the Experiential metafunction of 60 car advertsanalytically and systematically as subphases in the material process of driving,thereby suggesting the validity of multimodal concordancing as an ana-lytical and teaching approach. But, however systematic this may be, this isonly a provisional finding for if we are to honour the definition of phases interms of Gregory's already mentioned concept of consistency and congruityechoed in Thibault's definition of phases as 'co-patterned semiotic selec-tions that are co-deployed in a consistent way over a given stretch of text'(Thibault, 2000: 325-326) and if we are to characterize their consequentclose identification with specific metafunctional configurations, we need,at the very least, to complete the picture by describing patterns thatemerge vis-a-vis the Interpersonal metafunction (many of which are likelyto be stereotypical) and even more crucially the types of configurationsthat emerge in relation to Interpersonal meanings when they are mappedonto the Experiential structure we have sketched out. This is a complexdescriptive operation. Thus, although the previous paragraph gives broadsuggestions as to how this mapping might take place in car adverts, a com-plete picture of the organization of car adverts into typical patterns ofphases and transitions still needs to be worked out. Such a picture needs tobe ascertained with more robust corpus description than the one currentlyavailable. But the important point to note is that both the type of corpusdescription and the corpus querying that this operation requires seem to bequite in keeping with MCA's capabilities, given that its core feature isits capacity to relate a wide array of disparate features over a wide rangeof texts. But even if the phasal patterns sketched out above prove to bevalid over a still larger corpus, they will not be a point of arrival. Rather theywill still be a point of departure into a more precise understanding oftransitions and transition types, whose careful description, as this paper hasattempted to suggest, is crucial to the success of the multimodal analysisof film texts.


Conclusion

What is a multimodal transcription and what is a multimodal concordancer?What is the relation between them and how can they promote Englishstudies, both from the standpoint of the researcher carrying out detailedcomparisons of texts and, more generally, from the standpoint of teachersand students of English? Why should we be looking at type as opposed toinstance? Most answers to these questions will, hopefully, have been providedin what has been stated above. A characterization of phase and transitiontypes would seem to lead to a better understanding of the features ofdynamic genres of which TV ads are just one exponent, one that at the veryleast provides a guiding framework for students taking their first steps in theanalysis of dynamic texts.

A few concluding notes are, however, in order. While the multimodal tran-scription can be a useful starting point for an understanding of the ways inwhich resources such as gaze, gesture and language combine in typical phasalpatterns, it has its limitations, some of which have been noted above. In theearly stages of this work, Baldry and Thibault developed a dynamic version ofthe static multimodal transcription, a forerunner of MCA, which allowed theuser to generate the individual rows of a transcription through a query mech-anism, and which facilitated understanding of how visual objects and theirmovements could be analysed in terms of Halliday's metafunctions.

Unlike a lemma-based linguistic concordancer such as OCP orWordsmith, MCA does not search throug Textual data directly in thesearch for patterns but does so indirectly: it searches the corpus for patterns indescriptions which have been previously created by the researcher usingMCA's annotational tool. The annotational patterns so far used in the con-struction of a corpus of car adverts relate mainly to the metafunctional andphasal organization of the texts. As we have seen, in the analysis of The Fan caradvert, driving a car is not just a question of driving: rather a car advert can bedefined in terms of the relationship between the car driver and the car itself,with car-drive (CD) phases intertwining with car-stationary (CS) phases.

Above all, though, MCA is the result of efforts to create transcription andannotational tools that meet functional criteria in a way that was notachieved by the first generations of lemma-based concordances. In thisrespect, it has to be stressed that the needs of the research community havechanged in recent years in such a way as to privilege specialized corpora,including the analysis, whether comparative or otherwise, of specific texts,all of which are clearly reflected in the design characteristics of MCA. MCAhas been specifically designed as an online tool so that the research andteaching community can easily access it. In this respect, work is currently inprogress to establish what integrations can be achieved with other systems,for example, with HyperContext Web which uses techniques born in arti-ficial intelligence that keep track of the user's progress and which are fun-damental in teaching applications of corpora (see Pavesi and Baldry, 2000;Piastra and Lombardi, 2000).


Multimodal concordancing is in its infancy. MCA may have been on-linefor more than a year now with a constantly growing user base. But it is still aprototype that requires inputs and co-developments by various researchteams, including the efforts of specialists in computer-based multimodalannotational systems. One area, for example, in which MCA and instru-ments like MCA may be expected to develop further, in particular if theyare to be used as a teaching tool, is in terms of their incorporation ofpredefined sets of parameters so as to reflect different linguistic and multi-modal theories and traditions. Here MCA will depend heavily on theexperience gained by other research teams, in particular the work carriedout at the National University of Singapore (for example, O'Halloran andJudd, 2002). Another development will be in relation to subtitling (Baldry,2002; Baldry and Taylor, in press) where a project is underway to associatelanguage-learning subtitles with the films in MCA's database. Rather thanas faced overlays incorporated in the film itself, the subtitles, rather as hap-pens with DVD, will be generated independently of the film text, in the caseof MCA, through specific queries using the relational querying mechanism.

Acknowledgements

This paper is part of research within the Linguatel Project, an Italian inter-University project, co-financed by MURST/MIUR and co-ordinated byCarol Taylor Torsello, University of Padua and its successor the DidactasProject, co-ordinated by Chris Taylor, University of Trieste, which issimilarly financed. Michele Beltrami has developed MCA to the author'sdesign requirements as part of this project. Now in its second release,MCA is viewable through the Pavia pages of the Linguatel Website:claweb.cla.unipd.it/Linguatel/Pavia/MCA.htm or directly at: mca.unipv.it[default User name: guest and default login: iamguest; see also New Regis-tration] using Microsoft Explorer.

I thank Vauxhall Motors for the inclusion of five frames from theiradvertisement, and I also wish to thank Antonio Cerlenizza and OliverBartholomay, respectively Direttore Audi Italia and Responsabile MKT-Audi of Autogerma, Divisione Audi S.p.A, Verona and Roberta Mottino ofVerba s.r.l. Milan for their kind permission to reproduce parts of The Fanadvert for the Audi A4 model. However appreciative and supportive of theadvert's organization and goals, the interpretation given above remains, ofcourse, entirely mine.

References

Baldry, A. P. (1999) Multimodality and multimediality. In M. Karagevrekis (ed.),Compelling Learning Techniques in ESP/EAP, Proceedings of the 3rd ESP Conference., 25thSeptember 1998. Thessaloniki: Zefyros, 5-32.

Baldry, A. P. (ed.) (2000a) Multimodality and Multimediality in the Distance Learning Age.Campobasso: Palladino Editore.


Baldry, A. P. (2000b) ESP in a visual society: historical dimensions in multimodalityand multimediality. In A. P. Baldry (ed.), Multimodality and Multimediality in theDistance Learning Age. Campobasso: Palladino Editore, 41—89.

Baldry, A. P. (2000c) Introduction. In A. P. Baldry (ed.), Multimodality and Multimedial-ity in the Distance Learning Age. Campobasso: Palladino Editore, 11—39.

Baldry, A. P. (2002) Computerized subtitling: a multimodal approach to the learningof minority languages. In G. Talbot and P. Williams (eds), Essays in LanguageTranslation and Digital Learning Technologies in Honour of Doug Thompson. London:Matador-Troubador Books, 69-84.

Baldry, A. P. (in press) Promoting comparative multimodal concordancing: its role inlanguage education, teacher training, subtitling and minority language learning.In N. Vasta (ed.), Atti del Convegno Forms of Promotion, Bologna: Patron.

Baldry, A. P. and Taylor, C. (in press) Multimodal corpus authoring system: multi-modal corpora, subtitling and phasal analysis. In Proceedings of the LREC Congress,Las Palmas, June 2002.

Baldry, A. P. and Thibault, P. J. (2001) Towards multimodal corpora. In G. Astonand L. Burnard (eds), Corpora in the Description and Teaching of English. Bologna:CLUEB, 87-102.

Bernstein, B. (1971) Class, Codes, and Control, Vol. I: Theoretical Studies Towards aSociology of Language. London: Routledge and Kegan Paul.

Cook, G. (1992) The Discourse of Advertising. London: Routledge. (2nd edn 2001)Gregory, M. (1995) Generic expectancies and discoursal surprises. John

Donne's The Good Morrow. In P. Fries and M. Gregory (eds), Discourse in Society:Systemic—Functional Perspectives. Meaning and Choice in Language: Studies for Michael Hal-liday. Norwood, NJ: Ablex, 67-84.

Gregory, M. (2002) Phasal analysis within communication linguistics: two con-trastive discourses. In P. Fries, M. Cummings, D. Lockwood and W. Sprueill(eds), Relations and Functions within and around Language. London: Continuum,316-345.

Gregory, M. and Malcolm, K. (1981) Generic Situation and Discourse Phase: An Approachto the Analysis of Children's Talk. Mimeo, Applied Linguistics Research WorkingGroup. Glendon College, York University, Toronto.


Halliday, M. A. K. (1978) Language as Social Semiotic: The Social Interpretation of Languageand Meaning. London: Edward Arnold.


Halliday, M. A. K. and Hasan, R. (1985) Language, Context and Text: Aspects of Languagein a Social-Semiotic Perspective. Geelong, Victoria: Deakin University Press.(Republished by Oxford University Press, 1989.)

Lombardo, L. (2001) Selling it and Telling it. A Functional Approach to the Discourse of PrintAds and TV News. Roma: Istituto Linguistica Moderna, Luiss, Guido Carli.

O'Donnell, M. (2002) Systemics Coder. http://www.wagsoft.com/Coder/index.html

O'Halloran, K. L. andjudd, K. (2002) Systemics 1.0. [CD-ROM]. Singapore: Singa-pore University Press.

O'Toole, M. (1994) The Language of Displayed Art. London: Leicester University Press.Pavesi, M. and Baldry, A. P. (2000) Learning to read scientific texts: integrated self-

access courseware and corpora for university science students. In A. P. Baldry

http://www.wagsoft.com/Coder/index.html

http://www.wagsoft.com/Coder/index.html


(ed.), Multimodality and Multimediality in the Distance Learning Age. Gampobasso:Palladino Editore, 227-245.

Piastra, M. and Lombard!, L. (2000) The HyperContext Web Project: dynamicauthoring for distance learning. In A. P. Baldry (ed.), Multimodality and Multimedial-ity in the Distance Learning Age. Gampobasso: Palladino Editore, 247-262.

Taylor, G. and Baldry, A. P. (200la) Computer assisted text analysis and translation:a functional approach in the analysis and translation of advertising texts. InE. Steiner and C. Yallop (eds), Exploring Translation and Multilingual Text Production:Beyond Content. Berlin: Mouton de Gruyter, 277-305.

Taylor, G. and Baldry, A. P. (200Ib) Computer-assisted text analysis and translation(characteristics of interactive self-access computer modules incorporating a func-tional approach in the analysis and translation of advertising texts). InG. Torsello, G. Brunetti, andN. Penello (eds), Corpora Testualiper Ricerca, Traduzionee Apprendimento Linguistico. Studi Linguistici Applicati. Padova: Unipress, 273-292.

Thibault, P. J. (2000) The multimodal transcription of a television advertisement:theory and practice. In A. P. Baldry (ed.), Multimodality and Multimediality in theDistance Learning Age. Campobasso: Palladino Editore, 311—385.

Vasta, N. (2001) Rallying Voters: New Labour's Verbal—Visual Strategies. Padova: Gedam.Zago, S. (2002) A multimodal analysis of six television adverts. Unpublished thesis.

Dipartimento di Lingue e Letterature Anglo-Germaniche e Slave, University ofPadua.

5 Visual semiosis in film

Kay L. O'HallomnNational University of Singapore

Introduction

The aim of this paper is to investigate a method for capturing and interpret-ing the spatial and temporal dynamics of visual semiosis. This is achievedthrough the description of an analysis of a short segment from the dynamicmedium of film. The analysis is based on a systemic-functional frameworkfor film, and the use of software which allows the editing of digital videoimages in order to display visually the nature of different semiotic choicesacross a range of systems. From this point, the problematic nature of suchan enterprise becomes apparent and possible directions for future researchare suggested.

The film medium parallels a significant dimension of our experience ofthe world: it involves sequences of change and repetition in the visual andauditory realm. Film, however, involves playing with time sequences in atwo-dimensional frame to represent our three-dimensional lived-in materialexperience of the world where the faculties of hearing, sight, smell, tasteand touch are sources for sensory, and therefore semiotic, input. Thus whilelimited in the sense that the discussion presented here only incorporates thevisual aspect of semiotic exchange, this paper is nonetheless a further tenta-tive step towards incorporating the meaning of the dynamic in systemic-functional theory. For it is not only the culmination of choices made acrosssemiotic resources in their interaction with other resources that makesmeaning, but also the temporal and spatial unfolding of those choices.Although images of instances frozen in time may become lodged within ourconsciousness, generally we do not make meaning from a series of snapshotimages of the world, but rather our daily experience of the world is based onpatterns of change; that is, meanings derived from systems in flux. Ourperceptual apparatus is oriented towards detecting and assimilating changeand contrast, rather than relying on the stability and continuity which, in thenormal course of events, we learn to layer on top of that experience. Anadequate model which accounts for our social construction of the world,therefore, necessarily needs to account for changing states which have trad-itionally been the concern of other domains, which include film theory,mathematics, physics and studies of perception in cognitive science.


Although not reproduced here (Paramount refused copyright permis-sion),1 two short scenes from the film Chinatown, directed by Roman Polanski(1974), were analysed for this paper. While film is evidentiy staged and dir-ected behaviour with sequences which have been edited to achieve particulareffects, the analysis of this medium is at least a step in understanding semiosisin everyday life. That is, despite the scripted and edited nature of film per-formance, this environment provides us with some means to start investigat-ing everyday discourses-in-flux. Using a systemic-functional framework forfilm, this paper is a preliminary attempt at a method for capturing andanalysing the dynamics of visual semiosis in a digitalized video format.

The social semiotic framework presented in this paper is based onMichael Halliday's (1994) systemic-functional grammar of the Englishlanguage. Halliday's theorization of language as a social semiotic withsystems for Interpersonal, Experiential, Logical and Textual meaning hasbeen extended by O'Toole (1994, 1995, 1999) to the realm of displayedart; for example, paintings, architecture and sculpture. While O'Toole'ssystems for paintings are included in the proposed framework for film, theformer are concerned with analysing the single semiotic of the static visualimage. In film, however, there are multiple semiotic resources being usedspatially and temporally. Thus the multiple resources which result inchange, similarity and contrast are included in the systemic model for filmpresented here. In addition, O'Toole (1999) represents his theory in aninteractive CD-ROM format. This method of visual representation in theelectronic environment provides the basis for the investigations undertakenin this paper.

The focus of early studies in multimodality has primarily been directedtowards the analysis of static texts; notably Lemke's (1998b, 2003) earlypioneering work in scientific discourse and mathematics, Kress and vanLeeuwen's (1996) Reading Images: The Grammar of Visual Design, and othermore recent studies2 (for example, Baldry, 2000; Kress and van Leeuwen,2001; O'HaUoran, 2003a, 2003b; Ventola et al, forthcoming). However,current research is increasingly turning towards the analysis of the dynamictext (for example, Baldry, this volume; Callaghan and McDonald, 2002;ledema, 2001; Lemke, 1998a, 2000; Mclnnes, 1998; Martinec, 2000;Thibault, 2000; van Leeuwen 1999).

With the exception of Baldry's Multimodal Corpus Authoring (MCA)system (see this volume), however, few (if any) attempts have been made toanalyse dynamic semiosis in digitalized format using computer-based tech-nology. Baldry's MCA is a Web-based instrument which is designed foranalysing dynamic multimodal texts, that is, film and video texts whichdisplay different and constantly varying configurations of sound, image,gesture, text and language as the text unfolds in time. Baldry harnesses thepotential of computer technology to develop the MCA system with the aimof developing a metafunctionally based transcription method which canhighlight the types of shots, cuts, phases and transitions. The analyst canrecord choices in a relational database format so that comparisons can be


made across a corpus of texts. This concordance instrument thus analysesthe dynamics of semiosis through methods which involve recording anno-tated entries. As Baldry (this volume: 105) explains:

MCA does not search through Textual data directly in the search for patterns butdoes so indirectly: it searches the corpus for patterns in descriptions which havebeen previously created by the researcher using MCA's annotational tool. Theannotational patterns so far used in the construction of a corpus of car advertsrelate mainly to the metafunctional and phasal organization of the texts.

One aim of this paper is to suggest ways in which the user can directly searchfor patterns in visual Textual data. In other words, I explain how com-mercially available software can be used in conjunction with a visualgrammar to capture changing patterns in dynamic text. This exploratorystage is viewed as a first step towards a new methodology afforded by theelectronic medium which could eventually be included in a system such asBaldry's MCA. In addition, there is the potential to incorporate softwaresuch as Systemics 1.0 (O'Halloran and Judd, 2002) in such applicationsin order to analyse the linguistic choices as they unfold in time. The chal-lenge remains for us to capture and analyse choices across all semioticresources in such a way that the dynamics of meaning-making can truly beinvestigated.

A visual grammar for visual images

The inspiration for the approach adopted in this study stems from O'Toole's(1994: 24, 1999) framework for the analysis of paintings where a constituentstructure approach with ranks PICTURE, EPISODE, FIGURE and MEMBER isadopted. O'Toole's chart documents the systems of meaning for theExperiential, Interpersonal and Textual metafunctions which are respectivelylabelled representational, modal and compositional. While many of thesesystems can also be seen to operate within the realm of film, the differentmedium of production and the fact that the text unfolds in real time meanthat there are further dimensions to the analysis. Also, given the cause-effectrelations in film narrative, the logical metafunction is also included.

In the innovative CD-ROM, Engaging with Art, O'Toole (1999) creativelyutilizes computer technology in an interactive multimedia hypertextenvironment to display choices visually from his systemic-functional frame-work. For example, in Plate 5.1 O'Toole effectively captures choices fromthe system of light which function to engage the viewer in Rembrandt'spainting The Night Watch (1642). In another instance, O'Toole (1999) demon-strates how Vertical Lines are one resource which functions compositionallyin Seurat's Sunday Afternoon on the Island of La Grande Jatte (1884-1886). Healso gives an amusing demonstration of the change in meaning which wouldoccur in Botticelli's Primavera (1478) with alternative choices for the directionof Gaze for each of the figures in the painting.


Plate 5.1 Visualizing the system of light (O'Toole, 1999)

O'Toole's (1999) Engaging with Art thus represents a major advance intheory of semiotic analysis where choices in the visual semiotic are displayedvisually rather than being described linguistically. This method means thatpatterns in visual semiosis may be marked in such a way that the viewer canimmediately grasp the significance of such choices. As I describe in thispaper, there also exists the potential for displaying visually the overlappingdynamic choices in-flux across systems. The advantages of this approachmay be appreciated through a comparison with an alternative methoddeveloped by Thibault (2000).

As a major step in theorizing a comprehensive semiotic analysis of atelevision advertisement,3 Thibault (2000: 374—385) proposes a static lin-guistic description in table format with dimensions 'Visual Image', 'KinesicAction', 'Soundtrack' and 'Metafunctional Interpretation of Phases and Sub-phases' which are denned as constituting 'an intermediate level of analysiswhich lies between the microlevel lexicogrammatical, kinesic, and imageselections and the global structuring of the text as a whole' (Thibault, 2000:365). Following Gregory (1995, 2002), Thibault (2000: 325-326) definesphase as 'a set of co-patterned semiotic selections that are co-deployed in aconsistent way over a given stretch of text'. Here the change of phase ismarked by a salient metafunctional choice which marks the transition.


Although a highly significant and useful methodology for capturing integra-tively multimodal social meaning-making, the linguistic description does notcapture the import of such choices and also it fails to map visually thechoices as a sequence of continuity and change.

The potential exists for the viewer to actively engage with the digitalizedfilm segments to illustrate the impact of different semiotic choices. This isachievable through the use of facilities in video editing software such asAdobe Premiere 6.0, which permits the user to segment a digitalized videoclip into sections according to frame number (for example, 1, 2, 4 and6 frames) or time intervals (for example, 1, 2, 4 seconds). The software allowsthe user to manipulate the visual footage in multiple ways; for example, theimage may be adjusted for brightness, contrast, colour (which can bereplaced and matched) and special effects such as blurring, distortion, per-spective, edge definition and shadowing (to name but a few) may be applied.The software also allows the user to create multiple transparent matteswhich act as overlays on the original film footage so that text can be insertedand lines, vectors, figures, outlines and shadings can be drawn. In addition,visual transitions between parts of the footage can be marked in variousways. These facilities allow the user to mark explicitly the nature of visualsemiotic choices which have been made. Just as one enters a linguistic analy-sis by tagging the linguistic text in software such as Systemics 1.0, so theanalyst can enter the analysis of the visual images through direct Textualengagement.

In the following discussion of the analysis of the visual dimensions of thedynamics of the film footage, I do not consider the soundtrack. Therefore, inthis limited discussion it is important to keep in mind Baldry's (this volume:94) claim that transitions in phases take many forms:

Thus, transitions are not necessarily equated with the cutting from one shot toanother, nor indeed with what is happening in the visual. While transitions willoften be related to what is happening in the visual, this will not always be the case[. . .] transitions, as Thibault (2000: 320) and Gregory (2002: 323) have pointedout, essentially relate to changes in the metafunctional organization of the text andas such may very well be related to changes in the soundtrack and not just to whathappens in the visual.

Video-editing tools, therefore, allow the user to highlight the different semi-otic choices visually and view the impact of such choices when they com-bine in the text in real time. The method which was adopted for this paperinvolved the use of Adobe Premiere 6.0 to explore how salient semioticchoices may be highlighted in a short extract from the film Chinatown. How-ever, as previously noted, unfortunately it has not been possible to reproducestill frames from this analysis in this publication due to Paramount Studio'srefusal to give copyright permission. Nonetheless, the results of the visualanalysis are described in some detail.


A systemic-functional framework and Chinatown (1974)

The systemic-functional model proposed here4 has been developed in con-junction with the film theory presented in Bordwell and Thompson's (2001)Film Art: An Introduction. Bordwell and Thompson are concerned with theimage in the visual frame and the accompanying audio soundtrack. In whatfollows, I discuss the proposed systemic framework and demonstrate howsuch an approach may be applied for the analysis of compositional andInterpersonal meaning in two short scenes from Chinatown. In order to situ-ate the analysis, I first briefly discuss this film.

Written by Robert Towne and produced by Robert Evans with directorRoman Polanski and production designer Richard Sylbert, Chinatown is adetective film set in 1937 in Los Angeles with Jack Nicholson as Jake Gittes(the private detective), Fay Dunaway as Evelyn Mulwray (the wife of HollisMulwray, chief engineer of Water Energy and Power) and John Huston asNoah Cross (former partner with Hollis Mulwray of a private water com-pany for LA). The plot unfolds as Jake unearths the corruption behindCross's plan to build a new reservoir. This involves investigation of themurder of Hollis Mulwray who opposes the plan, and unearthing the his-tory of Evelyn Mulwray who was raped by her father Noah Cross at the ageof 15. Cross's partner Hollis Mulwray subsequently married Evelyn andsupported her daughter Katherine. After Jake becomes aware of the reasonsfor Evelyn's actions, he organizes her escape from her father with Katherine.However, Cross forces Jake to disclose their whereabouts with the result thatEvelyn is killed by the police. Jake once again unwittingly aids the death ofsomeone he is trying to protect, which is a repeated scene from the days inwhich he was a police officer in Los Angeles' Chinatown.

Chinatown has been the subject of much discussion in film theory (forexample, Eaton, 1997; Heisner, 1997; Krutnik, 1991; Tuska, 1984).Based on the history of pumping water to Los Angeles in the first quarterof the twentieth century, Eaton (1997: 43) explains that Chinatown is 'acomplex detective thriller with dimensions which are political (about thenature of power), sexual (about the nature of gender), metaphysical (aboutthe nature of the evil), psychological (about the nature of the self) andphilosophical (about the nature of knowledge)'. According to Eaton (1997)the subtext is concerned with the theme of American greed. In addition,Heisner (1997: 63) explains that Robert Towne has explored the 1930spopular conception of 'the inscrutable Orient' which is 'unknowable; it isdense and powerful and corrupt'. In the film Chinatown, this view is appliedto the entire world.

The proposed systemic-functional framework involves classifying the filmaccording to type, form and genre. The semiotic analysis of the film is basedon a metafunctionally organized rank constituent structure with ranksFilm Plot, Sequences, Scene, Mise-en-Scene and Frame. Though beyond thescope of this paper, the notion of metafunctionally based phases and transi-tions may be incorporated within this framework. The aim of the analysis


undertaken here, however, is to demonstrate how a visual grammar can beimplemented in the dynamic digitalized environment of film.

fiction, documentary, experimental and animatednarrative, categorical, rhetorical, abstract and associationmultiple types; for example, narrative films include sciencefiction, western, musical, comedy, suspense, and actionthrillers with sub-genres horror, detective, hostage andgangsterFilm PlotSequencesScenesMise-en-Scene (the shot)Frame

Film type /form

Bordwell and Thompson (2001) categorize films as fiction, documentary,experimental and animated based on how the film material was chosen,arranged and the nature of the filming. They further propose that filmsalso have a basic film form, or a system of relationships among the partswhich may be categorized as Narrative, Categorical, Rhetorical, Abstractand Associational. The narrative form, however, is dominant in mainstreamcinema. Bordwell and Thompson (2001: 60) define narrative as 'a chain ofevents in cause-effect relationship occurring in time and space'. In a narra-tive film, the viewer is presented with the plot, 'the arrangement of materialin the film' from which the viewer individually creates the story 'on the basisof cues in the plot' (ibid.: 62). Most films employ narrative where causalityand time are central.

In classic Hollywood cinema, the action usually springs from individualcharacters as causal agents where the narrative usually centres on personalpsychological causes such as decisions, desires, choices and traits of char-acter (Bordwell and Thompson, 2001). The narrative subordinates time,motivation and other factors to the cause-effect sequence. There is usuallystrong closure where the causal chain is completed with a final effect. 'Weusually learn the fate of each character, the answer to each mystery, and theoutcome of each conflict' (ibid.: 77).

In Chinatown Jake Gittes desires to know the truth surrounding Evelyn andthe murder of Hollis Mulwray. As Eaton (1997) explains, Evelyn chooses notto speak because she knows too much about her father's corruption andpower to share Jake's faith in revelation. Jake considers her a betrayer but helearns that in fact she is the victim. The cause-effect relations in Chinatownare extremely complex as new revelations continually occur in the unfoldingof the plot.

Film type:Film form:Genre:

Ranks:


Genre

There are no rigid criteria to define the different genres of film (Bordwell andThompson, 2001). Some classifications are based on subject/theme (forexample, crime for gangster movies), while others are defined by emotionaleffect (for example, amusement for comedy). Genre conventions are alsobased on plot, thematic development, film techniques and iconography. Fur-ther to this, genres change and new hybrid types are continually emerging.However, despite this fluidity the audience generally recognizes genre con-ventions. Genres are seen to be institutionalized and ritualized dramas 'whichare satisfying because they reaffirm cultural values . . . [such as] self sacri-ficing heroism, the desirability of romantic love' (Bordwell and Thompson,2001: 99). Bordwell and Thompson (2001) further explain that these reaf-firmations distance the viewer from real social problems and the more finiteand anxiety-ridden aspects of life such as death, disease, breakdown andinsecurity. Genres may also be seen to 'exploit ambivalent social values andattitudes' which 'arouse emotion by touching upon deep social uncertaintiesbut then channel those emotions into approved attitudes' (ibid.: 99).

Chinatown is a detective story with an investigative structure (Eaton, 1997).'As Poe so clearly put it, the detective exists "to play the Oedipus'" (ibid.:17), the truth seeker. Chinatown is a story where 'wrongs can ultimately beuncovered but the seeker after truth is not only completely incapable of right-ing them but his very search will only make matters worse' (ibid.: 21).

Chinatown is also recognized as film noir and, more specifically, reflects theorigins of the neo-noir. The subject of much study (for example, Christopher,1997; Hirsch, 1981; Kaplan, 1998; Krutnik, 1991; Palmer, 1994; Tuska,1984; Voytilla, 1999], film noir is a descriptive term for American crime filmfrom early 1940s to late 1950s where doomed men are obsessed with seduc-tive women, as exemplified by Double Indemnity (1944) and Scarlet Street (1945).In the 1960s and 1970s films with noir flourishes include Klute (1971), PlayMisty for Me (1971), Taxi Driver (1976) and Chinatown (1974).

Definitions of film noir vary but there seems to be general agreement thatthe term designates films with a low-key visual style which contrasts to thebright balanced studio look of the 1930s. There are noir movies of differentgenres, for example, mystery, suspense thriller, psychological drama, andgangster films (Krutnik, 1991). Critics generally agree that there is also anobliqueness and often confused temporal narrative plot. There is usually ageneral mood of dislocation and bleakness, and the noir world is deceptiveand uncertain. ' "The world is a dangerous place" is one of the axioms ofnoir' (Hirsch, 1981: 13).

Chinatown, however, is filmed in the non-expressionistic 'classical' style ofPanavision and Technicolour with a straightforward narrative style. How-ever, 'the cynicism and despair which permeates the social vision of the filmnoir... is present... in the final act of this Polish exile's [Roman Polanski's]film' (Eaton, 1997: 57-58). However, according to Eaton (1997: 58), thedepiction of Evelyn Cross Mulwray is where the noir-ish influence is most


obvious. 'The dark lady, the spider woman, the evil seductress who temptsman and brings about his destruction' [Place, 1998: 47] is how the "femalearchetype" of film noir has been characterized and this is the image of thefemale lead which is now consciously evoked [in Chinatown]' (ibid.: 58).

The figure of the woman in film noir has been the focus of feminist filmtheory since Chinatown was produced. The emergent newfemmefatale in filmsin the 1990s, for example, Basic Instinct (1992), is 'redefined as a sexualperformer within a visual system which owes as much to soft-core porn-ography as it does to mainstream Hollywood' (Stables, 1998: 172-173). Thenew woman takes an active role in initiating sexual practices which areperceived as deviant, marginal or transgressive to the dominant culture. Inthe analysis below, we shall investigate the semiotic construction of EvelynCross in the role of 'spider woman' which has subsequently led to suchconstructions of women in contemporary cinema.

The Film Plot and Sequence

The form which gives rise to the plot is the overall interrelation amongvarious systems of elements and every element in this totality has one ormore functions (Bordwell and Thompson, 2001). In the model presentedhere, the Film Plot is constructed from the series of Sequences where themotivation is similarity and repetition, and difference and variation.

In Chinatown., repetitive elements and motifs are significant (Eaton, 1997;Heisner, 1997). The Scenes take place in different locations which reinforcethe theme of drought-stricken Los Angeles. The symbolism of water con-tinually appears in the unfolding of the plot with constant screen imagesand references to water. A second motif is the lens in the form of glasses, carmirrors and binoculars which contribute to the theme of distorted vision.These themes of voyeurism and blindness are 'not simply about seeing, it isabout seeing wrongly' (Eaton, 1997: 29). Other motifs in Chinatown., forexample, the horse and rider, are metaphors for desire and sexuality. In theMise-en-Scene analysed below, we shall see these themes reappear in differ-ent forms.

Scene and Mise-en-Scene

The Mise-en-Scene is concerned with everything which is seen within theframe as it unfolds in time together with the accompanying soundtrack. Assoon as the camera shot changes, even though still centred on the samesetting, we will be concerned with a new Mise-en-Scene. The Mise-en-Scenecomplex, or the unfolding series of Mise-en-Scene, forms the Scene. Thetotal of Scenes forms the Sequence, which in film theory is the term for thefragmentation of the film into segments.

The Mise-en-Scene forms the basic unit for analysis because the majorsystems for each metafunction across the semiotic resources are operationalat this rank. For instance, the higher rank of Sequence does not allow


comprehensive analysis of the choices across semiotic resources, while thelower rank of Frame frozen in time excludes analysis of speech, music andother sound effects.

Following Baldry (this volume) and Thibault (2000), the soundtrack canmark a transition, and in the case of the framework presented here, thetransition may take place within one Mise-en-Scene. In effect, this wouldcreate a 'rankshifted' Mise-en-Scene. That is, if the soundtrack changes toindicate a transition within the single camera shot, we have a Mise-en-Sceneembedded within the ranking Mise-en-Scene of the camera shot. In a similarmanner, the soundtrack can continue across several Mises-en-Scene to forma Mise-en-Scene complex. The Mise-en-Scene complex is therefore con-strued by the nature of the setting and other structural elements whichinclude the soundtrack.

As displayed in Table 5.1, the Mise-en-Scene is analysed according toVisual Imagery, Speech, Music, Sound Effects and the subsequent Inter-weaving of the Visual Imagery and the Soundtrack. For Visual Imagery, theranks are Movement-Action-Event in a shot, temporal episode, temporalfigure and temporal member. In addition to making dynamic O'Toole'ssystems for paintings, further systems are included for the analysis of thetemporal unfolding of the text. At the rank of Mise-en-Scene, these includesystems for: (a) Interpersonal meaning such as Patterns (Kinesic, Proxemic,Rhythm, Gaze and Shape), Duration of the Image, Speed of Motion andPoint of View; (b) Representational meaning, for example, Movement-ActionSequence; (c) Logical meaning, for example, Narrative Cause-Effect Rela-tions; and (d) Compositional meaning, for example, Changes in Gestalt, On-Screen/Off-Screen Space, Camera Angle, Camera Level, Camera Distanceand Mobile Frame. The Mobile Frame allows changes in the camera positionin the Mise-en-Scene. The Mobile Frame thus interpersonally orients theviewer towards the image and furthermore contributes to the represen-tational meaning in the form of the Point of View constructed within the film.

The analysis described below is concerned with the visual imagery in twoMise-en-Scene from Chinatown. As the goal of this exercise is to demonstratethe usefulness of the Textual application of the visual grammar, the discus-sion is only concerned with selected choices in systems for Interpersonal andcompositional meaning. The original analysis appears in the form of a moviewhere choices from the visual systems are marked on the digitalized film clipfrom Chinatown as they unfold in real time. We may note that, composition-ally, the Framing in Chinatown (which may be marked visually) is widescreenwith ratio 16:9. This allows the action sequences to be framed against anexpansive setting which contributes to establishing one of the key themes ofChinatown'. Los Angeles in a drought.

The analysis of two Mises-en-Scene in Chinatown

The first Mise-en-Scene occurs at the end of the Scene where Jake andEvelyn meet in a restaurant. Jake is largely unsuccessful in his attempts to get


further information from Evelyn, and in the ensuing Mise-en-Scene outsidethe restaurant, a somewhat angry and frustrated Jake informs Evelyn thather husband may have been murdered. In the newly released 1999 DVDversion of Chinatown., director Roman Polanski states that this scene outsidethe restaurant is one of his favourite shots. We shall soon appreciate at leastsome of the reasons why Polanski thought this way about this part of thefilm. The dialogue which takes place outside the restaurant is reproducedbelow.

Key: EM: Evelyn Mulwray JG: Jake Gittes

EM: Oh no ... I have my own car. Ahh . . . the Packard.JG: Wait a minute sonny [to the car attendant]. I think you [Evelyn] had better

come with meEM: But why. There's nothing more to say. Will you get my car please [to the

attendant].JG: Okay go home. But in case you're interested, your husband was murdered.

Somebody's been dumping thousands of tonnes of water from the city'sreservoirs and we are supposed to be in the middle of a drought. He foundout about it and he was killed. There's a waterlogged drunk in the morgue -involuntary manslaughter if anyone wants to take the trouble which theydon't. It seems like half the city is trying cover it all up which is fine by me.But Mrs Mulwray. I goddamn near lost my nose and I like it. I like breathingthrough it. And I still think that you're hiding something

EM: Mr Git—tes [as JG drives away]

The restaurant Mise-en-Scene

The viewer's perception is attuned to difference rather than prolonged stim-uli, and attention is typically focused through contrasting patterns andmovement. However, in the selected Mise-en-Scene which occurs at the endof the restaurant Scene, the camera focuses on Evelyn (pictured from theshoulder upwards) who is silent and virtually motionless. Kinesics andRhythm through movement are absent. What functions to make this Mise-en-Scene so compelling? Through the analysis, we see that there are manysimultaneous choices at work which focus the viewer's attention on thisportrayal of Evelyn as the 'spider woman'.

The Lighting Quality, Lighting Intensity, Lighting Direction and LightingSource in the restaurant scene function to make Evelyn visually salient. Thesoft background Lighting may be marked visually through the use of thespecial effect 'lens flare' which allows the light source to be highlighted. Aswell as providing a contrast for the next Mise-en-Scene, the choice of thewarm reddish colours from the system of Colour/Cohesion has implica-tions for more immediate Interpersonal and Experiential meanings as weshall soon see.

At the rank of Member, the Clarity and Focus of Evelyn's beautiful, paleand sculptured face attracts the viewer's attention. Further to this, Evelyn's

Table 5.1 Functions and systems in the Mise-en-Scene

Semiotic Resources/Rank Modal Representational Logical C ompositional

MISE-EN-SCENECOMPLEX(the edited scene)

Contrasts Narrative continuity and Cause-effect relations Continuity anddiscontinuity discontinuity

MISE-EN-SCENEThe Temporal-Spatial FrameComplex Relation: The Shot

Visual Imagery

Movement-Action-Event in a Shot Patterns:KinesicProxemicRhythmGazeShape

Colours and ContrastLighting QualityLight IntensityLighting DirectionLighting SourceClarityFocusFilm TonalitySpecial EffectsDuration of ImageSpeed of MotionPoint of View (Viewer)

Movement-Action-EventSequenceFigures/ObjectsNature of ScenePropsLighting ColourNarrative as CauseEffectRelationsPoint of ViewVisual Motifs

Narrative Cause-Effect Relations

Frame DimensionFrame ShapeChanges in Gestalt:

FramingHorizontalVerticalDiagonal

Colour Cohesion/ContrastPerspective RelationsOn-Screen/Off-ScreenSpaceCamera AngleCamera LevelCamera DistanceMobile FrameFilm Editing

Semiotic Resources/Rank Modal Representational Logical Compositional

Temporal Episode Relation to Movement-Action-Event:

ScaleDepthCentralityRelative Prominence

DurationClarityFocusLight

Sequence of Sub- Actions,Side Sequences andEventsInterplay of Actions

Contribution toNarrativeCause-EffectRelations

Relative Relation ofAction in ChangingGestaltSubframingParallelism andOppositionRelative On-Screen/Off-Screen SpaceCamera AngleCamera LevelCamera Distance

Temporal Figure Colour Coordination/ContrastColour IntensityCostume StyleFrontal ViewChange in SizeChange in ProminenceGaze PatternFocusDepthLight

Character of FigureCostumeBody Behaviour/GestureProps

Contribution toCause-EffectRelations throughIntertextualMotif

Relative Position inChanging GestaltSubframingParallelism andOppositionRelative On-Screen/Off-Screen SpaceCamera Angle-Camera LevelCamera Distance

Temporal Member ColourColour IntensityStyle of Costume PartMakeupFacial Expression

Body PartMakeupFacial ExpressionGestureRole in action

Contribution toCause-EffectRelations throughIntertextualMotif

Relative Position inChanging GestaltSubframing

Semiotic Resources/Rank Modal Representational Logical Compositional

GestureLightChange in SizeChange in ProminenceFocusDepth

Parallelism andOppositionRelative on-screen/off-Screen spaceCamera level and angleCamera Distance

Soundtrack

Speech

Music

NegotiationSpeech FunctionMoodModalityPolarityAttitudeCommentAppraisalLexical 'Register'TonePitchVolume

IdeationTransitivityTenseLexical ContentErgativityVerbal Motifs

Conjunction andContinuityLogico-SemanticRelations

IdentificationThemeCohesionInformation

VolumePitchTimbreRhythmFidelityBeat

Genre:Experiential ContextIntertextualityMusical Motifs

Narrative Cause-EffectRelations

Sound Perspective(Diegetic, Non-Diegetic)

Frame24 Frames/Second

Viewed as Mise-en-Scene

Semiotic Resources/Rank Modal Representational Logical C ompositional

Sound Effects VolumePitchTimbreRhythmFidelityBeat

Experiential ContentIntertextualityOral Motif

Narrative Cause-EffectRelations

Sound Perspective -Diegetic and Non-Diegetic

Visual Imagery +Soundtrack

Interweaving VisualImagery and Sound

Direction of Engagementthrough ForegroundedSemiotic ChoiceChange in Phase Marking

Development of theNarrative Plot for StoryLine through DirectedContent Input

Development ofCause-EffectRelations

Organization of theUnfolding of theNarrative


Gaze towards Jake (which may also be marked visually through vectors) isoblique and so the viewer can openly scrutinize her face, Makeup and Cos-tume throughout the extended Duration of the Image. After her husband'sfuneral, Evelyn is wearing a black dress and a hat with a netted black veilwhich covers the top half of her face. Her Gaze in effect is veiled. Jakecomments in the next Mise-en-Scene, 'And I still think you are hiding some-thing'. Here the motif of distorted vision is reinforced. In this case, Jake isnot gazing through a camera or car mirror, rather he is trying to penetratethe protective veil through which Evelyn views the world.

The use of Colour in the restaurant scene is significant for several reasons.Digital colour matching (which can be displayed) reveals that Evelyn's redlipstick exactly matches the colour of the couch upon which she is seated.The motif of sexuality is represented through this use of the colour red inEvelyn's makeup which coheres with the intimate setting. The characteriza-tion of Evelyn as the 'spider woman' is thus created; she is veiled, oblique,sexual and potentially dangerous. This portrayal of Evelyn largely remainsin place until the final scenes in the movie.

The street Mise-en-Scene

In the next Mise-en-Scene the viewer is confronted with a bright street sceneas Evelyn and Jake walk into the open glare of sunlight outside the restaur-ant. Compositionally, the contrast in Colour Cohesion/Contrast may be dis-played through the use of colour matching and replacement. The analystbecomes conscious that the dominant background colour of bright yellowhas replaced the subdued colours in the restaurant. The dark quiet world ofthe spider woman is contrasted to the stark brightness of the street wheresunlight shines against the buildings and normal day-to-day activity takesplace as the attendant rushes to open the car door for Jake and Evelyn.

Through the use of overlays and drawing tools to mark the perspectiveand the placement of the Figures in the Mise-en-Scene, it may be appreciatedthat the On-Screen Space initially occupied by the attendant works perfectlyin conjunction with the perspective provided by the buildings. Activities areordinary, orderly and public in this Mise-en-Scene where the sound of carhorns is heard and people walk down the street arm in arm.

Jake and Evelyn become the focus of attention as they walk out ontothe street. They continue to occupy the central On-Screen Space in theremainder of the Mise-en-Scene, and a dynamic visual tracing of the outlineof their two Figures reveals the perfect compositional balance that is achievedwithin the widescreen frame format. The Colour Contrast provided by thebright background also functions to highlight the figures of Evelyn and Jake.The effect of the light provided by the sun may be marked visually throughthe use of'lens flare' to insert an accentuated light source. The analyst againbecomes aware that the motif of hot dry weather is invoked.

As the Mise-en-Scene unfolds, the exchange between the two centralcharacters becomes increasingly intense as Jake responds with frustration to


his lack of understanding of the situation. The intense gaze between Jakeand Evelyn, which accompanies her refusal of his offer to drive her home,may be indicated visually by vectors. The On-Screen Space dominated byEvelyn and Jake continues to remain perfectly balanced, and the analyst canbegin to appreciate how effectively the camera work and background settingfunction in this Mise-en-Scene. In addition, there is a lightly coloured ban-dage on Jake's nose which is marked with visual prominence despite itscohesiveness with the background colours. This visual prominence of thebandage is matched by the linguistic choices in the dialogue which takesplace as we shall see in a moment.

The triangle of social relationships between Jake, Evelyn and the carattendant is construed visually as well as linguistically. The attendant is aminor participant as indicated by his backgrounded physical position in theMovement-Action-Event when Jake and Evelyn walk out of the restaurant.Jake's use of the vocative 'sonny' in the command 'Wait a minute sonny'reinforces this position. Jake's attempts at exercising power over Evelyn,however, do not succeed.

Jake fails in his bid to drive Evelyn home, and there is a pause before heturns to confront her. Evelyn remains detached and supposedly nonchalantby focusing her Gaze on her gloves, which may be indicated visually by linevectors. Evelyn's hand movements may also be highlighted visually to indi-cate Gesture. After a short silence, the Interpersonal relations between Jakeand Evelyn intensify. The Gaze becomes direct and focused as the Proxemics,which may be displayed by visual vectors, decrease. The Mobile Frame hasbeen brought into play so that the Camera Distance is decreased. This com-positional strategy further draws the viewer into the exchange between Jakeand Evelyn. The Interpersonal intensity of Jake's delivery continues as heexplains that Evelyn's husband was murdered. Evelyn's Gaze, which againmay be marked by visual vectors, shifts downwards as Jake refers to her latehusband. Jake, however, continues regardless of Evelyn's silent response.

When Jake refers to a situation where he was physically attacked and hisnose sliced by a knife [hence the bandage], 'but Mrs Mulwray I goddamnnear lost my nose', the Interpersonal intensity of the exchange increases.The use of vectors may explicidy demonstrate how distance in the Proxem-ics has again decreased with a resulting increase in the intensity of gaze. Inaddition, Jake's use of'goddamn near' reinforces the affect of his speech toEvelyn, which is somewhat mocking given that he addresses her as'Mrs Mulwray'.

The climax in this Mise-en-Scene is reached when Jake accuses Evelyn of'hiding something'. Here the motif of the truth seeker looking through a veilof deception is reinforced. While he is correct that Evelyn is withholdinginformation, it is not exactly the sort that Jake envisages. However, in theremainder of the street scene, Roman Polanski allows the viewer to gainsome insight into Evelyn's situation.

The final frames of the Mise-en-Scene capture one of the rare momentsin Chinatown where the Point of View switches from Jake to Evelyn. The


viewer is aware of Evelyn's latent appeal to Jake ('Mr Git—tes') as he drivesaway. Evelyn maintains her position within the Frame but the MobileCamera effectively retreats to leave Evelyn pictured completely alone in thestreet scene. The appeal is reinforced through Evelyn's downcast Gaze andGesture of moving her hand to her throat.

At this stage, the viewer gains an understanding of Evelyn's efforts at self-control. With her eyes temporarily closed, the absence of Gaze and thecontinuing Gesture are made salient through the Duration of the Image andthe Framing of Evelyn within the street scene. Jake's departing car is theonly Temporal Episode in relation to Evelyn's Movement-Action-Event. Asomewhat resolute Evelyn opens her eyes with a straight gaze realized as ahorizontal vector as her car is reversed by the attendant. In the finalframes of this Mise-en-Scene, Evelyn has again opened her eyes to a worldwhich does not understand her position nor the reasons for her actions.

Conclusion

This necessarily incomplete description of the analysis of two Mises-en-Scene from Chinatown seeks to describe how a visual grammar may beapplied to the dynamic visual image. In the discourse analysis of a linguistictext, the analyst directly engages with the linguistic choices which have beenmade in order to interpret the text. In a similar manner, the description ofthis analysis seeks to demonstrate the effectiveness of directly engaging visu-ally with a Mise-en-Scene to make salient the choices which have been made.Through such an analysis, we start to appreciate the reasons why directorRoman Polanski favoured this particular scene in Chinatown.

The bright public street setting marks a stark transition from the intimaterestaurant scene where Evelyn's sexuality is marked. The compositionalaspects of the narrow street setting are perfect; the actors are framedthrough perspective, on-screen space, colour cohesion and contrast. Theyellow tones of the background setting with light and shadows provided bythe sun, the buildings and other lighting effects further enhance the visualsalience of the two actors in the setting. The camera moves in to record thegrowing intensity of the exchange between Jake and Evelyn against a back-drop of day-to-day life which continues despite the drama being played outbefore the viewer's eyes. Through the use of gaze, gesture and proxemics thevisual aspects of the interaction effectively construct Jake's growing frustra-tion and anger with Evelyn in his search for truth. The camera later lingersto capture a subtle shift in the point of view where the unenviable positionof Evelyn is signalled to the viewer. Jake's arrogance transforms her per-ceived strength into a web of deceit and corruption which rightfully shouldbe attributed to her father.

Roman Polanski ensured that the usual generic conventions were not fol-lowed in the movie Chinatown. In Robert Towne's original script, Evelyn issaved and her father exposed. Thus the usual generic tropes such as 'lovetriumphs' and 'youth defeats old age' and 'corruption resulting in a new


healthier social order' are eliminated by Polanski. As Heisner (1997:63) states,in Chinatown 'Evil and power have triumphed, corruption has won out'. AsHeisner further explains, the pessimism of the ending extends beyond Jake'scynicism. In the final line in the film Jake is told ' "Forget it Jake. It's China-town. It's Noah Cross. It's the power structure. It's the world"' (ibid.: 64).

The proposed methodology for analyzing dynamic visual images, how-ever, presents a range of difficulties. First, it proved near impossible tosimultaneously record dynamically the metafunctional choices across thedifferent semiotic systems, even in the case when each metafunction is con-sidered separately. The reason is twofold: first, the complexity and range ofsystems from which options are chosen; and second, the problem of thetemporal unfolding of those choices in real time.

In the first case, visually marking semiotic choices across a range of systemsfor one metafunction proves problematic. For example, recording on-screenspace for compositional meaning precluded including choices for colourcohesion and contrast because the resulting footage became too dense andconfused. In a similar manner, choices from Interpersonal systems such aslighting and colour could not be combined with the analysis of gaze andproxemics. This situation gave rise to the second problem. In attempts tocombine the metafunctionally based analysis in real time, the temporalunfolding of the resultant footage for each metafunction was too fast for theviewer to grasp the significance of the different aspects of the analysis. Itbecomes apparent that we perceive so much visual data in a short time spanthat it is impossible to mark this visually in real time. If the analysis for allfour metafunctions were recombined in the footage, the problem would beexacerbated.

In order to overcome the difficulties described above, it is suggested thatthe analysis for one system should be documented and the shifts annotatedwithin a system such as the MCA. After the analysis for each system hasbeen entered, the resulting footage could be recombined to mark salienttransition points which occur as the result of the conflation of choices acrossthe systems. These higher-level transition points could also be recorded in adatabase format.

Despite the difficulties of using a visual grammar to interact directly withthe dynamic visual image, the usefulness of such an approach is that theanalyst becomes sensitized to meaning through choice in visual semiosis. In amanner analogous to language, the analyst can only become attuned tometafunctionally based choices if one has in a sense directly engaged with thetext. The advances in computer technology mean that this is becoming a veryreal option for our investigation of the dynamics of semiosis in real time.

Notes

1 Despite repeated written requests to Paul Hrisko, the Manager for the Film ClipLicensing Division for Paramount Studios, copyright permission to reproducestill frames from the movie containing the analysis of Chinatown was not given. I


am, however, most grateful to Roman Polanski who kindly wrote in support ofmy requests for copyright permission.

2 See Visual Communication (Sage Publications), a journal devoted to the theory andanalysis of visual images and multimodal texts.

3 See also Baldry (this volume) for the analysis of car advertisements.4 See ledema's (2001) social semiotic framework and analysis of a television

documentary. His framework consists of six levels: frame, Shot, Scene, Sequence,Generic Stage and Work as a whole.

Acknowledgements

I would like to thank Michael O'Toole for his kind permission toreproduce Plate 5.1 from the CD-ROM Engaging with Art (Perth: MurdochUniversity, 1999) [copyright Michael O'Toole] with acknowledgement tothe Rijksmuseum of Amsterdam for the original image of Rembrandt's TheNight Watch.

References

Baldry, A. P. (this volume) Phase and transition, type and instance: patterns in mediatexts as seen through a multimodal concordancer, 83—108.

Baldry, A. P. (ed.) (2000) Multimodality and Multimediality in the Distance Learning Age.Campobasso, Italy: Palladino Editore.

Bordwell, D., and Thompson, K. (2001) Film Art: An Introduction (6th edn). New York:McGraw Hill.

Gallaghan, J. and McDonald, E. (2002). Expression, content and meaning in lan-guage and music: an integrated semiotic analysis. In P. McKevitt, S. O'Nuallainand C. Mulvihill (eds), Language, Vision and Music. Selected papers from the 8th Inter-national Workshop on the Cognitive Science of Natural Language Processing, Galway, Ireland,1999. Advances in Consciousness Research, Volume 35. Amsterdam: Benjamins, 205—220.

Christopher, N. (1997) Somewhere in the Night: Film Noir and the'American City. New York:The Free Press.

Eaton, M. (1997) Chinatown. London: British Film Institute.Gregory, M. (1995) Generic expectancies and discoursal surprises: John Donnne's

The Good Morrow. In P. H. Fries and M. Gregory (eds), Discourse in Society: Systemic-Functional Perspectives. Meaning and Choice in Language: Studies for Michael Halliday.Norwood, NJ: Ablex, 67-84.

Gregory, M. (2002) Phasal analysis within communication linguistics: two contrast-ive discourses. In P. Fries, M. Cummings, D. Lockwood and W. Sprueill (eds),Relations and Functions within and around Language. London and New York: Con-tinuum, 316-345.

Halliday, M. A. K. (1994) An Introduction to Functional Grammar (2nd edn). London:Arnold.

Heisner, B. (1997) Production Design in the Contemporary American Film. Jefferson: MeFarland.

Hirsch, F (1981) The Dark Side of the Screen: Film Noir. New York: Da Capo Press.ledema, R. (2001) Analysing film and television: a social semiotic account of hos-

pital: an unhealthy business. In T. van. Leeuwen and C. Jewitt (eds), Handbook ofVisual Analysis. London: Sage, 183—204.


Kaplan, E. A. (ed.) (1998) Woman in Film Noir (rev. edn). London: British FilmInstitute.


Kress, G. and van Leeuwen, T. (2001) Multimodal Discourse: The Modes and Media ofContemporary Communication. London: Arnold.

Krutnik, E (1991) In a Lonely Street: Film Noir, Genre and Masculinity. London: Routledge.Lemke, J. L. (1998a) Metamedia literacy: transforming meanings and media. In D.

Reinking, L. Labbo, M. McKenna and R. Kiefer (eds), Handbook of Literacyand Technology: Transformations in a Post-Typographic World. Hillsdale, NJ: Erlbaum,283-301.

Lemke, J. L. (1998b) Multiplying meaning: visual and verbal semiotics in scientifictext. InJ. R. Martin and R. Veel (eds), Reading Science: Critical and Functional Perspec-tives on Discourses of Science. London: Routledge, 87—113.

Lemke, J. L. (2000) Multimedia demands of the scientific curriculum. Linguistics andEducation, 10(3): 247-271.

Lemke, J. L. (2003) Mathematics in the middle: measure, picture, gesture, sign andword. In M. Anderson, A. Saenz-Ludlow, S. Zellweger and V Cifarelli (eds),Educational Perspectives on Mathematics as Semiosis: From Thinking to Interpreting to Know-ing. Ottawa: Legas Publishing, 215-234.

Mclnnes, D. (1998) Attending to the instance: towards a systemic-based dynamicand responsive analysis of composite performance text. Unpublished Ph.D.thesis. University of Sydney.

Martinec, R. (2000) Construction of identity in Michael Jackson's 'Jam'. SocialSemiotics, 10(3): 313-329.

O'Halloran, K. L. (2003a) Educational implications of mathematics as a multi-semiotic discourse. In M. Anderson, A. Saenz-Ludlow, S. Zellweger, and V VCifarelli (eds), Educational Perspectives on Mathematics as Semiosis: From Thinking toInterpreting to Knowing. Ottawa: Legas Publishing, 185-214

O'Halloran, K. L. (2003b) Intersemiosis in mathematics and science: grammaticalmetaphor and semiotic metaphor. In A.-M. Simon-Vandenbergen, M. Taverni-ers, and L. Ravelli (eds), Grammatical Metaphor: Views from Systemic FunctionalLinguistics. Amsterdam: John Benjamins, 337—365.

O'Halloran, K. L. and Judd, K. (2002) Systemics 1.0. [CD-ROM]. Singapore:Singapore University Press.

O'Toole, M. (1994) The Language of Displayed Art. London: Leicester University Press.O'Toole, M. (1995) A systemic-functional semiotics of art. In P. H. Fries and M.

Gregory (eds), Discourse in Society: Systemic—Functional Perspectives: Meaning and Choicein Language: Studies for Michael Halliday. Norwood, NJ: Ablex, 159-179.

O'Toole, M. (1999) Engaging with Art. [CD-ROM]. Perth: Murdoch University.Palmer, R. B. (1994) Hollywood's Dark Cinema: The American Film Noir. New York:

Twayne Publishers.Place, J. (1998) Women in Film noir. In E. Anne Kaplan (ed.), Women in Film Noir (rev.

edn). London: British Film Institute, 47-68.Stables, K. (1998) The postmodern always rings twice: constructing the femme

fatale in 1990s cinema. In E. A. Kaplan (ed.), Woman in Film Noir (rev. edn).London: British Film Institute, 164-201.

Thibault, P. J. (2000) The multimodal transcription of a television advertisement:theory and practice. In A. P. Baldry (ed.), Multimodality and Multimediality in theDistance Learning Age. Campobasso, Italy: Palladino Editore, 311—385.


Towne, R. (1974) Chinatown (R. Polanski, Director; and R. Evans, Producer).Hollywood GA: Paramount Studio.

Tuska, J. (1984) Dark Cinema: American Film JVoir in Cultural Perspective. Westport, GN:Greenwood Press.

van Leeuwen, T. (1999) Speech, Music, Sound. London: Macmillan.Ventola, E., Charles, C. and Kaltenbacher, M. (eds) (forthcoming) Perspectives on

Multimodality. Amsterdam: John Benjamins.Voytilla, S. (1999) Myth and the Movies: Discovering the Mythic Structure of 50 Unforgettable

Films. CA: Michael Wiese Productions.

6 Multisemiotic mediation in hypertext

Arthur Kok Kum ChiewNational University of Singapore

Introduction

This paper is an attempt to understand how an institution and its objectivesbecome translated, transmitted and received through the hypertextmedium. The notion of hypertext is first clarified with the aim of abstract-ing methodological categories which may be used for a semiotic analysis.Following this, systemic functional models (Halliday, 1994; Kress and vanLeeuwen, 1996; O'Toole, 1994) are employed to examine the semioticchoices made within a selected webpage, the Singaporean Ministry of Edu-cation (MOE) site,1 in order to examine the meanings produced by thesechoices and the context circumscribing this choice-making and meaningproduction. The interaction of meanings across different semiotic instanti-ations also features in this analysis.

Genesis of hypertext

The precedence of verbal over written language in human groups is firmlyacknowledged in conventional histories of writing, with only certain culturesdeveloping a recording-writing system for reasons of trade, religion or polit-ics (Kress and van Leeuwen, 1996: 18-19). In Euro-American history, theadvent of print technology made recordable texts not only vastly replicablebut also more readily available compared to the past. In this sea of data,however, information retrieval posed a serious difficulty because textsremained in an unchangeable linear format.

Early theorists concerned with presenting and retrieving informationenvisaged a system for providing complete access to the 'endlessly expansiveworld of texts' (Tuman, 1992: 55). The term 'hypertext', coined by TedNelson in the 1960s, was used to refer to a form of electronic text where themode of publication was characterized by 'non-sequential writing'; that is,'text that branches and allows choices to the reader' in the form of 'a seriesof text chunks connected by links which offer the reader different pathways'through an interactive screen interface (Landow, 1997: 3). In the late 1960s,theory moved towards reality when the Advanced Research Projects Agency(AREA) of the Department of Defence in the United States of America set


up ARPnet, an inter-computer communication network which was designedto be impervious to communication disruptions in the event of a nuclearattack (Moore, 1994: 4). While it initially connected selected academicinstitutions, network technology soon expanded the use of hypertext. Soft-ware applications, such as web browsers, were made available to onlinecomputer users, and these combined with other software applications (forexample, word-processing software) so that hypertext could be edited,updated, copied and, in a word, 'acted' on. Juxtaposed to static and linearprint technology, hypertext became dynamic, alterable and multi-sequential.

Interpretations and applications of hypertext

Espen Aarseth (1997), appreciating the interactive co-partnering betweenthe reader and the creator during Internet surfing, describes hypertext as'ergodic, using a term appropriated from physics that derives from the Greekwords ergon and hodos, meaning "work" and "path"'. 'Ergodist' has beencoined to refer to the person who interacts with the hypertext in this way(Lim, 1998: 31). It is perhaps necessary to discuss what is meant by 'ergodic'so as to more fully investigate the notion of the 'ergodist'.

'Ergodicity' describes, first, the complexity of path predetermination and,second, how these paths can either be followed or bypassed, thereby creatingnew paths. The former implicates a 'creator' of the path, and the latter achoice-making individual who is faced with these paths. The 'ergodist' is thischoice-making individual who may follow predetermined paths suggestedby hypertext links which connect one webpage to another, or alternatively,may forge his or her own path. In moving through hypertext, a complextripartite relationship exists between the ergodist, the hypertext and thehypertext creator. As the next section will show, ergodist acquires its def-initional fullness at a particular abstraction of hypertext.

The notion of hypertext has been rethought in various fields of studyincluding deconstruction, structuralism, post-structuralism, reader-responsetheory, narratology, critical literacy (see Landow, 1997), and multiliteracy(Kress, 2003; Kress and van Leeuwen, 2001; Lemke, 1998; Unsworth,2001). A reactionary view of hypertext sees it as artificial, a threat to face-to-face or 'real' communication, and an usurper of older communicative tech-nologies such as the nostalgic pen(cil) and paper manuscript.2 On the otherhand, certain grandiose pro-hypertext statements claim hypertext to be anevolutionary superior that will replace linear writing; that better communi-cation will result simply because multiple interpretations and voices arelinked; and that hypertext will democratize society and education, evensurmounting artificial divisions between the disciplines. These and certainother hyperbolic construals of hypertext detract from an understanding ofthe nature of this new technology and what it can and cannot do for us. Itherefore propose a definition which opens up hypertext to further (multi)-semiotic investigation (see also Kress, 2003; Kress and van Leeuwen, 2001;Lemke, 1998; Unsworth, 2001).


Proposed working definition of hypertext

Consensus seems to place hypertext as a new technology or medium forcommunication which allows new dimensions of human interaction hith-erto not possible. Indeed, hypertext is a means of communication wheremultisemiosis as fact impinges upon the user. From these formulations, Ipostulate the following working definition:

Hypertext is a computer supported online telecommunication technology thatmakes possible the assembly, retrieval, display and manipulation of texts, whichare realizations of a single semiotic resource or a combination of semioticresources, some of which include visual, linguistic, phonic and music.

The crucial qualification 'makes possible' arises for two reasons: first, multi-semiotic texts can be assembled by technology other than the hypertext;second, a whole host of factors can curtail what hypertext affords; forexample, 'secure' websites that can only be accessed by certain knowledge-able people (whether one possesses the password or is an expert hacker),incompatible or missing software, lack of technical savoir-faire., and so on. Onanother note, my definition excludes CD-ROM programs for standalonecomputer workstations. These CD-ROMs, while possessing certain hyper-text features (such as connected scrollable pages and multimedia), are notrelated or potentially relatable to other webpages or software in a largerconnected network of workstations. This exclusion holds until a website iscreated for supporting the said CD-ROM program in a web-browser win-dow, in effect, making it relatable to other webpages. One is forced to admitthat technological innovation continues to problematize the notion ofhypertext.

Orders of abstraction of hypertext

With this working definition of hypertext in place, it is now possible toextract what I perceive to be different orders of abstraction with which onecan talk about hypertext. These orders of abstraction should not, however,be confused with ranks or levels which are posited for different semioticresources. Halliday (1994) proposed such constituent ranks for the linguisticsemiotic. Borrowing this notion of levels, O'Toole (1994) suggests rankscales for visual art, sculpture and architecture. Here the notion of rankorders and relates systems of meaning-making across the differentmetafunctions in what are essentially theoretical formulations of the'grammar' of different semiotic resources. As such, the ranks operate withinthe confines of the 'text' produced. These ranks become useful when oneseeks to uncover the choices made in instantiations of each of the semioticresources.

The orders of abstraction posited here for hypertext are methodologicalcategories construed to handle this to-date slippery technology. As we shall


soon see, these orders of abstraction are not necessarily related to each otherby constituency. Indeed, the orders of abstraction are different in nature tothe aforementioned semiotic ranks because hypertext is not a semioticresource, but a platform for the codeployment of different semioticresources. The orders of abstraction proposed for hypertext are ITEM, LEXIA,CLUSTER and WEB. As these terms require theorization, I start with the lowestorder of abstraction and develop these concepts to the highest or mostinclusive category of hypertext.

Item

An ITEM is any instantiation from any meaning-making system that is sup-portable by hypertext technology, and to date, these semiotic resourcesinclude the linguistic, visual, music and phonic. The question of whatinstantiation(s) count as an ITEM is necessarily preceded with a brief discus-sion of ranks (in italic font below) in semiotic systems.

A linguistic instantiation such as 'I could fly' is easily identified as aClause. In contrast, the instantiation 'Move!' is simultaneously a Clause, aVerbal Group and a Word. O'Toole (1994: 12) observes the same phenom-enon in certain paintings where a Work may simultaneously be an Episode,a Figure or simply a Member. Ostensibly, ranks within any one semioticsystem are not impermeable to each other. In any one semiotic, an ITEM maytherefore be a number of instantiations of different ranks of the one semi-otic combining together as a discernible whole. In multisemiotic texts, anITEM could be an instantiation of one semiotic resource, or a combination ofinstantiations of different ranks of different semiotic resources joiningtogether as a methodologically justifiable whole. In this light, ITEM encapsu-lates this permeability of the ranks within and across semiotic resources.

What are the semiotic choices that contribute to a sign or a complex ofsigns being designated as an ITEM? For either linguistic or visual semiosis,they are the choices made in the Textual or Compositional metafunctionrespectively. For a combination between the two resources, factors thatseparate one ITEM from another crucially rest on the choices made in theCompositional metafunction. These Compositional choices include thosefrom the system of Colour Cohesion, the system of Alignment and thesystem of Gestalt: Framing (see Table 6.2). This is not meant, however, toplay down the fact that choices made in the other metafunctions in bothsemiotic resources also contribute to the discreteness of a sign or complex ofsigns, but that the justification for ITEM rests primarily on choices made inthe Compositional metafunction with regards to the Textual organizationof the typographical/graphical instantiation of the linguistic/visual semi-otic choices.

As displayed in Plate 6.1, the order of ITEM could apply to a Word, aboxed-up Clause(s), an Element of a stylized gust of wind, an Episode of aman swatting a fly, the Work of an evening skyline serving as a backgroundgraphic, or even a complex of signs.

135

Plate 6.1 Examples of items

So far I have only concentrated on signs or complexes of signs that areeither linguistic or visual as these make up most of what appears on awebpage. However, hypertext makes available instantiations from othersemiotic resources as well. Can ITEM be extended to instantiations of thephonic semiotic resource? Perhaps non-linguistic phonic instantiationsbroadcast in hypertext may be designated as an ITEM. These may include asound clip such as the Microsoft Startup Window chime that is emittedwhen the Microsoft platform is launched on the computer. An ITEM mayalso be extended to melodic broadcasts, which again overlap between thephonic and music semiotic resources. Likewise, perhaps in certain cases ofhypertext where audio recordings of linguistic discourse are broadcast, theentire broadcast might be grouped as an ITEM. It is apparent that furtherwork in this direction is needed.

Lexia

The word lexia derives from Roland Barthes (1974: 13—14) and stands forthe scrollable webpage; that is, the 'text composed of blocks of texts' that anergodist sees on the computer screen (Landow, 1997: 3—4). ITEM, whichinclude hypertext links, become the constituents that make up a LEXIA. Inpractice, LEXIAS can be 'short' or 'long' depending on how many ITEMS areincluded and how they are organized. It is at this order of abstraction where(multi)semiotic realizations are organized in some meaningful way in rela-tion to others. 'Reality' is represented (multi)semiotically, and the ergodistengages with, and is placed in a particular relation to, what is displayed andthe producers of that display. The relation between LEXIA and ITEM is one ofcomposition where a LEXIA is made up of ITEMS. Instances of LEXIAS andITEMS are in turn realized from choices made in the metafunctional systemsfor different semiotic resources.

Cluster

CLUSTER refers to a number of connected LEXIAS due to associations createdvia hypertext links. These hypertext links are classified as 'LEXIA internal' as


they are located within the LEXIA itself and serve to 'call-up' another LEXIAshould the ergodist click on it. With hypertext links, one agency (institution,company, collective or individual) can link its many LEXIAS in such a way asto suggest (and so limit) the multidirectionality of traversing the LEXIAS thatmake up one CLUSTER. The notion of CLUSTER thus overlaps with the notionof a producer-created path, because it is the producers of particular LEXIASwho place hypertext links that in turn suggest or determine a pathway orpathways through the CLUSTER. A CLUSTER can appear discrete from othersby means such as strategic placing of'Back', 'Forward', 'Back to Homepage'buttons or even a sidebar with hypertext links to other LEXIAS within theCLUSTER.

A complication to this order of abstraction may be the fact that a num-ber of LEXIAS associated by one agency via hypertext links can join with oroverlap with others as a result of hypertext links put up by the same agencyor some other. This is not only a remote possibility, but an avenue exploitedby agencies who insert a hypertext link on their own LEXIA that links to alarger number of associated LEXIAS. Pushed to its logical extreme, thisnotion breaks down what is authoritatively the CLUSTER belonging to a par-ticular agency. For example, in December 1999, a hypertext link on theMOE homepage linked directly to a webpage belonging to the HousingDevelopment Board of Singapore (HDB), which was in turn linked with avast series of LEXIAS that the HDB produced. One asks where the MOECLUSTER ends and the HDB counterpart begins? This is precisely the prob-lem of designating CLUSTERS based on agency. The notion of CLUSTER isthus not concerned with agency perse., but associations formed via hypertextlinks. These links are finite, and a CLUSTER 'rounds off, or starts becoming amore discrete entity from other CLUSTERS with the termination of links.While the CLUSTER is constituted by LEXIAS based on internal hypertextlinks, these are temporal and changeable, thus making the associationsbetween LEXIAS transient and mutable. CLUSTER is as such Virtual' and anobservable disjunction occurs between this order and those of LEXIAand ITEM.

Web

WEB is the number of LEXIAS associable through hypertext links and otherfacilities internal and external to a LEXIA. Facilities that are LEXIA internal(but are not hypertext links) include search engines situated within a LEXIA,while LEXIA external facilities are those provided, for example, by the web-browser software. These appear on the web-browser window and includethe 'Forward', 'Back' and 'Home' buttons among other options. LEXIAexternal facilities also include the hardware, or the cable connectionsbetween computers. This notion of WEB thus includes LEXIAS potentiallyrelatable to each other by Local Area Networks (LANs), such as Ethernet,that join sets of machines within an institution or a part of one and alsoWide Area Networks (WANs) that join multiple organizations in widely


spread geographical locations. The contemporary terms 'Internet' and'World Wide Web' capture what is believed to be an increased global con-nectivity since LANs and WANs. WEB therefore characterizes the varyingdegrees of associations as well as the different means of forming associationsbetween LEXIAS and CLUSTERS.

The range of facilities both LEXIA internal and external make potentiallyall LEXIA accessible and traversable. Perhaps here is where hyperbolic state-ments about hypertext's infinitude arise. In reality, however, even if allLEXIAS that comprise a WEB were made freely accessible, they are still afinite number, and, furthermore, restrictions limit access to particular sites.For example, certain 'secure' websites such as private online email accountsare accessible only to the person with the password or technical expertise tobypass the password requirement. WEB as an order of abstraction is whatembraces all potential associations via devices that establish links originallyinternal or external to LEXIAS. Actual reading practice, therefore, is aninteraction between the two notions of CLUSTER and WEB, where an ergo-dist's route occurs either within or without the routes made by the producerof the websites.

Orders of hypertext for semiotic mediation and analysis

Much effort has gone into clarifying the notion of hypertext because theobjective of this paper is to give an account of how semiotic resources arecodeployed in hypertext. With the ordering of hypertext into differentabstractions, it becomes clear that it is at the categories of ITEM and LEXIAwhere multisemiosis, or the realization of different semiotic resources,occurs, and hence where multisemiosis as a fact impinges on the ergodist. Itis at these two abstractions of hypertext where multisemiotic analysis canmeaningfully operate. This is demonstrated in the following section whichfocuses on the analysis of the MOE homepage.

Context for the construction of the MOE homepage

Before the semiotic analysis proper, it is essential to include a brief con-sideration of webpages in relation to semiotic resources and the context ofsituation and culture. This relationship is represented diagrammatically inFigure 6.1.

Both processes of realization and instantiation imply a dialectic activationto the right and below, and construal to the left and above. For example,culture activates the use of semiotic resources while choices from the systemsof different semiotic resources construe culture. Likewise, culture activatessituation while situation construes culture. A certain complexity enters intothis relationship, however, when one appreciates that culture is not mono-lithic, that situations deriving from culture are not uniform and con-sequently LEXIAS are not entirely identical. Context, constituted by cultureand situation, thus needs to be appreciated as multidimensional.


Figure 6.1 Relation of culture, situation, semiotic resources and lexia (adaptedfrom HaUiday, 1991)

Nonetheless, a particularization of the aspects of a context is useful foruncovering the circumstances under which a webpage is produced. Forexemplification, I examine the MOE homepage 'frozen' at 7 January 2000,scaled down and reproduced in Plate 6.2. Note the analysis refers to theactual size of what is seen onscreen.

With the MOE homepage in view, one aspect of context is the socio-political climate constructed by the current ruling party in Singapore, thePeople's Action Party (PAP). Through the years, the PAP has selectivelyidentified and communicated to the local population concerns overSingapore's lack of natural resources, relative geographical smallness, het-erogeneous population and proximity to nations predominantly Malay-Islamic. With this 'crisis narrative', as some have called it, the governmentoffers economic survival among others as a solution-goal under which tounify and direct Singaporeans (Heng and Devan, 1995).

Unsurprisingly, the use of hypertext has been discursively predicated onthis larger concern of economic survival in the 'Information Age'. Forexample, in the Singaporean newspaper The Straits Times (9 February, 2000)an editorial entided 'Internet a driving force' claims that:

In Singapore, where the need to be a communications hub is, if anything, moreacute than it is in places less dependent on global economy, connectivity is not aslogan. It is a simple pointed imperative. Companies and employees must take

Plate 6.2 Lexia of the MOE homepage


seriously the Government's call for workers to upgrade their skills to find a placein the new knowledge economy.

As the educational arm of the PAP, the MOE works with such an end inmind. In a public release, in the section entitled 'Cornerstone of educationpolicy', the MOE reveals that one of its chief foci is 'the development ofhuman resources to meet Singapore's need for an educated and skilledworkforce' (Ministry of Education, Singapore 2000). Out of this contextconstrued by the PAP and, more specifically, the MOE, the homepageunder consideration is erected.

Another configuration of context, comprising the production norms forwebpages, forms a necessary second step to contextualize the MOEhomepage. LEXIAS can be constructed for a range of purposes. One suchpurpose is the display of information. Webpages that only serve this purposeemerge as 'content heavy'. Other webpages are used for administrativepurposes such as gathering feedback and so possess features whereby theergodist can 'enter' whatever he or she wishes. A particular type of webpageserves the function of welcoming and introducing the ergodist to a series oflinked webpages. Such a webpage is commonly referred to as the'homepage', since it is held to be the locus point to all the other linkedwebpages. Apart from welcoming and introducing the ergodist, homepagesmay also serve as an index of varying degrees by having visible hypertextlinks to the linked webpages.

The norms associated with a homepage provide an insight into oneaspect of the context that produces it. Most homepages have the genericlayout of masthead in the topmost position with various texts and hyper-text links beneath. This layout is generally adopted by commercial andinstitutional organizations perhaps because apart from welcoming andintroducing, it foregrounds the corporate identity behind the website. Withthe identity of the 'seller' disclosed, the ergodist as consumer may in 'goodfaith' accept the material goods, services or information proffered by thewebsite. Nonetheless, some websites do play with the rigid style of presen-tation or depart from it altogether to increase its engagement with theergodist. This is done either by experimenting with the different semioticresources in the hypertext environment or communicating in novelways through uniquely hypertext facilities to create a greater sense ofdynamism and unpredictability. For example, homepages may flout conven-tion by duplicating and relocating the masthead vertically at the sides of thewebpage, and such columns of words may flash alternative colourssequentially.

Whatever the case may be, the purposes served by a homepage are cir-cumscribed by situational and cultural demands of context. Context thusstands as a necessary preface to any semiotic analysis. With this in mind, onemay enter into an exploration of the semiotic choices and hypertext facilitiesemployed by the MOE homepage.


Semiotic analysis of the MOE homepage

The semiotic account of the MOE homepage tackles many intersectingquestions: Why are certain semiotic choices made? How do these semioticchoices work together to give meaning? What meanings are conveyed andfor what purpose? In effect, the following analysis works towards the centralconcern of this paper: to explicate the complex question of why the web-page comes to be written in the way it is. Such an analysis in turn necessi-tates an account of the interaction of meanings between instantiationsof different semiotic resources, and this is explored in the next section of thispaper.

The semiotic examination of the meanings put forth by the MOEhomepage is systematized first at the order of LEXIA followed by the order ofITEM. Such an analysis relies on the ranked functional systems for linguisticand visual semiotic resources posited respectively by Halliday (1994) andO'Toole (1994). Tables 6.1 and 6.2 provide a sketch of these ranked func-tional systems for both the linguistic and the visual semiotic.

Because Tables 6.1 and 6.2 are essentially 'unfinished' maps, the systemsare to a certain degree open-ended, implying that a greater level of ana-lytical delicacy is always possible. Out of these posited systems, choices aresimultaneously made to produce particular instantiations. Additionally, sys-tems across ranks may also work together for any one instantiation. Tocapture this complexity, semiotic choices discussed in this analysis are pre-sented in terms of'selection expressions'. These expressions use the systemsavailable in Tables 6.1 and 6.2 as 'entry points', and these are worked towhatever level of delicacy is needed (see Hasan, 1996 for a detailed presen-tation of selection expression and entry points). All references to these entrypoints in Tables 6.1 and 6.2 are henceforth in plain text with the initial lettercapitalized (for example Focus: Perspective), while those in italics representmy more delicate contribution (for example, Gestalt: Framing: Bordering).Pertaining to these selection expressions, there are several things to note: theleft-most element is the entry point for the discussion; colons precede a moredelicate choice in relation to the preceding element; and semi-colonsdistinguish elements of the same level of delicacy.

Analysis at order of lexia

Modal and compositional choices

Because Modal choices are very intimately related to Compositional choices,a discussion of the former cannot avoid invoking the latter. A quick surveyof the MOE homepage gives an impression of five sections represented inPlate 6.3. These divisions are strongly suggested by the Modal choices fromthe system of Scale to Whole, the system of Contrast and Conflict: Colour;Scale; Light; Line and the System of Relative Prominence. These choicesmay be more usefully explicated by complementary Compositional choices.


Table 6.1 Halliday's functional systems for language (adapted1999)

Function/Rank

Clause

VerbalGroup

NominalGroup

Adverbial(incl.Prepositional)Group

Word (incl.Lexical item)

IdeationalExperiential Logical

TransitivityTypes OfProcess,ParticipantsandCircumstances(Identity Glauses)(Things, Factsand Reports)

Tense(Verb Classes)

ConditionAdditionReportPolarity

CatenationSecondaryTense

Modification ClassificationEpithet Sub-Function ModificationEnumeration(Noun Classes)(Adjective Classes)

'MinorProcesses'PrepositionalRelations(Classes OfCircumstantialAdjunct)

Lexical'Content'(TaxonomicOrganizationOf Vocabulary)

NarrowingSub-Modification

CompoundingDerivation

Interpersonal

MoodTypes OfSpeechFunctionModality(The Wh-Function)

Person('Marked'Options)

AttitudeAttitudinalModifiersIntensifiers

Comment(Classes OfCommentsAdjunct)

Lexical'Register'(ExpressiveWords)(StylisticOrganizationOf Vocabulary)

from O'Toole,

Textual

ThemeTypes OfMessage(Identity AsText Relation)(IdentificationPredicationReferenceSubstitution)

Voice('Contrastive'Options)

DeixisDeterminers'Phoric'Elements(Qualifiers)(DefiniteArticles)

Conjunction(Classes OfDiscourseAdjunct)

Collocation(CollocationalOrganizationOf Vocabulary)

These include Relative Position In Gestalt, In Episode And To Each Other:Proximate, Gestalt: Framing: Bordering (for example, those borders under themasthead and below 'Corporate Information') and groupings of recogniz-ably similar instantiations under headings in Stylization: Font: Font Style:


Table 6.2 Ranked functional systems for the visual semiotic (adapted fromO'Toole, 1999)

FunctionUnit

Work

Representational

Actions, EventsAgents, Patients,GoalsScenes, Settings,FeaturesPortrayals, SittersNarrative ThemesInterplay OfEpisodes

Modal

Focus: PerspectiveClarityLightColourScaleVolume

Gaze: 'Eyework''Paths''Rhythms'Intermediaries

Frame'Weight'Modality: Fantasy

IronyAuthenticity

SymbolismOmissionIntertextuality

Compositional

Gestalt: FramingHorizontalsVerticalsDiagonals

ProportionLineRhythmGeometric FormsColour Cohesion'Theme'

Episode Groups AndSub-Actions,Scenes, PortrayalsSide Sequence)Interplay OfActions

Scale To WholeCentrality To WholeRelative ProminenceInterplay OfModalities

Relative Position InGestalt And ToEach Other

AlignmentCoherenceInterplay Of Forms

Figure CharacterActStanceGesture

Object CharacterizationPosition Relation To Viewer

GazeGestureContrast and Conflict:

ColourScaleLightLine

Relative Position InGestalt, InEpisode And ToEach Other

Parallelism andOppositionSubframing

Member Basic PhysicalForms:

Parts Of BodyObjectNatural FormsComponents

StylizationAttenuationChiaroscuroSynecdocheIrony

Cohesion: ReferenceParallelContrastRhythm

Plate 6.3 Sections of the MOE homepage


Bold (such as the ITEMS under 'Web Sites of Interest' and 'Corporate Infor-mation'). Because these sections are rectilinear and stacked vertically, theGestalt is one that positively suggests stability or negatively an absence ofdynamism (O'Toole, 1994).

The organization of linguistic and visual instantiation of this webpagereflects a certain trend. If one were to consider the linguistic texts on thewebpage, the selection is Relative Position In Gestalt: Formatting: Left Justified.,meaning strings of words are aligned from the same vertical point ofdeparture starting from the left. This left justification relates to the readingpractice associated with English texts which is left to right to the row below.Additionally, each of the hypertext links under 'Highlights' and 'CorporateInformation' has a graphic bullet that indicates the start of a 'new point' aswell as a distinct hypertext link. These bullets therefore function to draw theeye to the right and to signal the intended discreteness of linguistic instanti-ations. In much the same way, the MOE Shield at the top left corner of thewebpage calls attention to itself while bulleting the 'main point' of thehomepage: the Ministry of Education, Singapore.

More so than in other multisemiotic texts, the 'putting together' or con-struction of a hypertext involves a heightened awareness of bringing separ-ate elements together in spatial relation to each other. This construction isfundamentally achieved through Hypertext Mark-Up Language (HTML)that is used to 'write' computer commands which execute the webpage asseen on-screen. A source code thus details a particular webpage's HTMLconsisting of commands enclosed in pointed brackets such as '<P align-=centre>' to more complex ones such as <TABLE border=0 cellPadding= 5cellSpacing=5 width='101 per cent'>'.

In addition, sequentiality in the source code usually translates to theactual webpage displayed, as evinced by a simple comparison between thegiven source code and the MOE homepage. The HTML of the source codethus implicates a deliberate writer who is conscious of the spatial ordering oftexts as they appear on a webpage.

Representational choices

The above choices not only underscore the MOE as most salient (and this ismatter of course since it is the MOE homepage) but they also work in tandemwith Representational choices to construe the MOE's institutional 'face'.Contextualized with other homepages, the MOE homepage does 'reassure'with its 'generic' layout of masthead at the top with various texts and hyper-text links beneath it. As mentioned in the discussion on context, this layout isadhered to through a choice in Portrayal to foreground the corporate agency,and this functions to increase credibility to the end of encouraging theergodist to 'buy' what is offered on-screen. In the case of the MOE, it isinformation on local education-related issues that is being 'sold'.

Nonetheless, there are websites that play with the rigid style of presenta-tion, or depart from it altogether, to create a greater sense of dynamism and


unpredictability and so increase its engagement with the ergodist. Ofcourse, a website may go the extreme and end up deterring the mystifiedergodist. Regardless, the MOE homepage evidently has not experimentedwith the semiotic resources nor with hypertext facilities. Consider how therigid left to right framework set against the stark-white background makesthe masthead appear as a letterhead. A sense of a printed document, that'we have it in black and white', thus emerges. Perhaps because thehomepage is contextualized as a government website in an area of suchnational preoccupation, in its adherence to the 'standard' layout ofhomepages the MOE site has chosen to foreground credibility and back-ground 'playfulness'. The MOE thus foregoes the creativity that differentsemiotic resources and hypertext facilities afford, making the websiterelatively 'conservative' compared with other webpages. How does such awebpage act on the reader and what assumptions are embedded in thisrepresentation of the MOE? Such questions are explored following themore detailed analysis at the order of ITEM.

Scrollability

Before finishing the analysis at the order of LEXIA, one particular hypertextfeature gives cause for further thought. Due to several factors, such as a non-maximized web-browser window or a small monitor display, a LEXIA mayonly be presented in part. One facility hypertext opens up is what I call'scrollability' which determines how the semiotic choices ultimately contactthe ergodist. A deliberately lengthy or wide webpage exploits scrollabilitywhile simultaneously marking it as a feature for the ergodist.

The feature of scrollability has two types: vertical and lateral. As thedefault display of webpages is always the topmost and leftmost portion first,this means that for small displays, the option to scroll laterally arises, inwhich case one must always start from the left. The more common case isthe vertical scrolling option, starting always from the top. Noting this defaulttop-left display, it is not surprising that webpage designers usually situatewhat they deem as more important in these 'guaranteed viewing areas'. Inthe light of scrollability, the preceding discussion needs re-examinationbecause even with the largest monitor display presently available and maxi-mization of the web-browser window, the MOE homepage is only fully'read' by scrolling downwards. The downward scrolling process is repro-duced in Plate 6.4.

The initial window rules out all those ITEMS under the heading 'web sitesof interest' and below, ensuring that the already prominent masthead is evenmore salient. Ostensibly, the convention of locating the most importantinformation (in this case the MOE masthead) at the top is a recognition ofthe default top-left display.

What is deemed most significant is situated at the said guaranteed viewingareas with the rest arranged in a descending sequence according to import.This overall arrangement has a significant contribution to how the ITEMS

Plate 6.4 Scrolling sequence of the MOE homepage


are read. The questions of how the MOE homepage acts on the reader andwhat assumptions are embedded in this representation of the MOE maynow be more fully explored in an analysis at the order of ITEM.

Analysis at order of item

Working through a reading path

The following brief looks at the webpage's ITEMS works through a 'readingpath', a notion which relies on the assumption that 'all forms of semiosis areread syntagmatically' against the patterned whole of the text (O'Halloran,1999: 322-324). Whenever a new LEXIA is displayed on-screen, therefore,some ITEM/S will arrest or compete for the attention of the ergodist. Thisstarting point through which the ergodist 'enters' the LEXIA is what MarioGarcia terms the focal point or the Central Visual impact (CVI) (in Bohle,1990: 36, cited by Wee, 1999: 21). The CVI is compatible with the notion ofan ITEM because, as we shall see, both can be accounted for by salientCompositional (and in some cases Modal) choices. From the CVI, the ergo-dist engages sequentially with other ITEMS of the LEXIA, in effect workingthrough an idealized 'reading path'. From the above discussion on the fea-ture of scrollability and the semiotic choices instantiated at the order ofLEXIA, I suggest a reading path labelled alphabetically in Plate 6.5.

Being 'bulleted' by the MOE Crest to the left and punctuated to the rightby a Y2K symbol, the masthead captures the initial attention of the ergodistat Step A for the reasons discussed above. Ostensibly, Portrayals of authorityand preparedness (as embodied by the crest and the 'Year 2000 compliant'symbol respectively) function to bolster the credibility of the website. Thereading practice of left to right to next line down brings one to the missionstatement 'Moulding the Future of Our Nation' immediately beneath themasthead. The border below the mission statement 'closes off Step A,which constitutes the CVI due to its superordinate position through Com-positional choices. The complex interaction between the masthead and themission statement and what this interaction means are detailed later.

Next, the ergodist enters Step B via 'Highlights' in Stylization: font: fontstyle: bold, which labels particular entities as noteworthy or of news value.The eye is then quickly drawn by the diagonal arc cutting through the logoof the 2nd AEMM Education Ministerial Meeting. This option in Gestalt:Diagonals has a certain dynamism when set against the darker colouredcircle. Presumably, since the '2nd AEMM' hypertext link is topmost andalongside an image, the linked site is deemed (at least by the MOE) to be ofutmost interest if not importance. Perhaps the 2nd AEMM is ranked higherbecause of its international scope, and this reflects a bid by the MOE, andby extension the PAP, to accredit itself with global relevance.

In Steps C and D, hypertext links realized as linguistic instantiations areread in the manner of left to right to the row below, due to reading conven-tions which are reinforced by the bullets. These hypertext links are arranged

Plate 6.5 Suggested reading path for the MOE homepage


in a two-column top to bottom order. With the notable exceptions of thehypertext links for 'HDB 40th Anniversary Web Site' and 'Teacher: Createa sense of wonder. Offer new perspectives', all the other hypertext linkscontain nominal groups. The lengthy nominalizations, such as 'New Uni-versity Admission System from 2003', recall headlines which are a notablefeature for newspaper articles. Perhaps hypertext links are in general con-structed to serve as headlines, promoting or giving the gist of their respect-ively linked pages.

In Steps C and D, the 'CL "B" Syllabus and Bonus Points Scheme', 'NewUniversity Admission System from 2003' and other issues that relate insome way to the centrality of ensuring that one obtains a 'good' educationare deemed newsworthy. As in this case, the packing together of hypertextlinks becomes an index for the ergodist to obtain a limited overview ofassociated webpages while, on the other hand, allowing the webpagedesigner to limit and foreground what are considered important extensionsof the webpage. An index comprised of hypertext links emerges as one waythrough which the webpage designers construct and underscore pathwaychoices for the ergodist to choose from.

Reading conventions bring the ergodist to Step E through another header'Web Sites of Interest'. Like the header before, and the final header 'Cor-porate Information' below, all ITEMS serving as headers in this webpage area result of Stylization: Font: Font Style: Bold., deriving a visual distinctivenessagainst other linguistic ITEMS. Because of the option in Gestalt: Diagonals,the tilted magnifying glass draws the eye into the hypertext link of 'Teacher:Create a sense of wonder. Offer new perspectives'. The eye then movesrightwards through to Step I, in this case not only because of readingconventions. Here the subtle reduction in height of the hypertext links, theincreasing colour brightness to the right, together with the diagonals in thetelescope and the magnifying glass of hypertext links 'Educational Televi-sion' and 'NE.WS' respectively draw the reader across the page to Step I. Acloser look at the hypertext links in Steps F, H and I shows they relatedirectly to what is spelled out by the MOE as the 'Cornerstone of EducationPolicy':

Information technology will be used widely as teaching and learning resources todevelop skills in communication and independent learning. National Education isalso taught to foster strong bonds among students and develop in them a sense ofresponsibility and commitment to family, community and country . . . capable ofcontributing towards Singapore's continued growth and prosperity.

(The Ministry of Education, Singapore, 2000, emphasis mine)

The hypertext links under 'Web Sites of Interest' can thus be seen as primar-ily expansions of what are institutional-governmental goals rather thanwhat may be of some interest to the ergodist.

While one may examine any one of these ITEMS for any length of time,the leftmost ITEM draws back the eye through the hypertext facility of


animation. For the hypertext link 'Teacher: Create a sense of wonder. Offernew perspectives' the image of the magnifying glass over a flower morphsinto a girl in mid-jump and then back again in perpetual recursion. As acomplex extension of the visual semiotic, animation necessitates furtherresearch which, however, is beyond the scope of this enterprise. Nonetheless,this conscious use of animation implies that the MOE made a decision toforeground this particular ITEM. This link's power of attraction is alsoenhanced by the possession of two of the only three Mood: Imperativeclauses which, in effect, level a 'direct' address at the ergodist. Both anima-tion and the rhetorical stance carried by this ITEM function to attract andsituate the ergodist as someone who can 'Create a sense of wonder' and'Offer new perspectives'. In the recent context of a nationwide campaign toenlarge the teaching workforce, the relative magnetism of this ITEMbecomes meaningful when one recognizes the fact that it serves as a link toanother webpage that encourages individuals to join the teaching profes-sion. Regardless, the general paucity of direct address may be due to anaspiration towards a formal, objective register which interacts with the'headline' convention of hypertext links as discussed above.

Steps J and K comprise an ordered bi-column arrangement of linguistichypertext links as in Steps C and D. Notably, under 'Corporate Informa-tion', a choice from Gestalt: Framing tabulates the hypertext links. Represen-tationally the rectilinear framing is a choice which projects stability andimmutability which is meant to accord with the corporate, definitive natureof the information. In this light, perhaps among other reasons, the linguistichypertext links in Steps C and D are not framed because they are by naturetime sensitive. For example, in January 2000 the hypertext link for the '2ndAEMM Education Ministerial Meeting' appeared while simultaneously thehypertext link to 'ThinkQuest-Singapore' was dropped.

In Step L, a border marks off the final portion of the webpage whichcontains the MOE's contact information and the 'Last Updated' date insmall fonts. This contact information is obligatory insofar as authenticatingthe website and providing an avenue for dialoguing with the MOE. How-ever, this information may perhaps be obscured because it is deemed rela-tively less newsworthy to the purposes of the website which acts as a mediaarm for the MOE. As evidenced by choices in Relative Prominence, contact-ing the MOE through any of the channels laid out in the contact informa-tion is downplayed as an option for the reader. What is instead deliberatelyhighlighted are the definitive statements found in the LEXIAS that the MOEhas already scripted for the MOE CLUSTER.

Backtracking the steps of the reading path, one finds an increasing sig-nificance associated with the ITEMS, with Step A housing the 'main subject'from which the rest of the webpage is understood: the MOE. As the tour atthe orders of LEXIA and ITEM shows, different texts on a webpage stand outdifferently due to various Modal, Compositional and Representationalchoices, pulling in the ergodist's gaze at every step of the reading path. Inturn, these choices as a whole reflect an image of the MOE as construed by

MULTIMODAL DISCOURSE ANALYSIS

the MOE homepage for the ergodist: as the authoritative voice on localeducation in service of the governmental goal of economic viability throughan educated workforce in the global marketplace.

So far, the semiotic choices explored at the orders of LEXIA and ITEMhave been rather brief owing to the limitations of space. Nonetheless, thissketch sets the backdrop against which a more delicate account of semioticactivity may be further explored and detailed. This activity is exemplifiedwith a focused examination of Step A in the MOE homepage.

An account of intersemiosis

Visual semiosis

The ITEM that stands out as the CVI in the MOE homepage is that com-plex of signs constituting the masthead, reproduced in Plate 6.6.

Plate 6.6 The MOE masthead

The Relative Prominence of the masthead at the top of the page is insharp relief to the other ITEMS of the webpage, and this arrests the atten-tion of the ergodist and serves as the focal point through which the webpageis entered. Additionally, options in Scale To Whole realize the masthead aslarger than any other ITEM, thus heightening its prominence. Furthermore,the Contrast and Conflict: Colour of the rectangle (which is brown) acts todistinguish the masthead from the white background of the webpage, simul-taneously adding salience to the white words it encloses. In a binary fashion,the white of the masthead words is now in contrast to the rest of thepredominantly dark-coloured linguistic text, even as the latter derive theirclarity from the white background of the webpage. This interlocking con-trast is precisely what throws the linguistic instantiations in sharp relief toone another.

The masthead reveals a more delicate option in Contrast and Conflict:Scale in terms of Font Size. The words 'Ministry of Education, Singapore'are Font Size:24, which is noticeably larger than the rest of the linguisticinstances which are Font Size: 12 or less. Within Stylization, Font serves as afurther specification. While 'Ministry of Education, Singapore' approxi-mate Font: Times New Roman, a large number of the other linguistic instancesare Font: Arial or some derivation through joint options with Font: Font Style:Bold or Font: Font Style: Italics. The uniformity displayed in the majority oflinguistic instances may be related to the default setting of Font: Arial inHTML. Any other font in the actual webpage display must be deliberately

152


chosen at the programming stage from a larger range of font styles. Anyother font style apart from 'Arial' thus implies a certain degree of delib-erateness. The masthead with its non-conventional font is thus a deliberatechoice to make it stand out from the rest.

The 'effect' of Modal choices is thus intimately tied to how the texts arearranged in meaningful relation to each other, that is, the compositionalchoices made. Gestalt: Framing is selected for the masthead via a border withequidistant light and dark intensities of colour, suggesting both variationand regularity. The strong rectangular frame at once mirrors the rectilinearframe of the web-browser window and is echoed by the grid-like patternwithin itself. Although the criss-crossing lines segment and may thus fracturethe surface of the masthead, the continuity of the words 'Ministry of Edu-cation, Singapore' over the surface evokes at the very least a closely piecedtogether surface without chinks. What remains is a Parallelism connectingthese geometric Forms which relate 'to the horizontal axis and the verticalaxis [. . .] [and] contribute to stability and harmony' (O'Toole, 1994: 23).

What is crucially conveyed by the Modal and Compositional choices arethe discreteness, centrality and stability of the masthead. Representationally,the masthead with its tiled texture and patterned border suggests amongmany things some flat human-worked surface. Two important observationscan be made here: first, the range of visual meanings are suggested by theactual choices instantiated; and second, while the meanings are, accordingto Barthes (1977: 38-39), 'polysemous', they are nonetheless finite (Kressand van Leeuwen, 1996: 16).

Visual-linguistic intersemiosis

Barthes's (1977) attempt to 'fix' visual meanings has been criticized becauseit makes visual meanings dependent on linguistic choices, a phenomenon hecalled 'anchorage'. Nonetheless, perhaps Barthes observes part of a morecomplex process. Analyzing the masthead once again, the uncertainty of thevisual meanings is clarified somewhat as it interacts with the linguistic mean-ings it frames. The meanings of the Nominal Group may be uncovered byexamining the choices realized in its structure as we may see in Table 6.3.

At the rank of Word, the Lexical 'Content' of the noun head 'Ministry'allows for these taxonomic meanings:

1 a a government department headed by a minister, b the building which itoccupies. 2 a (prec. by the) the vocation or profession of a religious minister, b theoffice of a religious minister, priest etc. c the period or tenure of this. 3 (prec. bythe) [a] the body of ministers of a government or [b] of a religion. 4 a period ofgovernment under one Prime Minister. 5 ministering, ministration.

(Reader's Digest Oxford Complete Wordfinder 1994: 969)

At the rank of Nominal Group, the options 2 a, b, c, 3[b] and 4 are excludedby the following choices in Modification: the premodifying definite article


Table 6.3 Nominal Group structure in the masthead

The Ministry

Premodification HeadDeterminerArticleDefinite

of Education,

Postmodification

PostposedPrepositionalPhrase

Singapore

Postmodification

Postposed NounPhrase

Head

'The' establishes 'Ministry of Education, Singapore' as unique, monolithicand authoritative just as the postmodifers 'of Education, Singapore' imbuethe function and sphere of influence of the ministry. The Nominal Group isthus specified to mean:

(a) From la: The Ministry of Education headed by the Minister of Education.(b) From Ib: The building of this government department.(c) From 3 [a]: The body of ministers of this government department.(d) From 5: The administration of this government department.

The polysemy of meanings proposed by the instantiated linguistic choices iscrucially finite. What is represented visually as discussed above now comesinto relation with this range of linguistic meanings, and seems to furthercontract the range of linguistic meanings to allow for only (b), that is,the physical building. I call this 'Specification 1'. However, 'Ministry ofEducation, Singapore' also comes into relation with yet another linguisticinstantiation.

Moulding the future of our nation

As mentioned in the above discussion at the order of LEXIA, this relation-ship between these two discrete linguistic ITEMS is encouraged by the Com-positional choices Relative Position To Each Other: Proximate (instantiated astheir top-bottom proximity) and Gestalt: Framing: Bordering (instantiated asthe dotted line 'sectioning out' these two linguistic ITEMS from others). Inaddition, because the mission statement is non-finite, it can be thought of asa dependent Clause, which calls into question what it is dependent on.Causal relationships may also be implicit with dependent Clauses, and in thiscase, one may ask who or what agent is 'moulding the future of our nation'.As a result of these questions and the Compositional choices mentionedabove, the two linguistic ITEMS come into the following possible relation inTable 6.4.

The mission statement may thus be perceived as entering into Experien-tial relations with the masthead. The material process 'Moulding' stipulates


Table 6.4 Experiential relations between the masthead and the missionstatement

Ministry of Education, Moulding the Future of Our NationSingapore

Participant Process ParticipantActor Material Goal

a sentient Participant Actor. This is complemented with the option Font: FontStyle: Italics which also imbues the mission statement with a sense of dyna-mism, implicating an animate Actor. These options act to specify thelinguistic meaning of the masthead as (a) and (c) (see above). I call this'Specification 2'. As can be observed, a disjunction arises between Specifica-tions 1 and 2. The MOE as imaged by Specification 1 is solid, concrete,immovable and non-living. In contradistinction, Specification 2 suggests theMOE as the animate agent shaping Singapore's future. In this sustainedambiguity, the MOE is (re)presented as a human agency who is at the sametime 'faceless', impenetrable and incontestable. This depiction of the MOEderives perhaps from the premises of its uncontestable authority withrespect to educational matters and its existence as an arm of the PAP.

Abstraction ofintersemiosis

This discussion has been concerned with the way meanings across instanti-ations of various semiotic resources interact with one another to give a newmeaning or set of meanings. This complex interaction and production ofmeanings between instantiations of different semiotic resources is called'intersemiosis'. Though the prior analysis of Step A is sequenced as Specifi-cation 1 followed by 2 in keeping with the suggested reading path,intersemiosis does not in fact depend on any one sequence, but upon themeanings first conveyed by each instantiation. In other words, for multi-semiotic texts, there is no binding unidirectionality or sequentiality formeaning interaction. Rather, one instantiation comes into relation withanother, and each simultaneously specifies the other.

Intersemiosis as discussed so far has been circumscribed by Compositionalchoices such as Gestalt: Framing and Relative Position: Proximate that relateinstantiations that are spatially 'grouped'. A more complete notion ofintersemiosis recognizes that choices from the Modal and Representationalsystems can also bring instantiations that are spatially distant or ungroupedinto significant relations for the interaction and production of meanings.However, these non-Compositional factors for intersemiosis can only bepursued outside the confines of this paper.

An abstraction of the stages of visual-linguistic intersemiosis may beoffered at this point as Relation, Intersection and Manifestation (collectivelyRIM):


(1) Relation: Compositional, Modal and Representational meaning-makingchoices delimit what instantiations are 'connected'. It is in the context ofthis connectedness between instantiations of different semiotic resourceswhere intersemiosis occurs.

(2) Intersection: the range of meanings suggested by an instantiation of aparticular semiotic comes into some relation with another instantiationof the same or different semiotic resource. Between instantiations, mean-ings across instantiations that are similar underscore each other to pro-duce one focused meaning, or a specification. Conversely, meanings thatare not similar may be either backgrounded or foregrounded.

(3) Manifestation: specifications across different semiotic instantiations mayeither materialize in a single highly determined, focused meaning or anumber of focused meanings. In the latter case, a sustained polysemyresults. Ambiguity results when the polysemous meanings are divergent.

Therefore, contra Barthes (1977), it is not only linguistic meaning thatanchors visual meaning, but the reverse as well. Questions of how similar ordivergent meanings may be determined aside, the above approach uncoversto some degree the complexity of intersemiosis. However, it becomes clearthat further work in this area is needed. Nonetheless, the abstract stages ofRelation, Intersection and Manifestation (RIM) may provide a way todescribe the process of encircling the pool of meanings occurring inmultisemiosis.

Conclusion

This undertaking has been an exercise in increasing specificity. That is,against an expansive range of discourse on hypertext, four abstract ordersof hypertext are posited, out of which the two lower orders of LEXIA andITEM are identified as sites for semiosis. At these lower orders of abstraction,a multisemiotic analysis was applied to the MOE homepage to uncover themeaning-making choices which construe the MOE. A further particulariza-tion occurs when intersemiosis is demonstrated at the level of delicacy oftwo ITEMS. Finally, this exploratory attempt culminated in an abstractionof the process of intersemiosis, Relation, Intersection, Manifestation(RIM) which approaches the problem of how to illuminate this complexphenomenon.

The issue of whether non-linguistic semiotic resources are systemic raisesthe question of the validity of extending the notion of the systemicmetafunctions beyond language. The contention that there may not exist astratum of 'grammar' for a non-linguistic semiotic resource and that, even ifthere is, this stratum is of a comparable nature to that of language becomesan issue. These theoretical questions remain still very much questions inthemselves and there is no reason to date to reject the notion that non-linguistic semiotic resources are systemic and tri-metafunctional. This isnot to say that the metafunctional systems between semiotic resources are


identical. That is patently untrue for the simple reason that different semi-otic resources have different ways of meaning, and so have in themselvesdifferent meaning-making systems. The systems proposed for non-linguisticsemiotic resources are markedly different from the linguistic. One crucialquestion may be whether non-linguistic semiotic resources serve non-socialfunctions. The notion that semiosis is necessarily social seems to secure thenotion of the three metafunctions (see Kok, 2001).

While exploring the systemic choices in the MOE homepage, my analysishas worked with a suggested reading path. This does not, however, rule outthe fact that an ergodist can focus initially on an ITEM other than the GVI,or in a similar fashion, can work through a different sequence of engagingwith the ITEMS on a LEXIA depending upon the immediate contextualfactors such as the number of times the website has been viewed. Further-more, what is immediately demanding of attention for one particular cul-ture may not be so for another, although acculturation across cultures isbecoming more frequent with the spread of mass media, of which hyper-text is a part. Further to this, it appears that various meaning-makingchoices and facilities in hypertext, as demonstrated, function to secure cer-tain sites of immediate visual engagement so that a GVI becomes visuallyprominent.

This enterprise has been unwilling to divorce hypertext from contextualuse because as a means of communication, hypertext only acquires itsrichness and definition from its use in the social realm. The functions ofhypertext are not wholly determined either by technology or society, butby technology used in society. As future innovations in communicativetechnology surface, new ways of meaning-making will be introduced.What has been suggested in the course of this undertaking are some ofthe new systems of meaning-making enabled by hypertext. However, fur-ther work is needed to account for the many other systems opened up inthis new platform. Nonetheless, the value of this work lies in its potentialto explicate the process through which semiotic choices are made, howthey are made, for what purposes and to what effect. It is hoped that thishas provided some answers to enquiries concerning the shifting ways ofcommunication and works towards a fuller disclosure of multisemioticactivity.

Notes

1 Due to publishing constraints, the MOE homepage could not be reproduced incolour. As colour is an important resource for meaning, these constraints some-what compromise the reader's interpretation of the webpage and the analysispresented here. However, every effort has been made to overcome this deficiency.

2 Although it seems counterintuitive to say that means of writing prior to theprinting press or the typewriter are technologies, 'the papyrus roll and the vel-lum manuscript also exemplify technologies of writing . . . [as] . . . both requireddevices: the reed pen and papyrus in ancient Egypt, and the quill and parchmentin the Middle Ages' (Snyder, 1997: 1).

Acknowledgements

Plates 6.2, 6.3, 6.4, 6.5 and 6.6 are reproduced by courtesy of the Ministryof Education (MOE), Singapore. The screenshots of the MOE homepagewere captured on 7 January 2000.

References

Aarseth, E. J. (1997) Cybertext: Perspectives on Ergodic Literature. Baltimore: JohnHopkins University Press. Available from: http://www.hf.uib.no/cybertext/Ergodic.html.

Barthes, R. (1974) S/£(R. Miller, trans.) New York: Hill and Wang.Barthes, R. (1977) Rhetoric of the image. In R. Barthes (S. Heath, ed. and trans.),

Image—Music—Text. London: Fontana, 32—51.Bohle, R. (1990) Publication Design for Editors. New Jersey: Prentice-Hall.Halliday, M. A. K. (1991) The notion of context in language education. In

T. Le and M. McCausland (eds), Language Education: Interaction and Development:Proceedings of the International Conference, Ho Chin Min City, Vietnam 30 March-1 April1991.


Hasan, R. (1996) Ways of Saying: Ways of Meaning Selected Papers of Ruqaiya Hasan.London: Gassell.

Heng, G. and Devan, J. (1995) State Fatherhood: the politics of nationalism, sexual-ity and race in Singapore. In A. Ong and M. G. Peletz (eds), Bewitching Women,Pious Men: Gender and Body Politics in Southeast Asia. Berkeley: University of CaliforniaPress, 195-215.

Kok, K. G. A. (2001) What is material about hypertext? Unpublished masters thesis.National University of Singapore.

Kress, G. (2003) Literacy in the New Media Age. London: Routledge.Kress, G. and van Leeuwen, T. (1996) Reading Images: The Grammar of Visual Design.

London: Routledge.Kress, G. and van Leeuwen, T. (2001) Multimodal Discourse: The Modes and Media of

Contemporary Communication Discourse. London: Arnold.Landow, G. P. (1997) Hypertext 2.0: The Convergence of Contemporary Critical Theory and

Technology. Baltimore: John Hopkins University Press.Lemke, J. L. (1998) Metamedia literacy: transforming meanings and media. In

D. Reinking, L. Labbo, M. McKenna and R. Kiefer (eds), Handbook of Literacy andTechnology: Transformations in a Post-Typographic World. Hillsdale, NJ: Erlbaum,283-301.

Lim, B. L. L. (1998) Hypertext fiction: a narrative analysis. Unpublished honoursthesis. National University of Singapore.

Ministry of Education, Singapore. (2000) Education in Singapore; available fromhttp://wwwl.moe.edu.sg/educatio.htm.

Moore, M. (1994) Introducing the internet. In Wired Magazine: The Internet Unleashed.Indianapolis: Sams Publishing, 4—19.

O'Halloran, K. L. (1999) Interdependence, interaction and metaphor in multi-semiotic texts. Social Semiotics 9(3): 317—354.

O'Toole, M. (1994) The Language of Displayed Art. London: Leicester UniversityPress.


http://www.hf.uib.no/cybertext/Ergodic.html

http://www.hf.uib.no/cybertext/Ergodic.html

http://www1.moe.edu.sg/educatio.htm


O'Toole, M. (1999) Functions and Systems in Verbal and Visual Texts. Paper presented atthe 26th International Systemic—Functional Congress. Regional LanguageCentre (RELC), Singapore, 26-30 July 1999.

Reader's Digest Oxford Complete Wordfinder, The (1994) London: The Reader's DigestAssociation Limited.

Snyder, I. (1997) Hypertext: The Electronic Labyrinth. New York: New York UniversityPress.

Straits Times, The (9 February 2000) Internet a driving force.Tuman, M. C. (1992) WordPerfect: Literacy in the Computer Age. London: Falrner Press.Unsworth, L. (2001) Teaching Multiliteracies across the Curriculum: Changing Contexts of

Text and Image in Classroom Practice. Buckingham, UK: Open University Press.Wee, C. K. A. (1999) Multi-semiotic analysis of advertisements. Unpublished hon-

ours thesis. National University of Singapore.

Part IIIPrint media

7 The construal of Ideational meaning in printadvertisements

Cheong Tin YuenNational University of Singapore

Introduction

The investigation of the intricacies, complexities and nuances of multi-semiotic texts has been the Focus of recent research. This arises from theobservation that 'language, and typological modes of semiosis generally,have evolved to work in partnership with other, often more topologicallygrounded, semiotic systems' (Lemke, 1998: 111). O'Toole (1994), Kress andvan Leeuwen (1996, 2001), Lemke (1998), Wee (1999), O'Halloran (1999)and Baldry (2000) have made significant strides within this area of multi-semiotic text analyses from a systemic-functional perspective.

This paper aims to contribute to the development of a theoretical frame-work and vocabulary for the articulation of meaning in multi-semiotic textsas research in this realm has not been as extensive as the examination ofpurely linguistic texts. To limit text analyses to only the linguistic aspect anddisregard the non-linguistic features such as graphs and diagrams is tanta-mount to annihilating the efflorescence of meaning that can emerge from amulti-semiotic analysis. As aptly stated by Wee (1999: vi):

Compared to text with a single semiotic code, the meaning potential of multi-semiotic texts is greatly expanded. Hence, meaning creation becomes an inter-active, dynamic and symbiotic process.

Research into multi-semiotic texts is indeed underrepresented, which is iron-ical as 'computer technologies make multimedia genres more convenientand accessible for all purposes, [thus] it will become increasingly importantto understand how the resources of different semiotic systems have been andcan be combined' (Lemke, 1998: 111). In this information age, it is indeed ararity for texts not to be illustrated, and this further signals the need toinvigorate and fortify research in this area.

Gheong (1999) proposes a working systemic-functional model for mean-ing-making in print advertisements through proposing lexicogrammaticalstrategies for Ideational, Textual and Interpersonal meaning. Constraints ofspace here allow for only a discussion of the construal of Ideational meaningin multi-semiotic texts. In this paper the generic structure potential of


advertisements is proposed and illustrated through the examination offive advertisements. In the following sections I discuss five strategies forconstruing Ideational meaning: the Bidirectional Investment of meaning,Contextual Propensity, Interpretative Space, Semantic Effervescence andVisual Metaphor. Hasan's (1996) Generic Structure Potential for advertise-ments and Kress and van Leeuwen's (1996) concept of Given and Neware critiqued, together with a re-examination of Barthes's (1977) notions of'anchorage' and readerly and writerly texts.

Generic structure potential of a print advertisement

Hasan (1996: 41-42) proposes 'CaptureAFocusAJustification' as the genericstructure for an advertisement. Hasan (1996: 41) aims to encapsulate themulti-semiotic nature of advertisements, with the Capture functioning:

to attract attention . . . [and] realized in the written mode through the manage-ment of the visual layout, the typeface patterns and/or the presence of pictures.

According to Hasan (1996: 41), the Focus csingle[s] out that which is beingadvertised'. However, while stating that the Focus can be visually realized,Hasan (1996) does not clarify whether the Focus has a linguistic realization aswell. Hasan (1996) also establishes the presence of a visual aspect to theJustification, but in a similar manner does not include the component to givea 'detailed account of other elements of structure for an advertisement'(Hasan, 1996: 42). Suffice to say that Hasan's (1996) generic structure Cap-tures to some extent the multi-semiotic nature of advertisements.

Following Hasan's proposal, there is a need to provide a more detailedaccount of generic structure for advertisements. Hasan's (1996) model doesnot make explicit the complexities involved in the interaction between visualimages and linguistic text in advertisements. It is the aim of this paper toprovide a model that best Captures the multi-semiotic interaction betweenvisual images and linguistic text in print advertisements.

Based on this limited study of print advertisements, the Generic StructurePotential or GSP which ' [expresses] the total range of optional and obliga-tory elements' (Halliday and Hasan, 1985: 64) for advertisements may beCaptured as:

LeadA (Display) AEmblemA (Announcement)A (Enhancer)A (Tag)A

(Call-and-Visit Information)

Table 7.1 details the generic structure of a print advertisement. In thisframework the various visual and linguistic components in an advertisementare made explicit, together with the interaction between these semioticresources which creates differing levels of Ideational, Interpersonal andCompositional/Textual meaning.

PRINT MEDIA 165

Table 7.1 Proposed generic structure of print advertisements

Visual components

Linguistic components

Lead: Locus of Attention (LoA), Complement tothe Locus of Attention (Gomp.LoA)Display. Explicit, Implicit, Congruent,Incongruent (metaphorical)EmblemAnnouncement Primary, SecondaryEnhancerEmblemTagCall-and-visit information

Interaction to create Interpersonal, Ideational and Compositional/Textual meanings

Five advertisements are analysed in this paper: the Golf, the Epson, theMl, the Beetle and the Guess? advertisements, which are displayed inPlates 7.1-7.5. I discuss in the next section why the Lead and the Emblemare designated obligatory elements while the others are optional.

The Lead

The discussion that follows details the characteristics and function/s of thevarious components that constitute the Generic Structure Potential of aprint advertisement. I will begin with the Lead.

The Lead is thus termed as it is Interpersonally most Salient (Kress andvan Leeuwen, 1996) through choices in size, position and/or colour. TheLead is illustrated in Plate 7.1. On its own, the Lead has a wide spectrum interms of meaning potential, that is, many possible meanings emanate fromthe Lead. Interpreted independently of the Announcement, Enhancer, Dis-play and Emblem, the Lead is figuratively an efflorescence of meaning. Forexample, the sensual looking female who is the Lead in the Golf advertise-ment (Plate 7.1) could be calling to attention the new millennium look or shecould be an ambassador for women's rights. Therefore, on its own, the Leadhas a bounty or a kaleidoscope of possible meanings.

As I explain below, the Lead consists of the Locus of Attention (LoA) andComplements to the Locus of Attention (Comp.LoA). There is an elementin the Lead that by its very Salience, be it an unusual quality that challengesreality or outstanding size, colour and so forth, arrests the attention of theviewers. In Plate 7.2 depicting the Epson advertisement, it is the splash ofthe water outside the boundaries of the photograph. This attention-arresting element is termed the 'Locus of Attention' (LoA). The LoAembeds the central idea of the advertisement, that Epson produces lifelikequality prints. The three-fold functions of the LoA include Interpersonallyattracting attention, and Ideationally construing reality in a way intended bythe advertisers, where the viewer's perception of reality is manipulated.Textually, it is a springboard for further development of the central idea, for

166

Plate 7.1 Generic structure of the Golf advertisement

example, that Epson produces lifelike quality prints in the linguistic text thatfollows. 'The text serves to elaborate' the visuals (Kress and van Leeuwen,1996: 194). But by what specific strategies/systems, we are left uninformed.The following discussion serves to explain this.

MULTIMODAL DISCOURSE ANALYSIS

PRINT MEDIA 167

Plate 7.2 Generic structure of the Epson advertisement

Visually, the LoA encapsulates the central idea that Epson produces life-like prints. This central idea is reiterated in the linguistic text. That is, thereis a linguistic equivalence (be it in the form of sentences or particular lexis)that coheres ideationally with this central idea conveyed in the LoA. Idea-tionally, the following linguistic items, including clauses and nominal groups,encapsulate tightly and parallel the idea embedded in the LoA, that is,Epson produces lifelike prints:


(a) EPSON STYLUS PHOTO EX - crystal-clear, photographic qualityprinting

(b) //Six specially formulated colour inks deliver richer, more lifelike images//(c) //while EPSON PhotoEnhance provides realistic colour balance every

time//(d) //The EPSON Stylus Photo EX can transform your photography//(e) EPSON Stylus. The most advanced inkjets.

If the above linguistic items had occurred in isolation without theaccompaniment of the LoA, they would be mere statements weakened oftheir persuasive force to manipulate perception in a way intended by theadvertiser, thus diminishing the influence over viewers to purchase theproduct. However, with the LoA conveying visually the idea that Epsonproduces lifelike prints, the meaning potential in linguistic items (a)-(e) issignificandy enhanced. Interpersonally, the LoA provides the context inwhich linguistic items (a)—(e) are endowed with greater persuasive force toinfluence viewers to purchase the product. Meaning-making of (a)-(e)from the Ideational perspective is enhanced, as the LoA serves textuallyas a reference point for readers to make sense of what exactly is meantby 'crystal-dear photographic printing', 'lifelike images', 'realistic colour balance','Epson . . . can transform your photography' and 'EPSON STYLUS. The mostadvanced inkjets'.

When the rankshifted clause 'to make a bigger splash with your images' in theEnhancer (see Plate 7.2) is read within the context of the LoA, it is identifiedas a pun. There is an interplay between linguistic item and the visual imagethat enhances the meaning potential of the rankshifted clause as well as theLoA. Without the LoA, there would be no such interplay of meaning, thusthe rankshifted clause 'to make a bigger splash with your images' would notbe interpreted as a pun, reducing the overall affective appeal of the adver-tisement. Extending Wee's (1999) concept of symbiosis, the LoA and thelinguistic text act on each other, mutually reinforcing and enhancing themeaning potential of the Lead.

Particular facets of the meaning potential of the Ideational meaning ofthe LoA can be articulated linguistically, that is, the Ideational meaning ofthe visual code can be translated into the linguistic code. Items (a)—(e) aboveare an articulation and representation, in linguistic form, of the meaningsembedded in the LoA. Conversely, it can be stated that the Ideational mean-ings in Items (a)-(e) are loaded into the LoA. The LoA is a visual compres-sion of the linguistic meaning in (a)-(e). The LoA can thus be interpreted asa Visual Metaphor as explained below.

Bohle (1990: 36) mentions Garcia's centre of visual impact (CVI), 'wherethe reader enters the page . . . without the CVI, a page is a mass confusionof elements competing for attention'. Wee (1999) further states that the CVI'becomes the entry point for the reading path of the multi-semiotic text. It isthe Theme of the entire text' (Wee, 1999: 21). Though paraUeling the CVFsfunction in engaging the viewer Modally, the proposed LoA functions

PRINT MEDIA 169

beyond a mere Interpersonal engaging of viewers' attention. It is not limitedto just being the theme, 'the point of departure of the message' (Halliday,1994: 37). Functioning as a Visual Metaphor, the LoA ideationally elucidatesand enhances the Ideational meaning potential of the linguistic text in theadvertisement.

The Complements to the LoA (Comp.LoA) refer to components in theLead which are comparatively less Salient than the LoA. They functionallyenhance the Interpersonal and Ideational Salience of the LoA. In otherwords, the Comp.LoA plays a subordinate role, to channel and Focusviewers' attention on particular aspects of the LoA. In the following discus-sion of the Ml advertisement, accompanied by Plate 7.3 depicting theGeneric Structure of the Ml advertisement, I illustrate how the interactionbetween the Comp.LoA and the LoA brings out the Ideational and Inter-personal Salience of the LoA.

In Plate 7.3, which advertises the perks of the Ml telecommunicationsservice, the woman is Salient (Kress and van Leeuwen, 1996), while inO'Toole's (1994) terms, she has Prominence. She is the most illuminated bythe Emblem, that is, the logo of the 'Sun' which represents Ml. Her mega-watt smile lends her affective appeal. The female model is thus the LoA.

Plate 7.3 illustrates two Complements to the LoA in the Ml advertise-ment:

(1) Comp.LoA 1: the two boys beside the LoA, who are reduced in size andare lacking in affective appeal, and therefore visually less inviting thanthe smiling LoA. Their Stylization (O'Toole, 1994) differs from themodel's. They do not hold the Ml placard, and are not smiling, whichimplies also that they would never be able to say 'Everything they offer isbrighter, nicer and more fun'. The Comp.LoAl subordinates itself to bringthe LoA and the Emblem into Focus. The LoA and the Emblem becomethe confluence of all attention.

(2) Comp.LoA2: the background, which remains in dark hues and fails tobe illuminated, despite the spotlights. The backgrounded stalls and thegoods the stalls are selling are generally obliterated and unobservable.This may be contrasted with the LoA who is illuminated by the Emblemof the 'sun' (that is, Ml) she is holding, while the spotlighted back-ground, ironically, fails to brighten up. Thus the Comp.LoA2 under-scores the prominence of the LoA and by extension, the prominence ofthe product (that is, Ml). The Comp.LoA2 thrusts the LoA and theproduct (that is, Ml) into viewers' attention.

Juxtaposing the LoA with the Comp.LoAl, we see an interplay of meaningbetween the visual images, that is, the LoA represents those who have andenjoy the Ml benefits and therefore are happy, while the Comp.LoAl repre-sents those excluded from such benefits. The ideology of exclusivity becomesapparent. Without an Ml subscription, life will not be 'brighter, nicer andmore fun f . Thus, be bright and make the wise choice of subscribing to Ml.


Plate 7.3 Generic structure of the Ml advertisement

PRINT MEDIA 171

The Display

Figure 7.1 The Display in a print advertisement

The LoA can also function as an Implicit Display in certain advertisementswhere the Display refers to the photographic Display of the product or ser-vice in the advertisement. If the product advertised is in a tangible form, forexample the Golf, it is termed Explicit Display. In comparison to the Beetleadvertisement (Plate 7.4) which employs symbolism as an advertising strat-egy, the Golf in the Golf advertisement can also be construed as a Congru-ent realization of the product, as no symbolism is involved. Therefore, theGolf is construed as ExplicitCongruent Display.

However, some products or services are intangible, or difficult to Capturein tangible form. For instance, the '118 off-peak hours every week' serviceprovided by Ml is not a tangible product with a physical form that can becaptured in print. Thus, the advertisers find a way of portraying such aservice through the smiling model who has obviously been the beneficiaryof such a service. The model, which has been previously established as theLoA, is then also the Implicit Display of the product/service. She personifiesthe '118 off-peak hours every week' service. As can be seen, a conflation offunctions is possible in an advertisement, that is, the LoA conflates with theImplicit Display, as illustrated in Plate 7.3.

Arising from creative advertising strategies in the Beetle advertisementdepicted in Plate 7.4, the insect beetle is creatively used as a substitute forthe car, the New Beetle. This substitution strategy makes the beetle anImplicit Display of the product which is a car. The insect beetle symbolizingthe car operates as an Incongruent realization of the product. Thus it can beconstrued as Implicit:Incongruent Display.

The Emblem, the Announcement and the Enhancer

The Emblem may be realized visually as the logo of the product/serviceadvertised and its linguistic realization is in the form of the brandname ofthe product/service. Ideationally and ideologically, it is the stamp of author-ity bespeaking and validating the authenticity of the product advertised.The Emblem functions to bestow an identity, as well as to confer status to a

Display

Explicit: pictures of a tangible product

Implicit: an intangible product or service given tangibleform through another medium

Congruent: product not realized through symbolism

Incongruent: product realized through symbolism


Plate 7.4 The Beetle advertisement

PRINT MEDIA 173

product. The Emblem may be positioned anywhere in the advertisement.However, it is interpersonally Salient to Capture attention. The Emblem inthe Ml advertisement is the logo of the 'Sun' and the brandname 'Ml', asdepicted in Plate 7.3.

In a print advertisement, the most Salient linguistic item/s are termed theAnnouncement. The Announcement has Relative Prominence in Scale andColour, Font and Size (O'Toole, 1994). Ideationally, the Announcement cap-tures and conveys the essence of an intended message the advertisers wish toforeground to the consumers. Figure 7.2 Displays the functional realizationsof an Announcement. The examples are taken from the Golf and Mladvertisements (Plates 7.1 and 7.3 respectively).

The Enhancer comprises linguistic items only, usually in paragraph form,as exemplified by the labelled advertisements above. The Enhancer builds onor modifies the meaning emanating from the interaction between the Leadand the Announcement. Interpersonally, its function is to persuade andinfluence viewers to purchase the product, thus the Enhancer containsInterpersonal lexis (in bold print below), which carry an attitudinal and/oraffective thrust. Through Interpersonal lexis, 'texts/speakers attach an inter-subjective value or assessment to participants and processes by reference toemotional responses or to systems of culturally-determined value systems'(White, 1999). Ideationally, it details the advertisers' reasoning/argument asto why the product is worth the customers' attention and money, and soLogical relations and rankshifted clauses are evident. The Golf advertisement

Announcement

The catch-phrase of an advertisement(E.g. 'Everywhere under the sun' (Ml))

Figure 7.2 The Announcement in print advertisements

The less interpersonally salient announcement/s amongother announcements in the same advertisement(E.g. 'I get the feeling that Ml wants me to enjoy value - and enjoylife. Everything they offer is brighter, nicer and more Jim!' (Ml))

(E.g. 'Bigger value. Better service. Brighter smiles. Nobody coversit all as nicely as Ml' (Ml))

Secondary

Defined as the only announcement in theadvertisement(E.g. 'It doesn't make a statement. It's for people who alreadyhave one' (Golf))

Defined as the most interpersonally salientannouncement among other announcements in thesame advertisement(E.g. '118 Off-peak hours every week' (Ml))

Primary


is one illustration of the use of Interpersonal lexis and interdependencybetween clauses.

//In its own confident and quiet style [[that have won endless admirationsthe world over]], the New Golf has come of age with a sophisticationbeyond comparison//

a //[[Setting itself apart and in a blistering pace]], is a new and awesome1.8 litre turbo engine//

X (3 //to take its performance to a higher level////Not only that, the beauty and luxury of the New Golf is also graced withequally exciting refinement both inside and outside//

1 //Truly the New Golf hasn't changed in spirit and valour//+ 2 a //but has gotten better//X (3 //to assert itself as the ultimate hatchback//

//No wonder it has been hailed as '. . . a triumph of execution' by UK'sCar's Magazine (January '99)////And termed by others as the 'Rolls-Royce of hatchbacks'//

The abundance of Interpersonal lexis in the Enhancer suggests room forthe application of Appraisal Theory which 'is concerned more particularlywith the language of evaluation, attitude and emotion . . .' (White,1999). However, space does not permit the investigation of AppraisalTheory in this context; no doubt future research in this area would beproductive.

The Tag and Call-and- Visit Information

Certain elements of information about a product/service that are notincluded in the Enhancer are captured in the Tag. The Tag is usually in theform of one-liners in small print and is typically non-Salient as illustratedin preceding labelled advertisements. Grammatically, Tags are usuallyrealized as non-finite, for example, 'Based on Super Off-Peak rates of 5c perrnirH in the Ml advertisement, and as ellipted Subject and finite element,exemplified by 'Available in 1.8 Turbo and 1.6 Automatic1 in the Golfadvertisement. Grammatically, there could be exceptions to the above but itis not within the scope of this paper to explore the lexicogrammaticalrealizations of Tags. As can be seen in the preceding labelled advertise-ments, the Call-and-Visit Information is usually in small print and non-salient, comprising contact information as to where, when, how theproduct/service is available to the consumer. For example, from the Golfadvertisement in Plate 7.1, 'Cars and Cars Pte Ltd. 10 Leng Kee Road,Tel:474-llir.

Revisiting the Generic Structure Potential (GSP) for printadvertisements

The GSP for advertisements in this paper is stated as:

PRINT MEDIA 175

LeadA (Display) AEmblemA (Announcement)A (Enhancer)A (Tag)A

(Call-and-Visit Information)

Table 7.2 is a brief survey of the advertisements analyzed in this paper andreveals which elements are optional, and which obligatory.

Evident in Table 7.2 is the diversity of choice as to which elementsare included or excluded from the advertisements. This study indicatesthat only the Lead and the Emblem occur in all the advertisementswhich have been analyzed. Thus the Lead and the Emblem appear to beobligatory elements, while the rest are optional in the GSP of printadvertisements.

Cook (1992: 216), quoting Barthes, states that advertisements represent a'resdess' discourse type. He explains (ibid. 217):

The conventions of ads change fast, driven by an internal dynamic, by changes insociety, and by changes in the discourse types on which they are parasitic or inwhich they are embedded . . . they are . . . constantly transmuting andre-combining, so that at present any lasting characterization is impossible. Syn-chronically, there are too many exceptions. Diachronically, the rules are in a flux.

Deriving a GSP for advertisements is thus made difficult due to this 'rest-lessness' of advertisements. The GSP for advertisements which I havederived is at best tentative, insofar as advertisements metamorphose alongwith 'changes in society . . . constantly transmuting and re-combining'

Table 7.2 Tabulation of Elements in the five advertisements

Advertisement Element/ s present inadvertisement

Element/s absent inadvertisement

Golf Lead, Emblem, Display,Announcement, Enhancer, Tag,Gall-and-Visit Information

Epson Lead, Emblem, Display,Announcement, Enhancer, Gall-and-Visit Information

Tag

Ml Lead, Emblem, Display,Announcement, Enhancer, Tag,Call-and-Visit Information

Beetle Lead, Emblem, Display,Announcement, Enhancer, Call-and-Visit Information

Tag

Guess? Lead, Emblem Display, Announcement,Enhancer, Tag, Gall-and-Visit Information


(ibid.}. I venture further to say that the GSP for advertisements is chameleon-like, slippery to define and ever-evolving. Further research into the GSP ofadvertisements, which needs to be conducted in greater breadth and depth,may produce a different GSP. For example, Hasan (1996) establishes'CaptureAFocusAJustification' as the generic structure for advertisements butmy research has produced a GSP which differs in terms of the degree ofdetail and the ability to Capture the complexity of intersemiosis inadvertisements.

Strategies for Ideational meaning-making in multi-semiotictexts

The section above introduces the Visual Metaphor which ideationally eluci-dates and enhances the Ideational meaning potential of the linguistic text inthe advertisement. I introduce another four strategies for Ideational mean-ing in Table 7.3.

Table 7.3 The construal of Ideational meaning in print advertisements

Ideational meaning

Strategies for meaning-making 1. Bidirectional investment of meaningin a multi-semiotic text 2. Contextualization Propensity

3. Interpretative Space4. Semantic effervescence

Generic Structure Potential of a LeadA(Display)AEmblemA(Announce-print advertisement ment)A(Enhancer)A(Tag)A(Call-and-Visit

Information)

The Bidirectional Investment of meaning refers to the cross-investment oflexicogrammatical meaning in the linguistic text in the Announcement tothe visual image in the Lead and vice-versa. The Contextualization Potential(CP) refers to the degree to which linguistic items in a print advertisementcontextualize the meaning of the visual images. In a print advertisement,viewers have an Interpretative Space (IS) within which to create meaningand the wider the IS, the greater the Semantic Effervescence (SE) of theadvertisement. The sections below further elaborate.

Lead, Announcement and Enhancer: a triumvirate approach tomeaning-making

The Ideational metafunction is concerned with 'understanding] theenvironment' (Halliday, 1994: xiii), '[enabling] humans to ... make sense ofwhat goes on around them and inside them' (Halliday, 1994: 106). Figure7.3 outlines the four stages of triumvirate interaction between the Lead,

PRINT MEDIA 177

Effervescence ofmeaning; akaleidoscope ofmeaning;Low GP, wide IS,high SE

Contextualizationof meaning;options of meaningunintended byadvertisers closedoff

Meaning inadvertisementfunnelled towardsa preferreddirectionintended by advertisers

Stability inmeaning; Xnumber of meaningsintended byadvertiserscommunicated toand receivedby viewer.High CP, narrow IS,lowSE

Figure 7.3 Triumvirate Interaction of Lead, Announcement and Enhancer

Announcement and Enhancer in construing Ideational meaning in aprint advertisement. Stages 1-4 detail how new dimensions of meaning maybe accessed and made manifest through the interaction of the Lead,Announcement and Enhancer.

Stage 1

The Lead in the Golf advertisement is the most interpersonally Salient, asseen in Plate 7.1, and thus this element is first approached by viewers. The


gaze of the LoA locks with the viewer, and the latter is led into the adver-tisement. If interpreted independently of the Announcement, Enhancer,Display and Emblem, the Lead is figuratively an effervescence of meaning.As mentioned above, the Lead could represent an ambassador for women'srights, or a call to attention to the new millennium look. On its own, akaleidoscope of possible meanings characterizes the Lead. There is a widescope in terms of meaning potential in the Lead. At this stage, there is lowCP, wide IS and high SE.

Stage 2

In a print advertisement, the next most Salient item is the Announcement,thus the Primary Announcement is second in the reading path. There isBidirectional Investment of meaning between the Announcement (thelinguistic code) and the Lead (the visual code) as illustrated in Figure 7.3.The term Investment refers to the Bidirectional Investment of meaningfrom the lexicogrammatical choices in the Announcement to the visualin the Lead and vice-versa. For example, should the Announcement in theGolf advertisement 'It does not make a statement. It's for people whoalready have one' occur elsewhere, for instance on the back of a T-shirt,it would connote different meanings. Similarly if the Lead of the Golfadvertisement appears in a different context, for instance in a Playboymagazine, it would have different connotations from what it has here. Sohow are the viewers constructed by the advertisers to read the meaning inthe Golf advertisement that the LoA represents someone with a statement-making personality? After all, the Announcement is a linguistic code whilethe Lead is a visual one. How does the juxtaposition of two different codesresult in meaning that can be unambiguously conveyed by the advertisersand unambiguously decoded by the viewers? I propose that the juxta-position of the linguistic texts and visuals sets up Transitivity processes thatinvest meaning from the linguistic code to the visual code and vice versa.The discussion that follows unravels and explicates the mechanics of thisBidirectional Investment of meaning from the Announcement to the Leadand vice-versa.

Stage 2 a

In the Golf advertisement (Plate 7.1), there is a Relational:Attributive:Intensive process between the Primary Announcement and the Lead. TheAttribute 'statement-making personality' is invested from the PrimaryAnnouncement into the Carrier (that is, the LoA in the Lead) by virtue oftheir proximity, thus causing viewers to see the LoA as a person with astatement-making personality (Figure 7.4):

PRINT MEDIA 179

Primary Announcement

'It does not make a statement. It's for people who already have one'

Investment

Relational: Attributive: Intensive process occurs between PrimaryAnnouncement and Lead.The Attribute 'statement-making personality' is invested into the LoA inthe Lead. The LoA is construed as Carrier with such an Attribute.

Lead'Visual of LoA'

Figure 7.4 Investment of Meaning from Primary Announcement to Lead

Due to the Relational process between Primary Announcement and Leadwhich invests meaning from the former to the latter, viewers read theExperiential meaning in the Golf advertisement.

The LoA

The LoA

Statement-makingindividuals

Carrier

has

is

are

Attributive:Intensive

a 'statement-making personality'.

statement-making

beautiful, sensuous, stylish

Attribute

The Primary Announcement thus acts as a stabilizer for an otherwisesemantically efflorescent Lead. The Primary Announcement provides a con-text for viewers to adopt/pursue the preferred thread of meaning intendedby the advertisers. Whatever 'it' refers to, 'it' is only for beautiful, sensuousand stylish statement-making individuals. Even at this early stage, a fluxof ideologies emerges, that of elitism and exclusivity, where only statement-making individuals deserve the Golf. With elitism and exclusivity arise'social power' (Goldman, 1992: 115) and an endowment of status, alsogender and beauty stereotyping, as the LoA in the Lead defines and epitom-izes allure, beauty, charm and desirability.

Stage 2b

Figure 7.5 illustrates how the Lead enriches the Ideational meaningcarried in the Primary Announcement. The sophisticated, sensuous, coy-looking LoA in Plate 7.1 is a visual exemplification of the statement'people who already have (a statement to make)'. A RelationaLIdentifying:Intensive process occurs in the Investment of meaning from Lead toAnnouncement.


Primary Announcement

'It does not make a statement. It's for people who already have one'

InvestmentRelational: Identifying: Intensive process occurs betweenAnnouncement and Lead.The sophisticated LoA in die Lead is the Value exemplifying theToken 'It's for people who already have one'.

Lead

'Visual of LoA'

Figure 7.5 Investment of Meaning from Lead to Primary Announcement

Through the Identifying: Intensive process, viewers read the followingmeaning in the advertisement:

The LoA

Token

represents

Identifying: Intensive

'people who already have(a statement to make)'

Value

At this stage, another ideological perspective emerges: the promoting andendorsing by advertisers of how statement-making women should look.Should one paradigmatically replace 'It doesn't make a statement. It's forpeople who already have one' with 'Dangerous: wanted convict', the LoAwould assume a different meaning. Thus the point is she means what shemeans, whether as a statement-making person or as a dangerous convict,due to Relational processes that invest meaning bidirectionally from thePrimary Announcement to Lead and vice-versa. Barthes's (1977:40) conceptof'anchorage' operates when:

the text directs the reader through the signifieds of the image, causes him to avoidsome and receive others; by means of an often subtle dispatching, it remote-controls him towards a meaning chosen in advance. In all these instances ofanchorage, language clearly has a function of elucidation, but this elucidation isselective . . .

Barthes (1977), however, does not address how this selective elucidation isachieved. The ongoing discussion on Transitivity processes between theAnnouncement and the Lead, resulting in the Investment of meaning bi-directionally, is proposed as the crux to this selective elucidation.

Not all Announcements, however, enter into a Relational process with theLead. The Primary and Secondary Announcements in the Ml advertisement

PRINT MEDIA 181

(Plate 7.3) enter into Relational, Verbal, Mental and Material processes withthe Lead (Figure 7.6).

Defined as the most Interpersonally SalientAnnouncement among other Announcements in thesame advertisement(E.g. (1) '118 Off-peak hours every week')

The juxtaposition of the Lead and the Primary Announcement 1 gives rise tothe following possible meanings:

She

Carrier

She

Senser

Ml

has

Attributive: Possessive

enjoys

Mental: Affect

gives to her

118 off-peak hours every week(because she uses Ml)

Attribute

118 off-peak hours every week(because she uses Ml)

Phenomenon

118 off-peak hours every week(therefore she is smiling)

Announcements

Primary

The catch-phrase of an advertisement(E.g. (2) 'Everywhere under the sun')

Secondary The less Interpersonally Salient Announcement/samong other Announcements in the same advertisement(E.g. (1) 'I get the feeling that Ml wants me to enjoy value — and

enjoy life. Everything they offer is brighter, nicer and more fun!')

(E.g. (2) 'Bigger value. Better service. Brighter smiles.

Nobody covers it all as nicely as Ml')

Figure 7.6 The Primary and Secondary Announcements in the M1 Advertisement

At the most obvious level, the LoA in the Lead is the Sayer, with 'I getthe feeling that Ml wants me to enjoy value - and enjoy life. Everything they offer isbrighter, nicer and more fun!' as the Locution and the viewers as the Receiver.Underscoring this is an ideology of persuasion particularly through theAttributive:Intensive process in the Locution:

//Everything [[they offer]]

Carrier

is

Attributive: Intensive

brighter, nicer andmore fun!//

Attribute


Actor

She

Carrier

Material Beneficiary:Recipient

is


Range

bright [[to choose Ml (with its118 off-peak hours perk)]], thusshe stands out from the rest

Attribute

Again in Barthes's (1977) words, how does the language 'remote-control'viewers to read the above meanings in the advertisement? How does theAnnouncement selectively elucidate the meaning in the Lead? I propose thatRelational, Mental and Material processes occur between the Announcementand the Lead, resulting in a preferred reading intended by the advertisers.

Stage 3

The advertisers are able to convey, and viewers are able to receive, themeaning of a satisfied Ml customer unambiguously because the juxta-position of Announcements and Lead has resulted in the above Transitivityprocesses. Should the Lead be paradigmatically replaced by a visual ofuncongested roads in the city, the meanings conveyed would definitely bedifferent. Again, although the Announcements are linguistic codes andthe Lead a visual one, their juxtaposition still creates meaning due to theTransitivity processes between them. These meanings that result from theinteraction between the Announcements and the Lead are built on or modi-fied (depending on the advertisers' intention) by the Enhancer.

An analysis of the Transitivity processes in the Enhancer reveals how itbuilds on the meanings generated by the Announcements and the Lead. InStage 2b one meaning generated between the Lead and the PrimaryAnnouncement in the Ml advertisement is:

Ml

Actor

gives

Material

to her

Beneficiary: Recipient

118 off-peak hours every week(therefore she is smiling)

Range

Material processes in the Enhancer build on this meaning, with Ml as actor.

//Nobody

Actor

covers

Material

it

Range

all

Circumstance :Manner: Quality

as nicely as Ml//

Circumstance :Manner:Comparison

PRINT MEDIA 183

//Ml

Actor

offers

Material

(to you)

Beneficiary:Recipient

more off-peak hours

Range

than anyoneelse.//

Circumstance:Manner:Comparison

1

+2

//Weekendoff-peakhours,

Token

//

at 50% off

Circ:Manner:Quality

and

start

Rel Ident:Intensive

last

Rel Att: Circ

at 7 p.m.

Value

every Friday//

Circ: Frequency:Time

right through the weekend//

Attribute

Note: Att = Attributive (process), Benef = Beneficiary, Circ = Circumstance,Ident = Identifying (process), Rel = Relational (process)

Ml is constructed as the Actor providing services benefiting the Recipients.Thus, the Recipients have only to subscribe to Ml to enjoy all the benefits.The ideological perspective that emerges is one of persuasion. The Circum-stantiahManner: Comparison ('than anyone else') strengthens this persuasivevoice. So do the Relational participants, which emphasize the frequency andduration of Ml benefits.

As mentioned earlier in Stage 2b, another meaning that arises from theinteraction of Primary Announcement 1 and Lead in the Ml advertisementis:

She

Sensor

enjoys

Mental: Affect

118 off-peak hours every week (because sheuses Ml)

Phenomenon

The Enhancer builds on this meaning of enjoyment through theclause:

//But that

Carrier

is

Attribute: Intensive

not all [[an Ml customer enjoys]]//

Attribute


In the Relational clauses below, the 'Free talk time' is accorded a Value and anAttribute, to attract viewers to these benefits that they can enjoy. Herein,again, lies the ideology of persuasion.

//Freetalk time

Token

//which

Carrier

(is) worth

Identifying:Intensive

can be

Attribute:Intensive

(iporv3>4U

Value

as much as400 min//

Attribute

every month

Circumstance :Extent:Frequency

with the MlPrimePlan//

Circumstantial:Accompaniment

The reading path for the Ml advertisement is displayed in Figure 7.7,according to the layout of the original Ml advertisement.

Halliday's (1994) model of expansion for logical meaning can beadapted and applied to the Ml advertisement. The notion of expansionincludes:

(B) Primary Announcement 1: Second in Salience therefore read second.

The meaning of (A) is further enhanced, LoA is smiling because of the '118 off-peak hoursevery week'.

(C) Secondary Announcement 1: Third in Salience therefore its reading follows PrimaryAnnouncement 1. Meaning of Primary Announcement 1 further enhanced, that is '118off-peak hours eveiy weeK is not only of good value, it enables one to enjoy life. The '118 off-peak hours every weeK is a means to a 'brighter, nicer and morejuri lifestyle.

(A) Lead is visually most Salient therefore viewers interact with it first.

(D) Secondary Announcement 2: Next in Salience after (C) and enhances the meaning of (C).

(E) Enhancer: Read last since it is at die bottom. The Enhancer further builds on what itmeans to have a 'brighter, nicer and more Jim' lifestyle, by describing the benefits.

Figure 7.7 Reading Path for the Ml Advertisement

PRINT MEDIA 185

(a) elaboration, represented by the notation '—'(b) extension, represented by the notation '+'(c) enhancement, represented by the notation 'x'

In the M1 advertisement, the Logical relations between the elements in theadvertisement may be expressed as:

LeadX Primary Announcement 1X Secondary Announcement 1X Secondary Announcement 2X Enhancer

Stage 2a mentions the interaction between the Announcement and the Leadin the Golf advertisement which sets up Relational processes resulting in thefollowing meanings:

The LoA

Token

The LoA

The LoA

Statement-makingindividuals

Carrier

represents

Identifying: Intensive

has

is

are


'people who already have(a statement to make)'

Value

a 'statement-makingpersonality'

statement-making

beautiful, sensuous, stylish

Attribute

The Enhancer, which is the paragraph detailing additional informationabout the car, and by extension the LoA, builds on the meaning of 'state-ment-making personality'. The following Circumstance types, Carriers andAttributes are amplifications of what it means to be statement-making:

In its own confidentand quiet style[[that has wonendless admirationsthe world over]]

Circ: Manner:Quality

the NewGolf

Carr

has

Att:Poss

come ofage

Attr

with asophistication

Circ:Accomp

beyondcomparison

Circ:Manner:Quality


//[[Setting itselfapart even further...]]

Carr

//Notonlythat

the beautyand luxuryof the NewGolf

Carr

is

Att: Intensive

is

Att: Poss

also

Circ:Accomp

a new and awesome 1.8 litre turboengine [[to take its performance toa higher level]]//

Attr

graced

Att:Poss

withexcitingrefinement

Attr

both insideandoutside//

Circ:Location

Note: Accomp = Accompaniment, Att = Attributive (process), Attr = Attribute,Garr = Carrier, Girc = Circumstance, Poss = Possessive (process)

The Enhancer functions to amplify the meanings generated between thePrimary Announcement and the Lead. The Display (that is, the Golf), andthe LoA, if unaccompanied by the Enhancer, would not realize the mean-ings advertisers intended. Figure 7.8 illustrates the reading path according tothe Compositional layout of the Golf advertisement.

Adapting Halliday's (1994) system of Expansion for Logical meaning,relations in the advertisement may be expressed as:

Lead= Primary AnnouncementX Enhancer

To review, in Stage 1 of Figure 7.3, meaning in the Lead is initially effer-vescent and unstable. By the time the viewer reaches Stage 3, the initiallyeffervescent meaning is straitjacketed by the Announcement/s andEnhancer. The meanings intended by the advertisers become crystal-clearand are unambiguously communicated by the advertisers and unambigu-ously received by the viewers. We have moved from a meaning which iseffervescent, unstable and ambiguous to one which is stable and con-strained. Any meaning options not intended by the advertisers areeffectively closed off. Suffice to say at this point of the discussion, withreference to Figure 7.3, that the Interpretative Space is narrower andthe Semantic Effervescence low due to the contextualizing effect of theAnnouncement/s and Enhancer in Stage 3. The Contextualization Pro-pensity is thus higher in Stage 3. This is in contrast to the low Contextualiza-tion Propensity, wide Interpretative Space and high Semantic Effervescenceof Stage 1.

PRINT MEDIA 187

(C) Enhancer is least in Saliencetherefore read last. The meaninggenerated through the interactionbetween Lead and Announcementis 'the LoA exemplifies people whoalready have a statement', 'She isstatement-making'. The Enhancerbuilds on this meaning, thatstatement-making people have'exciting refinement', 'setting(themselves) even further apart',etc.

(B) Primary Announcement read second as it is second in Salience. ThroughRelational processes that invest meaning Bi-directionally fromAnnouncement to Lead and vice-versa, the Announcement serves to definethe Lead as a visual exemplification of the Announcement. There is semanticequivalence between Lead and Announcement.

(A) The Lead is visually most Salient therefore read first. The LoA carries somemeaning but we are not sure yet what meanings advertisers intend her tohave till she interacts with the Announcement.

Figure 7.8 Reading Path for the Golf Advertisement

Stage 4

The total meaning derived from the interaction between the Lead,Announcement/s and the Enhancer needs to be read in the socio-culturalcontext within which it is placed. The meaning of the entire advertisement,according to Wernick (1991: 42):

delivers back to the people the culture and values that are their own . . . [it is] areinforcement of whatever ideological codes and conditions [that have] come toprevail.

Moreover, advertisements (Dyer, 1982: 77) project:

the goals and values that are consistent with and conducive to the consumereconomy and [socialize] us into thinking that we can buy a way of life as well asgoods.

However, society's ideologies are in continual evolution and metamorphosis.The ever-shifting ideologies will influence the way society interprets


advertisements. Whether society reads a marked or unmarked interpret-ation in the advertisements is 'culturally determined and changes over timeand may also eventually result in a narrowing of the meaning of an option'(O'HaUoran, 1999: 320).

Gontextualization Propensity, Interpretative Space and SemanticEffervescence: a further exploration of Ideational meaning

I discuss in greater detail here the Contextualization Propensity (CP), Inter-pretative Space (IS) and Semantic Effervescence (SE). As mentioned above,the generic structure of a print advertisement constitutes visual as well aslinguistic components, and the interaction between these components cre-ates Interpersonal, Ideational and Textual meanings. I further illustrate inFigure 7.3 that through the Bidirectional Investment of meaning betweenvisual and linguistic components, the meaning of the visual images, such asthe Lead, is contextualized by the linguistic items, for example theAnnouncement/Enhancer. Without the contextualizing function of linguisticitems, the Lead, as previously mentioned, has a bounty, a kaleidoscope ofmeaning and has great meaning potential. The GP refers to the degree/extent which linguistic items in a print advertisement, be it the Announce-ment, the Emblem and/or the Enhancer, contextualize the meaning of thevisual images. Thus the degree of interconnectedness and the degree ofinterweaving of meaning between the Scene, Episode (O'Toole, 1994) andthe participants/processes in the visual images and linguistic text determinethe degree or extent of contextualization, as illustrated by the Epson adver-tisement. Such advertisements have high CP. Where a minimum of lin-guistic items accompany the visual images, and less definable relationshipsare established between the linguistic and visual codes, as illustrated in theGuess? advertisement, the meanings of the visual images are less contextual-ized. These advertisements exhibit a low CP.

The low CP Guess? advertisement and the high CP Epsonadvertisement

Advertisements with a high CP allow viewers to read specific strands ofmeaning intended by the advertisers. In the above discussion of the Epsonadvertisement, linguistic items (a)-(e) contextualize the meaning of the LoA,that is, the splash of water in the Epson advertisement, depicted in Plate 7.2.Linguistic items (a)-(e) are redisplayed below for convenience of reference:

(a) 'EPSON STYLUS PHOTO EX - crystal-clear, photographic qualityprinting'

(b) //Six specially formulated colour inks deliver richer, more lifelike images//(c) //while EPSON PhotoEnhance provides realistic colour balance every

time//(d) //The EPSON Stylus Photo EX can transform your photography//(e) 'EPSON Stylus. The most advanced inkjets.'

PRINT MEDIA 189

Linguistic items (a)-(e) provide the context within which the meanings of theLoA may be negotiated and established. As the LoA is more contextualizedby linguistic items (a)-(e), the meaning of the LoA becomes more strait-jacketed. Such a scenario defines a high CP in an advertisement. With ahigh CP, the viewers' interpretation of the LoA is constricted, with a low-ered freedom to read other meanings in the LoA given the semantic input bylinguistic items (a)—(e).

The CP, therefore, has ideological implications. A greater Propensity forContexualization implies greater effort by the advertisers (through the lin-guistic items) to introduce specific strands of meanings. One is discouragedto read alternative meanings in the LoA given the context by (a)-(e). Theviewers thus have limited IS, that is, space to create, invent and authormeaning. This of course does not mean that alternative readings do notoccur. A critical reader can interpret the intended meanings and offer fur-ther perspectives other than those intended by the advertiser.

As illustrated in Plate 7.5, the CP is low in the Guess? advertisement asthere is only one lexical item, namely 'Guess?' to contextualize the meaningof the entire Lead, made up of the LoA, that is, the model whose limbs shinewith metallic sheen, and the Comp.LoA, that is, the background. Apart fromthe possible reading that the LoA is in some way related to Guess?, which isthe brandname of a fashion product known for its watches and clothes, andthat there is the underlying message that Guess? fashion is trendy, chic and invogue, the entire Lead is an effervescence of meaning as there is a lack ofcontextualizing function by linguistic items. Arising from this lack ofcontextualization, that is, a low CP, a myriad of interpretations of the LoA ispossible: the LoA with the metallic sheen-like complexion is a probablepersonification of the futuristic stance Guess? adopts towards fashion; thecurrent Guess? trend is the minimalist look, as exemplified by the generousshow of legs and body swathed with a minimum of cloth; the Guess? con-sumer is bohemian in outlook, as is the LoA whose cascading hair is caressedby the wind and throws a cold, removed glance at the viewer and the world;the Guess? consumer looks down at the world in nonchalance, articulatingthe superiority of the product and hence the consumer who chooses to useGuess? products. Guess? is thus selling an attitude, a certain style of living;Guess? products hint of sexual attractiveness and availability (as signifiedthrough the high-split in the skirt and slightly parted legs), which can beextended to imply the non-conformist nature of Guess? products, whichchallenge the conservative mould of society; Guess? applauds the flat-chested female as opposed to society's fascination with and celebration of theamply endowed female, again a hint of Guess?'s non-conformist ideology;dark skies and seas fail to intimidate Guess? consumers, who are able to puttheir best foot forward in style and confidence, the Stance (O'Toole, 1994)adopted by the LoA; Guess? is beyond definition, there is no single aspect toits fashion statement. Guess? products, it seems in this particular advertise-ment, have limitless possible interpretations within the semantic realm of'the desirability' of this label, and that is likely to be the message intended by


Plate 7.5 The Guess? advertisement

the advertisers. The IS in the Guess? advertisement is thus wide. There is apun on the Emblem 'Guess?': viewers are left guessing the most likely validreadings of the advertisement. A low CP is no less ideological than a highCP. That the advertisers allow viewers a larger and wider IS suggests that

PRINT MEDIA 191

advertisers wish the consumers to purchase the illusion that consumers areempowered to create meanings for themselves in an advertisement. Theideology of manipulation is no less evident, for by thinking they have free-dom to interpret, the viewers have played themselves into the hands of theadvertisers. They have bought the ideology of Guess?, that is, there is nosingle definition of the Guess? fashion statement, so dress the Guess? wayand be open to interpretation by the (admiring?) eye of the public.

Graphical representations of CP, IS and SE

To summarize, a low CP allows a wider IS, as evidenced in the previoussections. There is greater SE of meaning in the Lead as a result of the lack oflinguistic items, which perform a contextualizing function in a print adver-tisement. Conversely, a high CP results in a narrower, limited IS, as seen inthe Epson advertisement. There is less effervescence of meaning in the Leadas viewer choice in the selection of meaning is constrained by the moreabundant linguistic items, which define more tightly the meanings of visualimages in a print advertisement.

The triumvirate correlation among the CP, IS and SE in the Lead can becaptured graphically, as illustrated below. 'A' in Figure 7.9 indicates theregion that the Guess? advertisement is likely to be positioned. With fewlinguistic items to provide an interpretative context for the Lead (that is, a lowCP arises), there is greater SE in the Lead, and thus greater InterpretativeSpace (wide IS) for the viewer to roam and make meaning. This situationcorresponds approximately to Stage 3 in Figure 7.3 above.

The Epson advertisement is likely to be positioned in the vicinity of'B' inFigure 7.9, as the advertisement contains an abundance of linguistic textwhich provides the context within which the meaning of the LoA may bederived. That is, there is a high CP, which narrows the IS of the advertise-ment. The Lead has lower SE due to the high CP. Stage 1 of Figure 7.3reflects this.

Barthes (1977: 26) writes: 'formerly the image illustrated the text (made itclearer); today the text loads the image, burdening it with a culture, a moral,an imagination'. However, how much imagination and by what means thetext loads an image is not explained by Barthes (1977). The CP, IS and SE

Figure 7.9 Correlation between IS, CP and SE


which I propose can be used as a tool to elucidate the degree/extent ofimagination invested into a visual image by a linguistic text. The lower theCP, the greater the SE in the Lead and hence the wider and more open theIS, indicating greater loading of imagination into the visual image. A highCP, conversely, limits a freer loading of imagination from text to image asthe IS is narrower.

CP, IS, SE and Barthes's (1977) notion of readerly and writerlytexts

Barthes introduces the notion that texts vary in the degree to which they letthe reader enter into this creation of meaning from both the Textual and theextratextual factors. On the opposing ends of the scale, he places 'writerly'and 'readerly' texts (Bruns, 1998).

Bruns (1998) further quotes Barthes, stating that for readerly texts, thereader is 'left with no more than the freedom to either accept or reject thetext', as in the case of a technical manual, as opposed to writerly texts, whichoffer 'the reader more choice and try much less to push them in one or theother direction' (Bruns, 1998). Barthes is further quoted, 'The writerly text. . . has no determinate meaning' and 'can create a number of possiblemeanings for readers' (Bruns, 1998).

The Epson advertisement, with its high CP, narrow IS and low SE in theLead, may, in the light of Barthes's (1977) readerly and writerly texts, beconstrued as tending toward a readerly text while the low CP, wide IS andhigh SE in the Lead of the Guess? advertisement lends itself more as awriterly text.

A new dimension to the 'New' in Kress and van Leeuwen (1996)

Information Value, according to Kress and van Leeuwen (1996: 183), isCompositionally determined. The information value of a left and rightComposition is construed as Given and New information respectively, where(ibid.: 187)

[the Given is defined as] something the viewer already knows, as a familiar andagreed-upon point of departure for the message. For something to be New meansthat it is presented as something which is not yet known, or perhaps not yetagreed upon by the viewer, hence as something to which the viewer must payspecial attention.

However, my proposal is that Given and New information need not be Com-positionally determined in this manner of left to right organization. TheGiven-New information value may be derived in any print advertisement, inany layout, whether with left-right or top-down Composition. The Guess?advertisement is a case in point.

PRINT MEDIA 193

From the low GP in the Guess? advertisement arises a multiplicity ofinterpretations of the LoA in the Lead, as discussed above, since there is alack of linguistic items to contextualize the meaning of the LoA. Thatviewers are given a wider IS to interpret the LoA, and that the LoA remainsSemantically Effervescent indicate that it is the Focus of the advertisement topresent the LoA. To reiterate Kress and van Leeuwen (1996: 187), 'some-thing which is not yet known, or perhaps not yet agreed upon by the viewer'.Ideationally, the LoA in Plate 7.5 is ambiguous, teeming with possible mean-ings. The LoA is thus construed as the New, while the Emblem 'Guess?' is theGiven, as viewers are not likely to have any argument with alternative

Figure 7.10 Mapping CP, IS and SE

(a)

Guess? advertisement Epson advertisement

Lesser degreeofCP

Greater degreeofCP

Degree of Contextualization Propensity

(b)

Epson advertisement

Narrower IS

Guess? advertisement

Wider IS

Expanse of Interpretative Space

(c)

Epson advertisement Guess? advertisement

Lesser amplitude of SE Greater amplitude of SE

Amplitude of Semantic Effervescence


interpretations of the brandname. Though the Guess? advertisement doesnot have a left-right composition, New and Given information can still bederived, thus strengthening my thesis that there is no need to limit Given-New information to a left-right Composition in a print advertisement.

Conclusion: moving towards a topological grammar

The analysis of multi-semiotic texts, such as print advertisements, necessi-tates the formulation of a topological grammar, one which can handle theanalysis of texts in terms of'degree, quantity, gradation, continuous change,continuous co-variation, non-integer ratios, varying proportionality,complex topological relations of relative nearness or connectedness, or non-linear relationships and dynamical emergence' (Lemke, 1998: 87). Theconcepts proposed in this paper, namely, Contextualization Propensity,Interpretative Space and Semantic Effervescence, can be seen as topological.These proposed concepts are resources for articulating gradients andnuances of meaning, and shades of significance in the multi-semiotic printadvertisement. Figure 7.10 explicates the varying degrees of CP, expanse ofIS and amplitudes of SE.

This paper has proposed a generic structure potential for advertisements,and, further to this, suggested strategies for construing Ideational meaning inmulti-semiotic texts. There still remains a vast expanse to be traversed, withexciting opportunities to further explore meaning-making of multi-semiotictexts from a systemic-functional perspective.

Acknowledgements

Plates 7.1 and 7.4 are reproduced with kind permission of Volkswagen.Plates 7.2, 7.3 and 7.5 are reproduced with kind permission of Epson,MobileOne Ltd and Guess? Inc, respectively. The credits for the photo-graph in Plate 7.5 are due to creative director Paul Marciano and photo-grapher Dah Len.

References

Baldry, A. P. (ed.) (2000) Multi-modality and Multimediality in the Distance Learning Age.Gampobasso, Italy: Palladino Editore.

Bardies, R. (1977) (S. Heath, ed. and trans.) Image-Music-Text. London: Fontana.Bohle, R. (1990) Publication Design For Editors. New Jersey: Prentice Hall.Bruns, A. (1998) Major Terms in Structuralism: Text, Reading, Author, Intertextuality,

Discourse, (http://www.uq.au/~zzabruns/uni/en22l-ass05.html).Cheong, Y Y. (1999) Construing meaning in multi-semiotic texts — a systemic-

linguistics perspective. Unpublished masters thesis. National University ofSingapore.

Cook, G. (1992) The Discourse of Advertising (2nd edn 2001). London: Routledge.Dyer, G. (1982) Advertising as Communication. London: Routledge.Goldman, R. (1992) Reading Ads Socially. London: Routledge.

http://www.uq.au/~zzabruns/uni/en221-ass05.html

PRINT MEDIA 195


Halliday, M. A. K. and Hasan, R. (1985) Language, Context and Text: Aspects of Languagein a Socio-Semiotic Perspective. Victoria: Deakin University. (Republished by OxfordUniversity Press, 1989).

Hasan, R. (1996) What's going on: a dynamic view of context in language. InG. Gloran, D. Butt and G. Williams (eds), Ways of Saying: Ways of Meaning. Lon-don: Gassell, 37-50.



Lemke, J. L. (1998) Multiplying meaning: visual and verbal semiotics in scientifictext. InJ. R. Martin and R. Veel (eds), Reading Science: Critical and Functional Perspec-tives on Discourses of Science. London: Routledge, 87—113.

O'Halloran, K. L. (1999) Interdependence, interaction and metaphor in multi-semiotic texts. Social Semiotics 9(3): 317—354.

O'Toole, M. (1994) The Language of Displayed Art. London: Leicester University Press.Wee, G. K. A. (1999) A systemic-functional approach to multi-semiotic texts.

Unpublished honours thesis. National University of Singapore.Wernick, A. (1991) Promotional Culture: Advertising, Ideology and Symbolic Expression.

London: Sage Publications.White, P. R. R. (1999) An Introductory Tour Through Appraisal Theory. English Language

Research, Department of English, University of Birmingham, (http://www.grammatics.com/appraisal/Frame.htm).

http://www.grammatics.com/appraisal/Frame.htm

http://www.grammatics.com/appraisal/Frame.htm

8 Multimodality in a biology textbook

Libo GuoNational University of Singapore

Introduction

Introductory biology textbooks in current use in educational institutionsinvariably contain words and visual images, for example, schematic draw-ings, photographs, and mathematical and statistical graphs. Further, it is notonly recently that biology texts have been multimodal; drawings of animalsand plants have been used as an aid to the study of living organisms foragricultural, medicinal and biological purposes since ancient civilizations(Ford, 1992).

Sociologists or ethnomethodological researchers, notably Lynch (1990)and Myers (1990, 1995), have attempted to theorize about the deploymentof visual displays in biology texts. Lynch (1990: 153-154), for instance,believes that 'visual displays are more than a simple matter of supplyingpictorial illustrations for scientific texts. They are essential to how scientificobjects and orderly relationships are revealed and made analyzable'. In asimilar vein historians and philosophers of science have turned their atten-tion to the evolution and philosophical aspects of scientific (includingbiological) illustrations (see, for example, Baigrie, 1996). Although theseinvestigations have made significant contributions to our knowledge andunderstanding, they often seem to lack a coherent framework to explain howthe various visual displays make meaning in their natural and social settings.These approaches have shown us what is happening through videotaperecordings, verbal accounts and historical documents, but they have notbeen explicit enough about the systems and functions that underlie the useof visual images.

This paper explores the potential of an alternative approach to the studyof meaning-making practices in scientific discourses. This is the social semi-otic approach developed by M. A. K. Halliday (1978, 1994) as systemic-functional linguistics (henceforth SFL) and the emerging SFL-informed the-ory of multimodality (Baldry, 2000a, 2000b; Kress, 2000, 2003; Kress andvan Leeuwen, 1996; Lemke, 1998; O'HaUoran, 1999a, 1999b, 2003). Dueto the main purpose of my study, that is, helping non-native universitylearners of English cope with English for Specific Purposes (ESP) or Englishfor Academic Purposes (EAP), I confine myself to the study of textbook

PRINT MEDIA 197

articles in biology, sharing Myers's conviction that textbooks are the type ofwriting that university students are 'most likely to face' (Myers, 1992: 3). Theexcerpts analysed here are from Chapter 17 Cell Division of Essential CellBiology: An Introduction to the Molecular Biology of the Cell (henceforth ECB) byAlberts et al. (1998). This textbook is used as required reading material forsecond-year biology majors for Bachelor of Science degrees at the NationalUniversity of Singapore for the module of Cell Biology.

This paper is organized as follows. I first discuss the semantics of biologyand what biologists do that characterize them as biologists. Second, follow-ing O'Toole (1994), Lemke (1998), and O'Halloran (1996, 1999a, 1999b) Ipropose frameworks for the analysis of visual images in the textbook, and,following this, I analyse two multimodal composites and discuss how eachtype of resource contributes to meaning-making. This paper concludes byoutlining some of the implications of a multimodal approach for teachingESP/EAP to non-native speakers of English.

Biology and miiltimodality

Biology is 'the study of living things past and present, including their struc-ture, function, chemistry, development, evolution, and environmental inter-actions', the 'environment' here including both the physical environmentand the biological environment (Purves, 1999: 769). Out of the differentapproaches to studying life, two are particularly important to modernbiologists: observation and experimentation. Observation is to experiencethe living world and take note of the living organisms. This represents thenaturalist tradition of doing biology, exemplified by Charles Darwin(1809-1882). And today's biology majors at universities are required to goon field trips as part of their degree programme. The key to experimenta-tion, on the other hand, is manipulation and control of 'conditions in orderto reveal or produce observations that contribute to the solutions of puzzles'(Janovy, 1996: 44). 'Certainly molecular biology and all its older relativesrely on experiments, and experimentation is becoming more a part of eco-logical field research every day' (ibid.}. Observation and experimentation astwo important ways of studying life are reflected in many universities' cur-ricula designed for biology majors. The practical classes for biology majorsin the Department of Biological Sciences at the National University ofSingapore, for instance, account for 27.0 per cent and 39.5 per cent of thetotal contact hours of a first- and second-year student's learning life respect-ively, which strongly suggests that hands-on skills are crucial for a biologist intraining.

On the other hand, a biologist also reads and writes papers, textbooks,and other documents. Similarly, the bulk of a biology student's learningtime in the first two years at college is spent in classes and tutorials (Haas,1994: 59-63) where he or she is required to read, write and interpret verbaland non-verbal messages. Minds-on skills are as important as hands-on ones.As pointed out by Osborne (2002: 206), 'just as there can be no houses


without roofs or windows, there can be no science without reading, talkingand writing'.

Due to the nature of the inquiry of the discipline and its methodologicalapproaches, biology texts have always been multimodal, that is, deploying arange of semiotic resources in addition to natural language. The reason forthis is clear: natural language alone cannot adequately communicate orconstruct the process and product of observation and experimentation; thepotential of natural language as a typologically oriented semiotic resource(Lemke, 1998) falls far short of the semiotic demands of the discipline. Forexample, it will be difficult, if not impossible, to describe in natural languagealone the colours, shapes and the flight path of a butterfly.

Like research activities in other fields, many of the investigations in thebiological sciences are quantitative, involving the collection, presentation,analysis and interpretation of numerical observations. The biologicalresearchers apparently need an objective method of organizing the datacollected from field trips or experiments. In addition, they must draw sens-ible conclusions from the analysis of the data. Many have been guided bystatistics. In the US, statistics was first introduced to the university curric-ulum for biology students as early as 1897 at Harvard (Zar, 1999: x); biosta-tistics, or biometry, has nowadays become an important part of a biologystudent's education. This involves the deployment of appropriate statisticalprocedures and graphs in biology texts.

The semiotic demands of the discipline do not stop here. In cell biology,in particular, recourse to non-linguistic semiotic resources has been neces-sary since Robert Hooke (1635-1703) first drew a picture of the 'cell' seenunder his microscope as reported in Micrographia (1665). This time-honouredmorphological approach to the studies of the cell, with the help of a lightmicroscope and an electron microscope, has recently culminated in what weknow as the ultrastructure of the cell. To communicate what was observedunder the microscope, the cell biologists have developed a range of devices,including light micrographs, electron micrographs, and schematic drawings,each of which has several sub-types, depending on the techniques adopted.More recently, however, cell biologists have attempted to investigate thebiochemical basis of the structure and function of the living cell. Ratherthan merely describe the mechanical or morphological features of the cellu-lar life, this new approach seeks to account for the cell and cell activity interms of the structure and function of its chemical components, the fourmajor families of small organic molecules (sugars, fatty acids, amino acidsand nucleotides) and the macromolecules (polysaccharides, lipids, proteinsand nucleic acids). Most of the macromolecules normally exist as specificbiologically significant three-dimensional structures called conformations,for example, the double helix for DNA, the extended chain conformationfor cellulose and the a-helix, (3-pleated sheet, p-turn and loop conforma-tions for proteins. As noted by McMurry and Castellion (1999: xvi),'[understanding many aspects of chemistry — such as the specificity andselectivity of enzymes, or the action of drugs - requires understanding the

PRINT MEDIA 199

three-dimensional nature of molecules'. That is, the introduction of bio-chemistry means that the semiotic demands of the discipline have exponen-tially increased so that natural language, however important it may be, isinadequate when deployed as a single resource. As a result, other semioticmeans such as chemical notation, ball-and-stick models, space-fillingmodels, animations, video recordings and so forth have evolved for com-municative purposes. Natural language alone has been inadequate withmorphological research; it is naturally insufficient as a means to describeboth the morphological and the biochemical.

Theoretical frameworks for the analysis of biology texts

In what follows, I present the frameworks for analysing the biology text,which include the frameworks for analysing schematic drawings and stat-istical graphs. I also discuss the issue of the reading path in introductorymulti-semiotic texts.

Myers (1990: 233-249) identifies, 'in terms of realism and abstraction'(ibid.: 247), five categories of visual displays in a sociobiology text: photo-graphs, drawings, maps, graphs/models/tables, and imaginary figures (ibid.:234). The first three types have some reference to our everyday visualexperience while '[gjraphs, models, and tables redefine space [. . .] so that eachmark has meaning only in relation to the presentation of the claim' (ibid.:235). In many textbooks on the molecular study of the cell, one of which isECB, biochemical symbolism constitutes yet another semiotic resource. Forlack of space, however, I present the frameworks for the analysis of twocommon types of visual displays in biology: schematic drawings and stat-istical graphs.

Framework for analysing schematic drawings

By schematic drawings I refer to those that are designed to depict in asimplified way some scene or process, actual or imaginary. The functionsand systems chart for the analysis of schematic drawings is displayed inTable 8.1.

Although the rank scale in the chart follows O'Toole (1994: 24), thefunctions and systems are not, unsurprisingly, identical. For instance, inO'Toole's (1994) model, in the Modal function at the ranks of Work andFigure, Gaze is an important means deployed by artists to attract the attentionof the viewer. In the biological schematic drawings I have analysed, Gazedoes not appear to figure as an important resource. More importantly, inthe Compositional function, unlike in paintings where usually littie morethan a tide is provided to indicate what is depicted, in scientific illustrations,Labelling appears frequently. This feature is related to the pedagogic use ofthe schematic drawing. An important part of a biology student's training isto learn to recognize the shapes of components of an organism and learnhow these components are named by the scientific community; for example,

Table 8.1

Unit/Function

Work

Episode

Figure

Member

Functions and systems in schematic drawing (adapted from O'Toole, 1994: 24)

Representational

Overall shape;Components of the structure;Whole process;Phases of the process

Shape;Colour;Size;Spatial relation to each other, and tothe structure;Actions, events

Components; Acts

Natural form: Shape, colour, etc. andspatial relationship to othercomponents

Modal

Frame;Size;Scale;Perspective;Full colour or black and white;Colour contrast;Shade or light

Relative Prominence: Colour, amountof detail;Centrality;Lettering (for label and caption): typesize, style (serif or san serif), Weight;Line and arrow width;Numerical sequence

Contrast: Scale, Line, Light, Colour;Omission of detail

Stylization;C onventionalization

Compositional

Gestalt Framing, Horizontals,Verticals, and Diagonals;Proportion;Geometry;Colour;Drawing's relation to running text:Spatial and Colour;Labelling: Positioning, Colouring andLeaders

Relative position in the structure orprocess;Colour contrast between components

Relative position in the component orphase;Colour contrast or similarity;Subframing

Cohesion: Parallel/Contrast in Shapeand Colour;Reference through language

PRINT MEDIA 201

a certain shape is named the 'stem', or 'root', or 'microtubule'. Labels andLeaders provide in part the means for the enculturation of the learner intothe discipline of biology. The Representational meaning of the schematicdrawing is what Lemke (1998) calls the 'topologicaP meaning, especially, theShape, Colour, Size, Spatial relation to each other and to the whole structure,and Action. Such meanings are also typological in that they fall into categor-ies; for instance, the Shape is round, square, rectangular, and so on. But thepredominant aspect of these meanings in biology is topological where theirregularity defies any linguistic encoding except in the most general terms.The exact Spatial relations and the moment-to-moment movement in spacecan best be shown in a drawing or video recording rather than by verbaldescription.

Framework for the analysis of statistical graphs

Statistical graphs for frequency distributions, which include bar graphs, his-tograms, frequency polygons and so on, derive from data tables, which inturn originate from linguistic and mathematical expressions of some quanti-tative relation between a set of variables. In most cases, statistical graphsmake use of the coordinate system, that is, the horizontal #-axis designatingthe independent variable and the vertical j-axis designating the dependentvariable stand for the Given, and the space circumscribed by the two axes isthe New where the relations between the two variables are shown. Also thejy-axis represents the quantitative information, ratio- or interval-scale data,which is capable of being scaled. This means that thejp-axis's quantitativevalues can be turned into visually perceptible heights which can be com-pared visually: the higher the bar or point, the higher the value of the yvariable for the corresponding x variable. There is nothing in the height ofthe bar per se except its assigned meaning concerning the quantitative value.That is, a statistical graph, quite unlike a photograph, is an abstract theor-etical entity although it may have material form: a photograph resemblesthe perceptible object while a statistical graph constructs a theoretical object,which may be invisible to human vision prior to its material formation. Theframework for analysing statistical graphs, adapted from O'Halloran (1996:161), is presented in Table 8.2.

Compared with that for natural language, the Representational meaningof a statistical graph is more specialized in that it deals only with the relativenumerical relationships between two sets of variables, or how an attributeof some entities, for example, the height or weight of school children, isdistributed among a sample or a population, the latter being in essence acomparison between the entities in terms of the attribute. This visuallyexpressed topological relationship powerfully complements the semantics ofnatural language, which is typically typologically oriented (Lemke, 1998).Depending on the nature of the variable designated by the ;v-axis, a curvewithin a coordinate system may mean either a material process, as theoutput of the crop increasing or decreasing, or a relational process, as the

Table 8.2 Functions and systems in graphs (adapted from O'Halloran, 1996: 161)

Unit/Function

Representational Modal G ompositional

Graph Statistical reality: topological meanings,such as trends, continuous co-variations, correlation and frequencydistribution;Comparisons of patterns of variation

Accompanying text in the form ofCaption, Tide and Labelling which areemphasized by Size, Positioning,Underlining and Font;Colour, Line width, Shading, LineSolidarity, Arrows;Curvature;Perspective;Framing;Scale;Style of production;Directionality

Gestalt: Framing, Horizontals, Verticalsand Diagonals;Positioning;Use of Lines, Curves and Bars;Interconnections established throughsymbolism and language for thelabelling of Participants and Processes;Cohesion: links to the running text

Episode Change, or Relations between Figures Prominence of interplay Labelling of interplay

Figure Participants;Circumstantial features;Portrayal of co-variation associatedwith process as a Curve, Line or Bar

Prominence of individual figures;Displayed trend of process throughLine, Bar, Curve

Labelling of Figures throughsymbolism and/or language;Portrayal of Process betweenParticipants as Axes and Figure withrelative Positioning and Size of Figureand salient features as displayed byLines, Curves, Bars, Colour, Linewidth, and Shadings

Part Title;Axes, Scale, Arrows;Labels;Lines, Curves, Intersection points;Slope of Parts of the Figure

Stylization;Conventionalization

Cohesion: Parallelism, Contrast;reference through language and/orsymbolism

PRINT MEDIA 203

comparison between the entities. Interpersonally and experientially,although in principle a graph is as reliable or unreliable as the data thatinforms its compilation and, for that matter, is subject to the semiotic choicesmade in the display (for example, the selection of the scale), whenever astudent encounters a graph in the textbook, he or she is normally expectedto believe it rather than doubt it. That is, the graph carries with it a self-authenticating power and a high modality. Further, as noted by O'Halloran(1999b: 18), with exceptions '[Interpersonal] strategies for engaging theviewer of the mathematical visual display do not operate through nuance asfound in forms of art, but rather select for a direct unmarked command,"look here"'. Gompositionally, x- andjy- axes provide the basis or groundingwhere the New is expressed in the form of Curve, Line or Bar. The axes'contribute to stability and harmony' (O'Toole, 1994: 23), while the Curve,Line or Bar 'create[s] energy and dynamism' (ibid.}.

The reading path in introductory texts

Another crucial question with a multi-semiotic text is the reading path itmay create for its hypothetical reader. Underlying this question is the rec-ognition of the page as a Textual unit where various semiotic resourcesmake meaning (Baldry, 2000b: 42). As O'Halloran (1999a: 322) points out,'[wjith multi-semiotic texts, the most important stage is a step-by-stepanalysis of the text through the reading path determined by the choiceswithin different semiotic codes'. It is to be noted that the reading path in amulti-semiotic text identified by O'Halloran (ibid.} is not linear, from left toright, or from top to bottom, but typically follows some specific sequence(see also Kress and van Leeuwen, 1996: 218 if., 1998: 205-209; Kress, 2003:156-160).

As will be illustrated below in the analysis of multimodal texts, there seemto be two aspects to the reading path: the intersemiotic aspect, that is, how thereader is expected to shift his or her attention from one semiotic to another,and the intrasemiotic aspect, that is, how the reader is expected to move fromone component to another within one semiotic mode. Very often theintersemiotic aspect of the reading path in an introductory textbook is thatafter a brief'modal "scanning" of the page' (Kress, 2003: 159), or a quickperusal of visually salient elements, usually images, the reader moves fromthe verbal text (expressed by specific typographical features) to the non-linguistic resources and then to the verbal text again, thus following a back-and-forth reading sequence. Initially the visual image on the page exerts astrong impact upon the reader through choices such as Colour and Framing.After the initial visual impact subsides, however, he or she normally beginsto study the verbal text and may later be linguistically instructed to study thevisual image in greater detail. The relative privilege the verbal text enjoys inthe reading path is partially explained by the fact that at this stage of thestudent's education it is largely through the verbal language in the runningtext that he or she is instructed explicitly when to view and study the


non-linguistic resources and how to interpret them. In other words, how amulti-semiotic text 'indicate [s] to the reader/viewer the possible ways ofreading the text and the relative information priority to be assigned to thedifferent component parts of the overall visual composition' (Baldry, 2000b:42) and how a reader/viewer is expected to respond to the text constitute avisual semiotic strategy to realize the educational goals within which a mul-timodal text is constructed and interpreted. As the context of situation andculture (Halliday, 1978) within which the text operates changes, the readingpath, along with the nature of the semiotic resources employed, also differs.For example, Kress and van Leeuwen (1996: 219) and Lemke (1996: 216)have observed that some scientists have non-sequential reading habits andthat many books are not designed 'to be read only in the strict linear order inwhich the text appears on the pages' (Lemke, 1996: 216). This applies toestablished scholars or scientists, people who are beyond their basic 'militarytraining' periods (Knight, 1992: 6, 143) and for whom texts are created andread for research purposes rather than learning purposes. Students of biol-ogy or literature or any other field need eventually to learn non-sequentialreading. While they are still students, however, they may need in the multi-modal textbook page a clear, pre-coded reading path in order to enter theparadigms of contemporary science (Kuhn, 1996) and the practices andconventions that characterize scientific activity.

Analysis of the multi-semiotic text: two examples

Drawing upon the discussion of the frameworks for schematic drawings andstatistical graphs presented above, this section contains an analysis of aschematic drawing and a statistical graph from ECB.1

Analysis of a schematic drawing: Figure 17—3

Figure 17—3 (ECB, p. 549), together with the relevant verbal text, is repro-duced in Figure 8.1 (see Note 1).

The reader is formally introduced to Figure 17.3 when he or she reads thefollowing clause:

These two processes together constitute the M phase of the cell cycle(Figure 17.3).

However, he or she may not wait until being instructed to view Figure 17.3.Since Figure 17.3 is a full-colour drawing, a picture more attractive than thelargely black and white verbal text, a reader's attention is more likely to bedrawn to the drawing than to the written description. Thus one plausiblereading session may be that a reader, at some point in his or her reading,turns his or her attention to the figure, and then back to the verbal text forcareful study and then back to the figure again, following a back-and-forthtype of reading path as explained above.

PRINT MEDIA 205

Figure 8.1 Reading path for Figure 17.3

The reading path within Figure 17.3 is marked in Figure 8.1 by thecapitalized and italicized Roman letters A to G. As is clear from Figure 8.1,the reading path is not linear, from left to right, from top to bottom, but isdetermined ideationally by what is in focus in the running text (the M phaseof the cell cycle), and interpersonally by the visual means of directing thereader's attention (for example, the bright yellow Shading and Capitaliza-tion of MITOSIS and CYTOKINESIS and light green Shading of M phase and thelarge square bracket embracing MITOSIS and CYTOKINESIS in the original text).This is, in verbal and common parlance, equivalent to saying 'Hey, look atwhat is highlighted first!'. Indeed, in this part of the reading, Steps C and Dare all an experienced reader needs to attend to. The highlighting devicessuch as arrows are equivalent to a lecturer's cursor in an actual classroom,where he or she, while talking to the students, points to relevant parts of thefigures. Although in viewing Figure 17.3, one's gaze, especially that of anovice, may work from Step G down to Step D due to the Interpersonalimpact of the downward-pointing arrows and the reading habit of a normalreader, it is nonetheless arguable that the reading path suggested above is


most economical for the experienced reader, that is, one that has followedthe Textual explication up to this point.

At the rank of Work, interpersonally, this figure thus employs an array ofvisual means to emphasize various parts of the cell structure and stages ofcell division. Ideationally, the figure is designed to tell a story about whathappens in a cell cycle, in particular the M phase of the cell cycle. Theideational meanings include: (a) material processes realized by changes inthe shapes at different stages, the arrows and the nominal groups in thelinguistic text; (b) intensive identifying processes realized by the labels, lead-ers and the pictorial elements, and, in the absence of leaders by the labels,the spatial proximity between the pictorial element and the labels, and thepictorial elements; and (c) possessive identifying relational processes realizedby the labels, square bracket, the pictorial elements and the linguistic text.The overriding Experiential content seems to be concerned with materialprocesses, although the intensive and possessive relational processes con-tribute significantly to the construction of biological knowledge. And text-ually, the drawing is not isolated from other parts of the text. It is related tothe main text and the caption and is placed in a specific position on thepage. The drawing is vertically positioned, with the Arrows connecting onestage with another. Other resources employed for the textual meaninginclude Geometry (e.g. circles), Colour Contrast or Similarity, Labelling(with or without leaders) and Framing. In what follows, I analyse selectedsteps in terms of the Interpersonal (Modal) meaning, Ideational (Represen-tational) meaning and Textual (Compositional) meaning, by reference to thefunctions and systems chart in Table 8.1.

Step A: the title

Distinctive typographical features, such as the bold face of the title and thegreenness of the figure's caption number, function to attract the reader'sattention and thus attach more importance to this linguistic message. Thetitle is also the only explicit link to the main text; it is the reader's entrance tothe pictorial world of the figure. It is designed to be read first and taken asthe point of departure for what is to come next.

The title is a nominal group and apparently does not select an Inter-personal stance at the rank of clause in terms of SPEECH FUNCTIONS (Offer orDemand) and MODALITY and MODULATION (Halliday, 1994). This is a nominalgroup whose function is termed by Halliday (1994: 96) as 'Absolute' in that it'could be either Subject or Complement in an agnate major clause'. Indeedall the linguistic components except the caption in Figure 17.3 are'[ujnattached nominals' (ibid.: 395) which function in this way. But suchnominal groups are nonetheless far from being free from any Interpersonalmeaning. As for this title, the nominal group presents the Process of a celldividing as a Thing, which is objective, absolute, visible and concrete. Such ahigh level of certainty about the state of affairs is attainable through nominalgroups or grammatical metaphor in the form of nominalization (Halliday,

PRINT MEDIA 207

1993, 1998). In other words, distillation of phenomena into entity or trans-formation of clausal grammar to nominalized form means that the reader isnot in a position to doubt the existence of a phenomenon, but is led tobelieve in its absolute, timeless and unconditional existence.

Ideationally, being a nominal group, the title serves to identify, and is thusequivalent to an intensive identifying clause (Halliday, 1994: 119-120), forexample 'This is the drawing of the M phase of the cell cycle'. It is import-ant to note that the nominal group identifies not only through language butalso by its spatial proximity to the schematic drawing. By itself this nominalgroup points to a nominalized process, the M phase of the cell cycle. Thus asequence of dramatic events, where one cell splits into two, has been trans-formed into a thing which has consequently been deprived of all the originalvigour, liveliness and particularities.

Step C: mitosis

This step can be broken into three sub-stages: Step C-l the word'MITOSIS', Step C-2 the arrow and Step C-3 the circle and the two over-lapping circles which contain the semiotic depiction of the cell.

Step C-2: the arrowInterpersonally, the single-headed arrow is a Command; it demands that thereader look in the direction of the arrow, in this case, from top to bottom ofthe page. Here, the Command effect is strengthened by the particular dark-ness and thickness of the arrow.

Ideationally, the arrow serves to signify the process and direction ofmovement, change or progression, or the numerous intermediate phasesbetween the circle above and the circles below. In terms of Peirce's (1985:9—12) trichotomy of signs into an index, an icon or a symbol,2 the arrow is ahighly stylized icon. That is, the arrow proper does not exist in the actualworld in the process of cell division; the designers have added it to the depic-tion. Besides, the direction of the arrow in the physical sense, that is, fromtop to bottom, is iconic of progression in time.

Step C-3: the circlesInside the circle (second from top), highlighting devices such as the Colour-ing of the two pairs of lines and pink Shading in the original text serve todraw attention to the essential defining features of a cell at this stage. Theblank space (Omission) between the outer ring of the circle and the pink-shaded central area is, in reality, just as occupied as other parts of the cell.This distortion functions as yet another means of highlighting the two pairsof lines. The outermost black circle and the adjacent blank space inward(Omission) surround the central pink-shaded area, serving as a Framing togive weight to what is highlighted in the centre. Although not evident here inthe black and white reproduction, the Contrast of colour between black, red,pink and white serves the same highlighting purpose.


Ideationally, the circle is drawn to represent a snapshot of a particularstage in cell division. It focuses on the separation of the two pairs ofchromosomes, omitting the changes taking place in the cytoplasm. TheIdeational meaning is realized by the changes in shapes and contents of thepink-shaded area and also by the Diagonal orientations of the two pairs oflines representing chromosomes. We need note that this circle is not anobvious icon (Peirce, 1985). The two pairs of lines inside and the circularshapes do somewhat look like some types of cell components, hence they areiconic. But the colours, the circle and the blank space testify to the symbolicnature of the iconic sign. For instance, the colour of a particular cell com-ponent one sees in a micrograph is the result of dyeing technique. However,what is shown in the micrograph is not necessarily reproduced in a sche-matic drawing; in a drawing further treatment is carried out to producewhat appears in the final printed book. In other words, what meets areader's eye in a schematic drawing is at least two steps away from what isreally there: in terms of choice of colour and diagrammatic transformation.

Compositionally, several devices contribute to the organization of thetext. For instance, Colour Cohesion and Contrast enable the viewer to rec-ognize similarity and difference in the Ideational meaning and Inter-personal meaning: in the colour reproduction appearing in the textbook, thecolours red, pink, black and white serve as a backdrop against which theIdeational and Interpersonal meanings are expressed. Similarly, the shapesof the components, that is, the lines, circles, and the Relative Position ofthe components also constitute a resource to organize the text. Below Idiscuss in greater detail the role of Horizontals, Verticals and Diagonals inthe Textual organization in the schematic drawing.

The two pairs of lines in the first circle in Step C are positioned diag-onally relative to the vertical-horizontal frame of the drawing. In the ori-ginal text, the pair to the right are coloured red and the pair to the left areblack. The red pair resembles the contour of a hill or sea wave, each ofwhich is perceived as the trace of drastic movement or thrust resulting fromthe physical or geographical forces such as the gravitational pull. The axis ofthe black pair is approximately 30° anticlockwise to the vertical axis of thedrawing. This tilt or obliqueness creates 'directed tension' (Arnheim, 1974:424-428), or 'energy and dynamism' (O'Toole, 1994: 23; Thibault, 1997:315-322). We may note that whereas the shape of the red pair of linesremains roughly constant throughout the drawing, the black pair tilts mostin Step C. This well fits the Ideational theme of the step, which is concernedwith drastic change in terms of chromosomes in the nucleus. On the otherhand, the Diagonal orientations of the two pairs of lines in the step alsoserve to connect this step with the preceding and following steps, thus con-tributing to the Textual organization or unity of the drawing. In other words,obliqueness in orientation of the lines is echoed or shared by all the steps inthe drawing albeit to varying degrees. It is true that in the laboratory cellbiologists will know that the cells are undergoing some transformation how-ever they are aligned relative to the mechanical stage of the microscope. But

PRINT MEDIA 209

when cells are represented in micrographs and in particular in schematic draw-ings, that is, when they are turned into lines, circles and so forth, to contrib-ute to the Textual organization, 'the canons of classical painting' (Bastide,1990: 199-200) are often respected. One such canon is what hasbeen discussed in this paragraph, that is, the deployment of oblique linesto represent 'energy and dynamism' (O'Toole, 1994: 23; Arnheim, 1974:424-428).

Step E: the caption

The black and white caption has less visual salience through the smallerfont size, normal type (i.e. not bold face) and shorter leading. This sug-gests that the caption is to be read later in the reading sequence. Spacelimitations preclude a detailed analysis of the lexicogrammatical featuresof the caption. It is worth noting, however, that Ideationally the captionpresents a possessive identifying relation and circumstantial identifyingrelation, realized respectively by the verbal groups 'consists of and 'fol-lowed by'. This repeats the information presented in the main text (forexample in the clause 'These two processes together constitute the Mphase of the cell cycle'). The caption, however, serves in particular tospecify what the square bracket in Step B refers to, that is, a visual iconicexpression of a possessive identifying relation. Here we can appreciatethat while the visual images are important in biological texts they have tobe given categorical meanings by linguistic resources. The value of thevisuals in this figure is that, in addition to representing or constructing theshapes of biological entities, they are a spatialization or icon of the tem-poral flow of events and also aid to construct a taxonomy of biologicalterms (the relations between M phase, mitosis and cytokinesis). However,language has to specify the relations and their visual transformation(Barthes, 1977: 38-41).

Step F: chromosome replication

This step can be broken into two sub-stages: Step F-l the words 'chromo-some replication' and Step F-2 the arrow.

Step F-2: the arrowCompared to the others, this arrow is short, indicating less Prominence inthe figure. This arrow also leads the reader's attention to the next visualrepresentation. Ideationally, this arrow denotes the process by which onepair of chromosomes duplicates into two pairs. One needs to note, however,that the shortness of this arrow misrepresents the length of the time period.That is, replication in the S phase takes much longer than the M phase. Atypical eucaryotic cell spends a fraction of its cell cycle time in the M phase,and most of it in interphase, as noted in ECB, p. 549. For example, a


mammalian cell of a 24-hour cell cycle requires only about one hour for theM phase to complete. This misrepresentation of the temporal dimensionfunctions to highlight the M phase of the cell cycle.

Step G: the structure of the cell

Step G is located at the top of the figure and an uninitiated reader maybegin viewing the figure here as this step provides the background forwhat follows. The labels in this step and the leaders functioning as theidentifying processes disappear in the later depictions. This means thatonce they have fulfilled their contextualizing function, they are discardedand are no longer made visible. Having previously established the struc-ture of the cell in ECB, the reader is now invited to study in detail the Mphase. As argued above, the experienced reader reads Step A first and thisstep last or simply skips this step, as would perhaps a lecturer in theclassroom. This step can be read in two sub-stages: G-l the circle andG-2 the labels.

Step G-2: the labelsLike 'chromosome replication' in Step F, the words in Step G-2 are madeleast prominent by means of smaller font size, no Shading and no Capitaliza-tion. The leaders are also made insignificant by means of Length and Weight.Ideationally, they identify the major components of a cell, as if saying, forexample, 'This is the nucleus of the cell'.

Having explored the systems and functions that a schematic drawing in acell biology textbook has created and drawn upon to make meaning, I nowundertake a partial analysis of a visual image of a different kind, a statisticalgraph, before discussing the implications of this research.

Analysis of a statistical graph: Figure Q17.1

Figure Q17.1 (ECB, p. 550) is a statistical graph which appears in Question17.1. Here the students are expected to solve the problem by reference toinformation from the main text and the verbal section of the question andthe graph. The Question including the graph is reproduced in Figure 8.2(see Note 1). The discussion below briefly deals with the recording path,the Ideational meaning of the graph and how the graph contributes to theproblem-solving required to answer the question.3

The expected reading path for this multimodal composite involves a shut-ding between the verbal and the visual: from the main text to the Question(including the graph), then to the 'problem' part of the Question and rele-vant main text, and finally to the graph again. Within the graph, afterlocating the orientations of the graph and identifying what the horizontal x-axis and the vertical jy-axis refer to, the reader would survey the green-shaded curve which is supposed to carry the New. At this stage the readermay have to mark the graph to solve the problem.

PRINT MEDIA 211

Question 17.1 Cells from a growing population were stained witha dye that becomes fluorescent when it binds to DNA, so that theamount of fluorescence is directly proportional to the amount of DNAin each cell. To measure the amount of DNA in each cell, the cells werethen passed through a fluorescence-activated cell sorter (FAGS), aninstrument that registers the level of fluorescence in individual cells.The number of cells with a given DNA content were plotted on a graph,as shown in Figure Q17.1. Indicate on the graph where you wouldexpect to find cells that are in the following stages: Gb S, G2 and mitosis.Which is the longest phase of the cell cycle in this populationof cells?

Figure Ql7.1

Figure 8.2 Reproduction of Figure Ql 7.1

Ideationally, at the rank of Graph, the graph shows visually the Result (orpart of the Result) of an experiment, the frequency distribution of cells withdifferent DNA contents in a population of growing cells. The x values referto the DNA content per cell, as the label indicates, and the y values thenumber of cells with a given DNA content. In other words, cells in thepopulation are divided into various types according to the amount of DNAthe cell contains: the type of cells on the right of the .x-axis contains moreDNA than a type on the left. The value in thejy-axis records the number orfrequency of occurrence of each type of cells in the population. A higherpoint on the graph means that the number of cells of a particular type isgreater. Thus Ideationally the graph is a visual equivalent to a group oflinguistic relational processes through its Curvature. In addition, this graphshows the 'conceptual relations, and not actual data' (Lemke, 1998: 102).For instance, we are not told how many cells there are in the population, theexact number of cells with different DNA contents, nor how much DNAeach cell contains, as there is no indication of the unit of measurement oneither x- oiy- axis. We are provided with the theoretical relation between the


two variables: the type of cell defined by its DNA content and its frequencyof occurrence in the population.

It is worth noting that this Ideational meaning resides uniquely in a graphand that it cannot be expressed as effectively by a verbal text or a mathemat-ical equation. For Figure Q17.1 visually expresses the general abstractpattern, or spatializes the quantitative relationship. It is a document withvisual impact, one that enables the viewer or reader to 'take in' the patternat a glance. However well a verbal clause or clause complex or a mathematicalequation may express the trend or relationship, a graph always does so witha strong visual impact.

I would also like to note that just as the move from concrete data record-ing to the abstract relationship between the values of two variables mayinvolve grammatical metaphor (Halliday, 1998), the visualization of theabstract relationships may involve semiotic metaphor as formulated byO'Halloran (1999a, 2003, forthcoming). By semiotic metaphor, O'Halloran(2003: 357) refers to the phenomenon in which 'when a functional elementis reconstrued using another semiotic code' there may occur 'a shift in thefunction and the grammatical class of [the] element, or the introduction ofnew functional elements'. The formulation of semiotic metaphors involvedin the movements between natural language, mathematical symbolisms andvisual displays is crucial for the ultimate solution to mathematical problems,as demonstrated by O'Halloran (1999a, 1999b, 2003, forthcoming).

Here I analyse the movements between the verbal text and the visual textin Question 17.1, which involves instances of semiotic metaphor. 'Thenumber of cells with a given DNA content' in the verbal section of Ques-tion 17.1 functions as one participant, the Goal, with 'The number of cells'as the head and 'with a given DNA content' as the embedded Postmodifier(Halliday, 1994: 191-192). Experientially the 'cells' functions as the Thingand 'with a given DNA content' the Qualifier. But the elements 'The num-ber of cells' and 'with a given DNA content' do not mean only withinlanguage; they are also to mean mfe^emiotically, that is, in relation to thevisual text. In other words, the Head and Postmodifier composite in thelinguistic text is transformed into two separate participants in the visualtext, the two variables represented by the j-axis and thejy-axis perpendicu-lar to each other. This shift from one linguistic participant to two visualparticipants of equal status may be considered an example of 'parallelsemiotic metaphor' (O'Halloran, 1999a: 348) in that the two participants inthe second semiotic derive from the Goal in the first. This movement fromthe linguistic to the visual code permits, however, the exploitation of themeaning potential of the visual semiotic. Once this shift has taken place, itis possible to represent the relationship between the number of cells and theamount of DNA content per cell in terms of the height of the points orlines in the coordinate system and to make visual comparisons and evenhypothesize some mathematical relationship between the two variables.4 Theprecise shape of the curve in the visual text did not exist in the linguistictext and thus may be considered as a case of 'divergent semiotic metaphor'

PRINT MEDIA 213

(ibid.} because a new participant is introduced with the movement from thelanguage to the visual image. In this case the divergent semiotic metaphor(the curve) occurs as a consequence of the parallel semiotic metaphor (theintroduction of two participants). As will be clear shortly, the solution of theproblem depends to a large extent on how much sense the student canmake of the two instances of semiotic metaphor together with the informa-tion contained in the main text. In what follows I discuss two questions: (a)how do the Question and the graph relate to the main text? and (b) how dothe main text and Question (including the graph) contribute to the solutionof the problem?

The relationship between the question, the graph and the main textThe relevant main text reads:

During S phase (S = synthesis), the cell replicates its nuclear DNA, [. . .] S phaseis flanked by two phases where the cell continues to grow. The G1 phase (G =Gap) is the interval between the completion of M phase and the beginning of Sphase (DNA synthesis). The G2 phase is the interval between the end of S phaseand the beginning of M phase. (Figure 17.4).

(ECB, p. 550)

This means that if a cell in Gl phase has 2n units of DNA content, then bythe end of S phase ('replicates its nuclear DNA'), it has doubled the amountof nuclear DNA content and in the G2 and M phases, it has 4n units ofDNA content. That is, the amount of DNA per cell in G2 and mitosis istwice the amount in Gl and S phase is in the transition from 2n to 4/z units.Then how do Question 17.1 and the graph relate to such informationcontained in the main text? The main text reveals the general facts, the'laws' in biology, the conclusion, and/or the theory, which scientists arrive atfrom numerous experiments (as can be seen in the use of simple presenttense in the quotation above). Question 17.1 (including the graph), on theother hand, reports just one experiment, complete with Method and Resultsof an experimental report (the verb tense in some of the first few clauses inthe Question is the simple past, for example, 'were stained' and 'were thenpassed'). That is, the main text presents the conclusion and the Questionpresents one of the experiments leading to such a general conclusion.

Question 17.1 is not, however, a real experimental report, but rather it is atextbook question. In a real experimental report, the conclusion is presentedin the final part while in the textbook question the conclusion is the point ofdeparture and the student is expected to apply this general rule to solve apractical problem.

The contribution of the main text, question text and the graph in solvingthe problemThere are two parts to the Question. The first part reads: 'Indicate on thegraph where you would expect to find cells that are in the following stages:Gl5 S, G2, and mitosis'. To answer this question, the student must understand


the change in the amount of DNA content at different stages of the cellcycle. That is, he or she must understand the relevant part of the main textquoted above. Then in relation to the question he or she must also knowhow to interpret the x-axis and know that at point b, the point on the x-axiscorresponding to Peak B (which is not displayed in Figure 8.2) the amountof DNA per cell is twice that at point a, the point on the x-axis correspond-ing to Peak A (again not displayed in Figure 8.2), and that Peak B is thereforethe place where one would expect to find cells in G2 and mitosis phases(chromosomes replicated, doubled) and Peak A the place to find cells in Gl

phase (chromosomes not yet replicated). Here the ability to deduce b = 2aon the x-axis is crucial to the solution of the problem. To know where to findthe cells that are in the S phase, the student must again understand therelevant main text. He or she must also be able to translate such main textinformation into the line segment ab on the x-axis and know that cells thatare in the S phase can be found between Peaks A and B.

The second sub-question reads: 'Which is the longest phase of the cellcycle in this population of cells?'. To answer this question, the student needsto interpret the divergent semiotic metaphor, that is, he or she needs to knowhow to interpret the frequency graph. That is, Peak A is the highest, indicat-ing that the number of cells with this DNA content, that is, cells at Gl phase,is the largest. This further suggests that Gl is the longest phase of the cellcycle, assuming that the cells were selected on a random basis.

In this subsection I have discussed the essential role that a knowledge ofthe linguistic and visual resources and how they interact with each otherplays in the solution of an in-text problem in ECB. In the final section of thepaper I discuss the implications of the preceding analyses.

Multimodal meaning-making: some concluding remarks

This paper has proposed tentative frameworks for the analysis of two typesof visual displays common in biology texts and has attempted to apply themto the analysis of multimodal meaning-making in the biology text. As maybe clear from the above discussion, the visual images in the biology text arenot redundant with language in meaning-making; they extend and com-plement it. The words, on the other hand, specialize in a range of typo-logical meanings and certain Interpersonal and Textual meanings and thus'anchor' and constrain the many possible meanings made in the visual(Barthes, 1977: 38-41). One is dependent upon and co-contextualizes theother (Thibault, 2000: 312). To understand the text, as in Figure 17.3 inECB, or solve a problem, as in Question 17.1, the reader must be able tointegrate the meanings made in the linguistic and the visual codes.

My analysis has also shown that each type of visual display carries with itdifferent sets of conventions of meaning-making, not only in the deploy-ment and interpretation of combinations of ink or paint (dots, lines, curves,etc.) but also in their relations to the verbal text. For example, a schematicdrawing, such as Figure 17.3 analysed above, spatializes the ideational

PRINT MEDIA 215

meanings made in the verbal text, while a statistical graph, such asFigure Q17.1, transforms a set of quantitative data into a visually per-ceptible object. Although many aspects of the interstratal relationshipbetween the visual signifiers and their signifieds remain to be explored(Thibault, 1997: 329-334), my analysis in the paper has shown that thevisual displays in disciplinary discourses as exemplified in the biology textare important for meaning-making and that what they mean and how theymean it are not always self-evident or universal.

What does all this mean for the teaching and researching in ESP/EAP?Research in these areas, both for native speakers of English and for non-nativespeakers, has almost exclusively concentrated on language issues (see, forexample, Swales's (2001) review of the developments of ESP/EAP in thepast forty years), assuming that once the learner crosses the language barrier,he or she will achieve academic success. Language, of course, constitutesour major means of meaning-making and may continue to be one of theproblems that hinder one's progress through his or her career. But as I haveshown in this paper, following Myers (1990, 1995) and Thibault (2001), inbiology textbook genres language is only one resource for making certainkinds of meaning. It is simply not able to make certain topological meaningsrequired in certain contexts and it means what it does mean in the first placeonly in co-deployment and co-contextualizations with other resources(Thibault, 2000: 312, 362). In professional scientific practice, as well-attestedby Lynch and Woolgar (1990) and Lemke (1998), 'as the fine edge and the finalstage' of some laboratory research, the 'tiny set of figures' drawn on thepaper rather than the '[bjleeding and screaming rats' in the lab 'is all thatcounts' (Latour, 1990: 39-40; emphasis in original). And the grant-proposalsin engineering must be written and designed in a way that enables the peerreviewers 'to find the abstract, [mathematical] formulas, tables, illustrations,and references with ease' (Johns, 1993: 82). Thanks to the pioneering workof Kress et al. (2001), Lemke (2000), O'Halloran (1996, 2000), Scott andJewitt (2003) and Johns (1998) we have been able to see that in scienceclassrooms 'learning can no longer be treated as a process which depends onlanguage centrally, or even dominantly [. . .] Learning happens through (or[. . .] learners actively engage with) all modes as a complex activity in whichspeech or writing [are] involved among a number of modes' (Kress et al.,2001: 1). Therefore, we ESP/EAP teachers and researchers need to takeseriously the multimodal nature of meaning-making in academicapprenticeship and professional life and refocus our research and teachingagenda so as to better prepare our students for their current and futureacademic and professional life. We need, for example, to complete moreresearch into the nature of the interactions between the verbal and thevisual in various genres and in various disciplines rather than assuming auniversal model. This is particularly important when the learner of ESP/EAP is a university student from a non-English-speaking background, wherethe visual images need to be related to the verbal resources in English. ESP/EAP teachers will be expected to 'give students a visual grammar that


supplements the grammar of English' (Baldry, 2000b: 53) and organizepractical classroom activities that are geared towards the development ofthe 'multimodal communicative competence' (Royce, 2002: 192). For wantof a thorough and systemic description of the 'full system of relations' thatseveral semiotic codes simultaneously enter into (Thibault, 2000: 362), wemay introduce our students to some basic multimodal analytical tools andprinciples and then encourage them to reflect on the intersemiosis in specificinstances in their disciplines. Following Baldry (2000b) we may guide ourstudents to compare the contemporary multi-semiotic meaning-makingwith that of the past. This guided reflection may benefit the students aswell as ESP/EAP teachers. In many circumstances it is also desirable forthe ESP/EAP teachers to consult the expert staff about the intersemioticmeaning-making in their teaching and professional research (Johns, 1998:193). It is encouraging to note that since the mid-1990s Baldry (2000b) andPavesi and Baldry (2000) have taken significant steps to design and offermultimodal ESP/EAP courses to both complete beginners and moreadvanced students. This includes the development of multimedia environ-ment self-access courseware and corpora. Finally, I also suggest that themultimodal construction of meaning should be reflected in ESP/EAPassessment, although this is largely absent in many parts of the world. Withthe new pair of spectacles called multimodal social semiotics, the nature andcomplexity of scientific discourse and how they might be more effectivelytaught to ESP/EAP students may be further explored.

Notes

1 Unfortunately, it has not been possible to reproduce these Figures in colour as theyappear in the textbook. However, the following glosses are provided on thecolours used in the original text.Figure 17—3, reproduced in Figure 8.1The words 'Figure 17—3' are green. The nucleus of the cell is shaded in pink. Inthe top circle the chromosome on the left is black, the one on the right red, andthis scheme is retained throughout. 'MITOSIS' and 'CYTOKINESIS' areshaded in bright yellow, and 'M phase' light green.Figure Q17—1, reproduced in Figure 8.2The Question is framed by a box which is marked by its yellow background. In asimilar manner, the actual graph is framed inside the yellow box by a whitebackground. The curve, the A and B, and the area underneath are green.

2 On the basis of how a signifier relates to the signified, Peirce (1985) classifiessigns into an icon, an index and a symbol. In simple terms, an icon is a sign thatrelates to its object in terms of their resemblance. This resemblance can besimilarity in 'simple qualities', as in images or photographs, or in 'relations', as indiagrams and algebraic formulae, or it can be 'a parallelism' as in metaphors(Peirce 1985: 10-11). An index is 'a sign which refers to the Object that itdenotes by virtue of being really affected by that Object' (ibid.: 8), for example,smoke as an indication of fire. A symbol is a sign that derives its meaning byconventions, by agreement between people (ibid.}, for example, the phonological

PRINT MEDIA 217

or graphological feature of the word 'man' and its meaning. The reason forapplying Peirce's trichotomy to the present analysis is that it brings to light thefact that the relationship between the signified and the signifier is not alwaysidentical or straightforward. Thus signs vary in the degree of the potentialsemiotic load they pose for students. An iconic photograph of some familiarobject is easy to decipher, less so the schematic drawing, and even less so thesymbolic signs such as 5'-UGC-3'.For lack of space I do not analyse the graph in a step-by-step manner, as I didwith Figure 17.3 above. However, one can always explore the visual resourcesthe graph exploits by reference to Table 8.2.According to Tilling (1975: 200-211), quantitative graphs were not only used byscientists such asj. H. Lambert (1728—1777) to present experimental data graph-ically but also help to analyse them, for example derive mathematical relation-ship between the variables (e.g. the rate of water evaporation as a function oftemperature as reported in one of Lambert's papers (Tilling, 1975: 201)).Apparently, the student reader in this question is not required to derive anequation from the graph but just to interpret the results displayed in the graphand draw some conclusions.

Acknowledgements

Figures 8.1 and 8.2: Copyright © 1998 from Essential Cell Biology: An Introductionto the Molecular Biology of the Cell by Alberts, B., Bray, D., Johnson, A., Lewis,J., Raff, M., Roberts, K. and Walter, P. Reproduced by permission ofRoutledge/Taylor & Francis Books, Inc.

References

Alberts, B., Bray, D., Johnson, A., Lewis, J, Raff, M., Roberts, K. and Walter,P. (1998) Essential Cell Biology: An Introduction to the Molecular Biology of the Cell. NewYork: Garland.

Arnheim, R. (1974) Art and Visual Perception: A Psychology of the Creative Eye. Berkeley:University of California Press.

Baigrie, B. S. (ed.) (1996) Picturing Knowledge: Historical and Philosophical ProblemsConcerning the Use of Art in Science. Toronto: University of Toronto Press.

Baldry, A. P. (ed.) (2000a) Multimodality and Multimediality in the Distance Learning Age.Gampobasso, Italy: Palladino Editore.

Baldry, A. P. (2000b) English in a visual society: comparative and historical dimen-sions in multimodality and multimediality. In Baldry (ed.), 2000a: 41-89.

Barthes, R. (1977) Rhetoric of the image. In R. Bardies (S. Heath, ed. and trans.),Image—Music—Text. New York: Hill and Wang, 32—51. (Originally published in1964.)

Bastide, F. (1990) The iconography of scientific texts: principles of analysis. InLynch and Woolgar (eds), 187-229.

Ford, B. J. (1992) Images of Science: A History of Scientific Illustration. New York: OxfordUniversity Press.

Haas, G. (1994) Learning to read biology: one student's rhetorical development incollege. Written Communication 11(1): 43-84.

3

4


Halliday, M. A. K. (1978) Language as Social Semiotic: The Social Interpretation of Languageand Meaning. London: Edward Arnold.

Halliday, M. A. K. (1993) On the language of physical science. In M. A. K. Hallidayandj. R. Martin, Writing Science: Literacy and Discursive Power. London: The FalmerPress, 54-68.


Halliday, M. A. K. (1998) Things and relations: regrammaticizing experience astechnical knowledge. In J. R. Martin, and R. Veel (eds), Reading Science: Critical andFunctional Perspectives on Discourses of Science. London: Routledge, 185—235.

Janovy,J.,Jr. (1996) On Becoming a Biologist. Lincoln: University of Nebraska Press.Johns, A. (1993) Written argumentation for real audiences: suggestions for teacher

research and classroom practice. TESOL Quarterly 27(1): 75—90.Johns, A. (1998) The visual and the verbal: a case study in macroeconomics. English

for Specific Purposes 17(2): 183-197.Knight, D. (1992) Ideas in Chemistry: A History of the Science. New Brunswick, NJ:

Rutgers University Press.Kress, G. (2000) Multimodality. In B. Cope and M. Kalantzis (eds), Multiliteracies:

Literacy Learning and the Design of Social Futures. South Yarra: Macmillan PublishersAustralia Pty Ltd, 182-202.

Kress, G. (2003) Literacy in the Mew Media Age. London: Routledge.Kress, G., Jewitt, G., Ogborn, J. and Tsatsarelis, C. (2001) Multimodal Teaching and

Learning: The Rhetorics of the Science Classroom. London: Continuum.Kress, G. and van Leeuwen, T. (1996) Reading Images: The Grammar of Visual Design.

London: Routledge.Kress, G. and van Leeuwen, T. (1998) Front pages: (the critical) analysis of news-

paper layout. In A. Bell and P. Garrett (eds), Approaches to Media Discourse. Oxford:Blackwell, 186-219.

Kuhn, T. S. (1996) The Structure of Scientific Revolutions (3rd edn). Chicago: Universityof Chicago Press.

Latour, B. (1990) Drawing things together. In Lynch and Woolgar (eds), 19—68.Lemke, J. L. (1996) Hypermedia and higher education. In T. M. Harrison and

T. Stephen (eds), Computer Networking and Scholarly Communication in the Twenty-First-Century University. New York: State University of New York Press, 215—231.

Lemke, J. L. (1998) Multiplying meaning: visual and verbal semiotics in scientifictext. InJ. R. Martin and R. Veel (eds), Reading Science: Critical and Functional Perspec-tives on Discourses of Science. London: Routledge, 87—113.

Lemke, J. L. (2000) Multimedia literacy demands of the scientific curriculum.Linguistics and Education 10(3): 247-271.

Lynch, M. (1990) The externalized retina: selection and mathematization in thevisual documentation of objects in the life sciences. In Lynch and Woolgar (eds),153-186.

Lynch, M. and Woolgar, S. (eds) (1990) Representation in Scientific Practice. Cambridge,MA: The MIT Press.

McMurry, J. and Castellion, M. E. (1999) Fundamentals of General, Organic, andBiological Chemistry (3rd edn). Upper Saddle River, NJ: Prentice- Hall.

Myers, G. (1990) Every picture tells a story: illustrations in E. O. Wilson's Sociobiol-ogy. In Lynch and Woolgar (eds), 231-265.

Myers, G. (1992) Textbooks and the sociology of scientific knowledge. English forSpecific Purposes 11 (1): 3-17.

PRINT MEDIA 219

Myers, G. (1995) Words and pictures in a biology textbook. In T. Miller (ed.),Functional Approaches to Written Text: Classroom Applications Vol. I, The Journal ofTESOL France, Paris, in association with US Information Service, Paris, 113—126.

O'Halloran, K. L. (1996) The discourses of secondary school mathematics.Unpublished Ph.D. thesis. Murdoch University, Western Australia.

O'Halloran, K. L. (1999a) Interdependence, interaction and metaphor in multi-semiotic texts. Social Semiotics 9(3): 317—354.

O'Halloran, K. L. (1999b) Towards a systemic-functional analysis of multi-semioticmathematics texts. Semiotica 124(1/2): 1-29.

O'Halloran, K. L. (2000) Classroom discourse in mathematics: a multi-semioticanalysis. Linguistics and Education 10(3): 359—388.

O'Halloran, K. L. (2003) Intersemiosis in mathematics and science: grammaticalmetaphor and semiotic metaphor. In A.-M. Simon-Vandenbergen, M. Taverni-ers, and L. Ravelli (eds), Grammatical Metaphor: Views from Systemic Functional Lin-guistics. Amsterdam: John Benjamins, 337—365.

O'Halloran, K. L. (forthcoming) Mathematical Discourse: Language, Symbolism and VisualImages. London: Continuum.

Osborne, J. (2002) Science without literacy: a ship without a sail? Cambridge Journal ofEducation 32(2): 203-218.

O'Toole, M. (1994) The Language of Displayed Art. London: Leicester University Press.Pavesi, M. and Baldry, A. P. (2000) Learning to read scientific texts: integrated self-

access courseware and corpora for university science students. In Baldry (ed.),2000a: 227-245.

Peirce, C. S. (1985) Logic as semiotic: the theory of signs. In R. E. Innis (ed.),Semiotics: An Introductory Anthology. London: Hutchinson, 4—23.

Purves, W. K. (1999) Biology. In The Encyclopedia Americana (international edition)Vol. 3. Danbury, CT: Grolier, 769-778.

Royce, T. (2002) Multimodality in the TESOL classroom: exploring visual-verbalsynergy. TESOL Quarterly 36(2): 191-205.

Scott, P. and Jewitt, C. (2003) Talk, action and visual communication in teachingand learning science. School Science Review 84(308): 117-124.

Swales, J. (2001) EAP-related linguistic research: an intellectual history. In J. Flow-erdew and M. Peacock (eds), Research Perspectives on English for Academic Purposes.Cambridge: Cambridge University Press, 42—54.

Thibault, P. J. (1997) Re-reading Saussure: The Dynamics of Signs in Social Life. London:Routledge.

Thibault, P. J. (2000) The multimodal transcription of a television advertisement:theory and practice. In Baldry (ed.), 2000a: 311-385.

Thibault, P. J. (2001) Multimodality and the school science textbook. In C. Torsello,G. Brunetti, and N. Penello (eds), Corpora Testuali per Ricerca, Tradu^ione eApprendimento Linguistico. Studi Linguistici Applicati. Padova: Unipress, 293—335.

Tilling, L. (1975) Early experimental graphs. The British Journal for the History ofScience 8(30): 193-213.

Zar, J. H. (1999) Biostatistical Analysis (4th edn). Upper Saddle River, NJ: Prentice-Hall.

9 Developing an integrative multi-semiotic model

Victor Lim FeiNational University of Singapore

Introduction

In this age of the multimedia, there is an increasing awareness that meaningis rarely made with language alone. As Baldry (2000), Kress (2003) andKress and van Leeuwen (2001) note, we live in a multimodal society whichmakes meaning through the co-deployment of a combination of semioticresources. Visual images, gestures and sounds often accompany the lin-guistic semiotic resource in semiosis. As such, there is a pressing need tounderstand the dynamics of meaning-making, or semiosis, in multimodaldiscourse. Academic disciplines that focus on mono-modality, such as thatof linguistics, must come into dialogue with other fields of research, forinstance, visual communication studies and media studies, to facilitate theinterdisciplinary nature of multimodal research.

In this paper, the Integrative Multi-Semiotic Model (IMM) (Lim, 2002) isproposed as a 'meta-model' for the analysis of a page or frame whichinvolves the use of both language and pictures as semiotic resources. Theterm 'meta-model' is used to describe the IMM as a model which bringstogether and incorporates the systemic-functional matrices and frameworkscurrendy available in the field of multimodal studies. This is undertakenwith the aim of unifying these contributions for the expression, content andcommunicative planes of language and visual images in the IMM. There isa need, however, to further develop the model into one that can account formeaning arising from other semiotic resources in dynamic environmentssuch as video texts and hypertext.

Systemic-functional linguistics (SFL), developed by Michael Halliday(1978, 1994) and extended by Martin (2002) and Martin and Rose (2003),provides the theory for this investigation into semiosis involving languageand visual images. Although originally conceived for the semiotic resourceof language, the application of SFL to other semiotic resources has beenproductive. Pioneering work in the application of systemic-functional the-ory to visual images, architecture and sculpture includes O'Toole's (1994)The Language of Displayed Art and Kress and van Leeuwen's (1996) ReadingImages. Following this, further applications of SFL to other semioticresources for the analysis of multimodal discourses in mathematics, science,

PRINT MEDIA 221

and three-dimensional museum displays have provided insights into thenature of intra-semiosis - meaning within different semiotic resources, andinter-semiosis - meaning across different semiotic resources (for example,Baldry, 2000; Baldry and Thibault, forthcoming; Pang, this volume; Lemke,2002; O'HaUoran, 1999a, 1999b, 2000, forthcoming; Royce, 1998a, 1998b,2002).

Of particular interest in this paper is the development of the theory ofthe interaction and integration between language and pictures in caseswhere these semiotic resources co-occur on a page as found, for example, inchildren's picture books and advertisements. ledema (2003: 30) refers tosuch intersemiotic shifts as 'resemioticization' which he defines as 'the ana-lytical means for . . . tracing how semiotics are translated from one intoanother as social processes unfold'. In this respect, of significance are Lemke's(1998) observation of the 'multiplication of meaning' which takes place inmultimodal texts and O'Halloran's (1999a, 1999b) identification of'semi-otic metaphor' which refers to the new 'semantic reconstruals' which occurintersemiotically with shifts between semiotic codes. Royce (1998b) also pro-poses an 'intersemiotic complementarity' which describes the deploymentof intersemiotic resources in a multimodal text. Further to this, Thibault(2000, forthcoming) uses phase theory to effectively conceptualize a frame-work to analyse the integration of language, visual images, sound and musicin television advertisements.

While the direct adoption of a linguistic theory for other semioticresources has been criticized (for example, Saint-Martin, 1990), Sonesson(1993: 343) cautions that 'the outright rejection of the linguistics model mustbe at least naive, and as epistemologically unsound as its unqualified accept-ance'. As such, a delicate balance between the adoption and rejection oflinguistics theories to visual analysis and intersemiotic processes must bemaintained. That is, theories and concepts used in linguistics may notbelong solely to the study of language and could be productive in theirapplications to other semiotic resources. For example, the systemic-functional theory and the tri-metafunctional organization of semioticresources, although originally applied to language, rest essentially on thebasic assumption of language as a social semiotic. Therefore, it is appropri-ate to interpret SFL as a semiotic theory rather than a particular theory oflanguage.

Proposing an IMM

Despite the advances made in recent research, there remains a lack ofunderstanding of how meanings arise in multimodal texts. Apart fromThibault's (2000; forthcoming) comprehensive framework for the analysisof television advertisements and Baldry and Thibault's (2001) conception ofphase in dynamic video texts, an overarching model and a meta-language todescribe the processes involved in semiosis and intersemiosis in multimodaltexts is lacking. As such, the IMM and the related concepts introduced in


this paper are proposed as a tentative step to account for the differentaspects of meaning arising from the use of multiple semiotic resources. TheIMM, which may be used for the analysis of a printed text involving the twosemiotic resources of language and visual images, is a modest step. None-theless, the necessity of developing a 'meta-model' with an accompanying'meta-language' to describe semiotic processes in multimodal discourse isdemonstrated through the discussion of the IMM and the issues raised bysuch a model.

The IMM, as displayed in Figure 9.1, demonstrates topologically thecomplex multifaceted nature of meaning made in a multi-semiotic text.The rectangular blocks are used metaphorically to represent the strata,planes and dimensions of meaning within and across language and visualimages. Following Martin (1992), three planes are conceptualized for thesetwo semiotic resources. That is, the language and visual image planeconsists of an Expression plane and a Content plane (which is furtherdivided into grammar and discourse semantics strata), and the Contextplane which consists of register, genre and ideology as displayed inFigure 9.1.

The top view of the model appropriately displays the Expression planewhich is referred to as 'Typography' for language and 'Graphics' for visualimages. This is significant as the Expression plane is the interface betweenthe text and the reader. As seen in Figure 9.1, this interface is mediated bythe medium and materiality of the text, which also mediates the otherplanes. This mediation may be seen in operation in the simple case of awedding invitation card which is usually printed on certain types of paper.This demonstrates that the Content, register and genre of the text (the wed-ding invitation) are related to the materiality options of the medium (the

Figure 9.1 The IMM (Lim, 2002: 37)

PRINT MEDIA 223

type of paper and print). Together, these choices carry ideological implica-tions, which in this case concern the elevated status of weddings in Westernsociety.

An elevated platform between the linguistic and pictorial modalities canbe seen from the top of the IMM. This is called the Space of Integration(Sol), which is the theoretical platform where intersemiosis occurs throughcontextualizing relations. The elevation of the Sol signifies topologically thesemantic expansions that result from the interaction and negotiationbetween semiotic resources in what Lemke (1998) terms as 'the multiplica-tion of meaning'. Below the Expression plane is the Content plane whichconsists of the lexicogrammatical and discourse semantics strata for lan-guage, and the visual grammar and discourse semantics strata for visualimages. As seen in Figure 9.1 the Sol also operates on the Content plane.

The lexicogrammatical and discourse systems for language are organizedaccording to the three metafunctions proposed by Halliday (1994); the idea-tional, Interpersonal and Textual metafunctions. The theory of metafunc-tionality has been extended to the systems which constitute the grammar ofother semiotic resources. For example, Kress and van Leeuwen (1996) andO'Toole (1994) extend the metafunctional hypothesis to the systems of avisual grammar. O'Toole (1994) proposes a detailed metafunctionally basedmatrix for the analysis of paintings. In addition to the lexicogrammaticaland grammatical systems, a discourse semantics stratum is also recognizedfor the pictorial modality as well as for the linguistic in the IMM. Althoughnot developed here, this extension follows from Martin's (1992) metafunc-tionally based discourse systems for language. The discourse semanticsstratum for language and visual images is useful for analysing children'spicturebooks, for example, which consist of a sequence of pictures and text(Lim, 2002).

The systems of meaning in the Expression and Content plane for lan-guage and visual images are seen to be organized metafunctionally in theIMM. The metafunctional distinctions within the systems on the grammarand discourse strata in the IMM are indicated through the three rectangularboxes of different Tone in Figure 9.1. Thibault (2000: 362) proposes that'metafunctions are best seen as a principle of integration for approaching theExperiential, Interpersonal, logical and Textual dimension of the text as awhole'. The commonalities of metafunctional organization across semioticresources are drawn upon and metafunctional distinction is used as a meansof conceptualizing meaning across the different strata in the IMM.

The term system-metafunction fidelity is used to signify the degree ofdedication of a system towards a specific metafunction. Although meaningis organized around the metafunctional classifications, the system-metafunction fidelity of the visual grammar is less rigid compared to thelexicogrammar in language. In other words, the metafunctional categoriesby which the systems for visual images on the grammar stratum are organizedmay be more fluid than depicted by the three rectangles in Figure 9.1. Forexample, the system of Rhythm in the grammar for visual images (O'Toole,


1994) may be oriented towards Interpersonal meaning (to capture atten-tion), Textual meaning (to cohesively link parts of a text) or Experientialmeaning (to indicate an action) in different instantiations of the system intext. The orientation of the system towards one metafunction rather thananother depends upon the surrounding co-text in the visual image.

The problem of system-metafunction fidelity is also relevant to the sys-tems which operate on the Expression plane. That is, the Expression planefor language and visual images, referred to as Typography and Graphicsrespectively, is also seen to be organized metafunctionally. However, themajor systems on this plane are not always dedicated primarily to a singlemetafunction. The system-metafunction fidelity is even lower than notedabove for the grammar of visual images. Although it is possible to dis-tinguish meanings as being ideational, Interpersonal or Textual, the systemson the Expression plane which are responsible for these meanings overlapwith regards to their metafunctional capabilities. For instance, the system ofColour can realize ideational, Interpersonal, and Textual meanings as notedby Kress and van Leeuwen (2002). The tendency of the instantiation of asystem to be orientated towards a particular metafunction is discussed inmore detail in relation to the notion of Critical Impetus (see below). However,at this stage we may note that a cline, rather than a categorical tri-metafunctional distinction, is used in Figure 9.1 to show the fluidity of thesystems which operate on the Expression plane. This cline is representedby graduation in the system of Tone in Figure 9.1.

Kress and van Leeuwen (2002) adopt an alternative approach to dealwith the metafunctional diversity of the systems which operate on theExpression plane by proposing that Colour, for example, is a semioticmodality in its own right. Thus, rather than positing colour as a systemwhich operates on the Expression plane, Kress and van Leeuwen (2002)attempt to locate Colour on the grammar stratum as a semiotic modalitywhich possesses its own grammatical systems - or rather scales - of mean-ing; for example, Saturation, Purity, Modulation and Hue. However, asadmitted by Kress and van Leeuwen, Colour differs from other semioticmodalities such as language and visual images in that it cannot exist on itsown: 'It can survive only in a multimodal environment' (Kress and vanLeeuwen, 2002: 351). In order to accommodate this limitation of Colour asa social semiotic, an alterative perspective is provided here. That is, Colouris conceived to be a system with a low system-metafunction fidelity on theExpression plane in the IMM. Systems such as Hue, Tone, Saturation andso forth, are seen as sub-systems of the system of Colour. To account for themetafunctional diversity of a system such as Colour, the notion of 'criticalimpetus' is developed.

The IMM rests entirely upon the Context planes of register, genre andideology. This is significant because meaning is located within the Contextof Situation and Context of Culture. Martin (1992) suggests that the soci-osemantic variables of Field, Tenor and Mode 'hook up' with the metafunc-tions on both the communication planes of Register and Genre. Another

PRINT MEDIA 225

layer, ideology, is also proposed by Martin (1992) to look at positions withindiscourse formations manifested across a range of texts. Meanings made onthis intertextual level are also heteroglossic in nature according to differentreading positions and subjectivities.

The IMM aims to provide the apparatus for the analysis of a text whichutilizes both the linguistic and the pictorial semiotic resources. Using theIMM as an approach also allows for a systematic evaluation of the meaningmade on various strata and planes and, at the same time, provides a plat-form for understanding the interaction between modalities and examiningthe occurrence of semantic expansion during intersemiosis. In what follows,two particular dimensions of the IMM which can contribute to a deeperunderstanding of the dynamics of intersemiosis are explored: the systemswhich operate on the expression plane, and the Space of Integration (Sol).

The scope of this paper

There is only space to investigate two issues raised by the IMM. The firstaspect is concerned with the systems such as Colour which operate on theExpression plane of language and visual images. This is an undertheorizedarea of research as the focus of interest has tended to be the Content planewhich consists of the grammar and discourse systems. The second aspect isconcerned with the interaction and negotiation between the two semioticmodalities on the Sol in a multimodal environment. An understanding ofthe intersemiotic processes which take place in the Sol is critical for anunderstanding of how meaning is made in a multimodal environment.

In the first case, the Expression plane of language, or the 'Typography'of printed texts, has often been neglected in linguistic theory. Likewise, theExpression plane of the pictorial semiotic, referred to here as 'Graphics',has also been undertheorized. Responding to this need, system networks areproposed to account for the Typographic and Graphic selections made fromwithin the linguistic and pictorial semiotics respectively (see Figures 9.4 and9.5). The system networks, still very much at an exploratory stage, areconceived in the tradition of SFL and thus are seen to be organizedmetafunctionally. The proposed networks represent a deliberate effort togive recognition to the role of the Expression plane in contributing to thefunctions and meaning of discourse which is traditionally seen to be locatedwithin the realm of the Content plane.

In the second case, Gestalt theory in art has long observed the phenom-enon of the whole as always greater than the sum of its parts (Gombrich,1960). Likewise, in the interaction and integration between the linguisticand pictorial semiotic resources, the total meaning made is more than justadding up the meaning made by each independent modality. In otherwords, semantic expansion or a 'multiplication of meaning' (Lemke, 1998)occurs during this co-deployment. To account for this expansion of meaning,an Sol in the IMM is proposed so that the contextualizing relations betweentwo modalities can be studied. As explained in a following section, semantic


expansion can occur through the mechanisms of'Homospatiality' and 'Semi-otic Metaphor' (O'Halloran, 1999a, 1999b, 2003, forthcoming) in the Sol.Other intersemiotic mechanisms have also been proposed: semiotic cohesion,semiotic mixing, semiotic adoption, juxtaposition and semiotic transition(O'Halloran, forthcoming). However, in this discussion only homospatialityand semiotic metaphor are considered. Before discussing these contextualizingrelations, the Expression plane for language and visual images is examined.

The Expression and Content planes for language and visualimages

Halliday (1978: 39) proposes that language is 'a system of meaning poten-tial'. Seen to operate on the levels of the Content and Expression plane,meaning potential is conceived as a network of options where meaning ismade through paradigmatic selections from the available system networks.Language is an abstraction (the system network) until it is materialized orexpressed through either speech or writing (the process in the form of atext). When the linguistic semiotic is expressed through sound, the Expres-sion plane consists of Phonology. When language is materialized as writ-ing, the Expression plane is Graphology, or in the case of a printed textunder consideration here, Typography.

The visual image is similarly a tool for meaning construction. Thatis, the pictorial semiotic resource is also seen as a conceptual abstractionwith systems of meaning constituting the meaning potential. As shown inFigure 9.2, language is conceived to possess abstract lexicogrammatical sys-tems of meaning where choices are expressed on the Expression planethrough Typography in printed texts. In the same manner, the grammar ofvisual images is also abstraction which is instantiated through choices fromnetworks of systems (such as Form, Perspective, Layout and Strokes) on theGraphics Expression plane. The separation of the Expression and gram-mar strata for the pictorial semiotic may be perceived as an uneasy one dueto the interwoven nature of the elements on both strata in meaning-making.Nonetheless, it is useful and necessary to differentiate between the two stratain order to investigate the systems' potential and understand the meaning-making process. Figure 9.3 is used to discuss the theoretical distinctionbetween the Expression and visual grammar planes.

Figure 9.2 Instantiation of language and pictures

PRINT MEDIA 227

Figure 9.3 An iconic face

The Expression plane of the Figure involves, for example, the systemsof Colour and Form used to make meaning. This refers to choices in the formof the black thin line, the two small black circles and the larger circle inFigure 9.3. Should any of the choices be altered at the rank of the Expres-sion plane; for example, should the eyes become green, or the thin blackline becomes a red brushstroke, the meaning of the picture would change.The choices from systems in the Expression plane (see Figure 9.5) are signifi-cant in terms of the meaning of the picture. This illustrates that choicesmade from systems on the Expression plane contribute or feed through tothe meanings made through systems operating on the Content plane. Thispoint is further discussed below.

The grammar stratum, as extensively theorized by O'Toole (1994, 1995)and Kress and van Leeuwen (1996), relates one disparate element toanother and explains how the whole functions cohesively to make meaning.Just as the grammar of language concerns itself with the chains of words toform coherent sentences, the grammar of visual images is about the piecingof one item with another to construct a coherent message. The relations ofthe parts to a whole, for instance, how the various shapes form the iconicface in Figure 9.3, operate on the grammar stratum. This grammar is cul-turally dependent and governs the way a reader 'reads' and understandsimages such as the iconic face in Figure 9.3.

Following O'Toole (1994: 24), a hierarchy of different ranks analogous toHalliday's (1978) rank scale for language, is proposed for the visual gram-mar. In this way, it is possible to examine the meaning made on each of therank units, which are Member, Figure, Episode and Work. This adoption of arank scale operating within the principle of constituency, where one rank isconstitutive of the next higher rank in the hierarchy, facilitates a moresystematic analysis of the meaning made in the different units on the visualgrammar stratum.

In a sense, the delicate distinction between the Expression plane andgrammar stratum can be made with the Expression plane being largelyconcerned with the surface instantial features of the text and the Contentplane with the interaction and negotiation between the different elements inthe text. In the same way that Context mediates the meaning of a text, theExpression plane mediates the choices made from the grammatical anddiscourse systems operating on the Content plane. The notion here is one of'mutual engendering' which has been used to describe the relationship


between language and social Context (Martin, 1992). In this case, themutual engendering encompasses the Expression plane and the Contentplane, the materiality and medium of the text, and the social and culturalContext within which the text was produced.

Perceptual equity between language and visual images

Saint-Martin (1990) claims that pictures are primarily objects of visual per-ception and therefore are distinct from language in many ways. Whileacknowledging this, it is also recognized that the linguistic semiotic resourceinstantiated through the system of Typography is also a visual experience.With the adoption of this position, some of the assumptions based on thecommonalities between the two modalities are discussed before introducingthe unique systems through which each semiotic operates to make meaning.

Since both the linguistic and the pictorial semiotics are expressed throughthe visual medium on a page and experienced visually through the sense ofsight, it appears reasonable to assume co-equal statuses between the twomodalities. This assumption challenges the conventional privileging of lan-guage over the visual image. Here it is recognized that both the linguisticand the pictorial semiotic resources serve different, though complementary,functions. Therefore, both are equally important as signifying systemsthrough the different roles they perform. This point is developed below.

Until recently, the pictorial text has often been relegated to the status ofmere illustrations to the linguistic text. In the field of semiology, recentinterest in visual communication may be traced to Barthes's (1977) influen-tial work, Rhetoric of the Image, where the visual images are seen to play asomewhat attendant role to language. That is, Barthes proposes that lan-guage serves to 'anchor' (by elaborating) or 'relay' (by extending) the mean-ing of the visual text. However, it is important to recognize that despite theconstant co-deployment of language and pictures in multimodal texts, boththe linguistic and the pictorial semiotic modalities have the potential tofunction independently. Some instances of these include the popularity ofwordless picture books, such as Monique Felix's (1980) The Story of A LittleMouse Trapped in a Book, and the increasing use of wordless instruction sheetsto transcend language barriers, such as the Swedish-based but internation-ally marketed IKEA furniture which utilizes only the pictorial semiotic inthe assembly instructions. The success of these examples of visual com-munication attests the ability of the pictorial modality to operate as anindependent semiotic resource.

The adoption of the stance that both the linguistic and the pictorial modal-ities should share an equal status is now widely recognized (for example,Baldry, 2000; Kress and van Leeuwen, 2001; O'Halloran, 2000; Thibault,2000). Van Leeuwen (2000), for instance, criticizes the negative comparisonsbetween language and visual images in his refutation of Barthes's(1977) earlier proposition that words have 'fixed meaning' while imagesare 'polysemous'. In addition to this, van Leeuwen (2000) confronts some

PRINT MEDIA 229

misconceptions regarding the pictorial semiotic such as the assertion thatvisual images cannot represent negative polarity. Van Leeuwen (2000: 179)also argues that visual semiotics should focus 'not only on the image asrepresentation, but also on the image as (interact5.

However, it is important to remember that each semiotic resource (lan-guage, visual images, mathematical symbolism, gesture for example) hasevolved to be used in conjunction with other semiotic resources, and this ratherobvious but often neglected fact has serious implications for the way we viewthe functions and resulting grammatical and discourse systems of eachresource. Examining one semiotic resource in isolation, for example lan-guage, results in an impoverished view of how that resource is organized formeaning. The grammatical and discourse systems of each semiotic resourceneed to be considered in relation to how they are organized to interact withsystems in other semiotic resources to accomplish particular functionswithin the whole realm of what can be achieved semiotically.

Lemke (1998), for example, observes that language and visual imageseach have their individual functions and strengths. He summarizes some ofthe key distinctions by noting that language is more adept in encapsulatingtypological meaning, or meaning by category. It is also a more time-sensitivesemiotic where the linear progression of time can be reflected. The pictorialsemiotic, on the other hand, has resources for the representation of topo-logical meaning, or meaning by degree. It is also a more space-sensitivesemiotic that supersedes the linguistic mode in representing spatial relations.Each with their own niches, it is hardly surprising to find them servingdifferent functions in a multimodal text. In addition, the co-deployment ofthese two modalities in a multimodal text can lead to meaning expansions aswell. Nevertheless, it is important to understand that systems within eachresource independently have the potential to realize unique meanings thatmay not necessarily be integrated during intersemiosis. This is the meaningmade by each independent modality on each stratum and is topologicallyreflected in the model as the area outside the Sol as shown in Figure 9.1.

The assumption of equal status means that both are accorded co-equalvalue in meaning-making. The implication of this on the Expression stratumis that of perceptual equity between the two semiotic modalities. It must benoted, however, that having the same status does not translate to the claimthat both the semiotic resources of language and the pictures have the samedegree of influence on each other in a text. It is not unusual to find that inone particular text, the linguistic semiotic may be more dominant in termsof meaning than the pictorial semiotic, and yet in another text, the visualsemiotic may be the primary semiotic source for meaning. This point isfurther elaborated when intersemiosis between language and visual imageon the Sol is discussed below. The proposed model shown in Figure 9.1allocates equal space for each semiotic resource thereby signifying topo-logically the equity in status between the two semiotic modalities.


Reading path

The assumption of perceptual equity on the Expression plane has pro-found implications for our approach to the analysis of the multimodal text.The Expression plane is the interface the reader experiences upon readingthe text. In this paper, the term 'reading', despite being a term derivedfrom the study of language, is taken to include visual perception or viewing.Following Sardar and van Loon's (2000: 44) work in media studies, readingis defined as 'the process of interaction when a text is analysed as well as thefinal result of that process, the interpretation'. Hence, in any multimodaltext, it is useful to chart a typical reading path that the hypothetical reader mayfollow in the reading of different episodes on a page. In a sense, the readingpath is the order by which the reader may process different episodes in amultimodal text.

As previously mentioned, Thibault (2000, forthcoming) and Baldry andThibault (2001, 2004) use phasal analysis in their deconstruction of a filmsegment, where salience or the 'use of foregrounding strategies' allows forcertain modalities to be thrust into prominence. Analysis is therefore guidedby the contrastive salience of a specific semiotic resource in each particularinstance. This presupposes and builds upon the theory of a 'reading path'where the viewer reads according to the contrastive salience of the semioticresources at each instantiation. O'Halloran (1999: 323) proposes that a prac-tical approach to analyzing a multi-semiotic text can be through a progres-sive analysis following the 'reading path determined by the choices withindifferent semiotic codes'.

The notion of a linear or uni-directional reading path, however, deservesto be more closely scrutinized. This conception seems to be appropriate fora reader reading a book or magazine, navigating across the pages or framesin a linear reading pattern, governed by literacy conventions. FollowingPang (2000), however, this would more suitably be termed as a directional pathrather than a reading path. The usefulness of a restrictive and regulatedreading path breaks down when analyzing the multimodal text on a page orframe. The reading path on a multimodal frame is seldom only uni-directional, as the hypothetical reader's eyes are led through contrastivesalience, possibly even in a back and forth fashion between two items orEpisodes (O'Toole, 1994) on a page. In other words, the path, althoughsequential due to constraints of human visual perception, may not be uni-directional but is free to be bidirectional (Pang, 2000) or multidirectional asdisplayed in Plate 9.1. Following the assumption of perceptual equity, thereading path may disregard the distinction between linguistic and pictorialsemiotic resources as the reader is drawn by the contrastive salience of asection or Episode.

Kress and van Leeuwen (1998) introduce the notion of scanning whichclarifies their earlier claim that readers tend to read in a left to right and upto down pattern. They describe scanning as a process that occurs beforereading. The 'scanning process sets up connections between the different

PRINT MEDIA 231

Plate 9.1 Unrestrictive bidirectional reading path across three Episodesreproduced from Wong (2000: 6)

elements, relating them to each other in terms of their relative importance'(1998: 205). This 'relative importance' is determined by the contrastivesalience between Episodes. The scanning process first locates our eyes on theCentre of Visual Impact (C VI), which signals the beginning of our reading pro-cess. The scanning pattern is closely related to salience of semiotic choiceswithin the multimodal page and the Context of the reader's literacyconventions.

The notion of a CVI is an interesting one. Bohle (1990) cites Garcia'sproposal of the CVI as the focal point where the reader enters the page.Working in the tradition of Gestalt psychology of picture perception, Sonesson(1993: 378) claims that evidence has been found for 'the existence, if not foran order of reading, then at least of certain points of fixation where theglance tends to cluster'. The initial point of fixation or the CVI is thehypothetical reader's point of entry into the multimodal text, which initiatesthe entire process of visual perception. Thus, on a web page for instance,although there may exist in theory multiple entry points into the text, inpractice semiotic choices function to ensure that the viewer's attention isinitially focused on one part of the text. This can be explained, for example,by the relative Interpersonal salience of semiotic choices, a point which isdeveloped below.

Critical impetus in metafunctional meaning in the Expressionplane

The purpose of this section is to introduce concepts which require furtherdevelopment and theorization, in particular the notion of 'critical impetus'which is used to explore the metafunctional diversity of the systems operating


on the Expression plane. This includes developing the notion that a vieweris drawn towards interpersonally salient components in a multimodal text.While system networks for some of the more prominent systems of thelinguistic and pictorial modalities on the Expression stratum are proposed inthe next section of this paper, these are not exhaustive and remain verymuch at a preliminary stage.

Although the meanings made through the systems in the grammar stra-tum are organized metafunctionally, the tri-metafunctional distinctionappears to be more uncertain on the Expression plane as previously dis-cussed. These systems with a low system-metafunction fidelity can be moreappropriately described as functioning on a cline and, as such, the classifica-tion of the systems is not based on metafunctionally based discrete categor-ies in Figure 9.1. Instead, systems operating on the Expression plane cancontribute to the ideational, Interpersonal and Textual meanings in a text. Itis therefore useful to examine the critical impetus, or the necessary conditionsand circumstances which reveal which particular metafunctional meaning islikely to emerge from choices within systems on the Expression plane.

The critical impetus for a dominant Interpersonal meaning on the Expres-sion plane is salience., and this can be achieved through contrast of Colour,Shape, Size, and so forth. The critical impetus for Textual meaning on theExpression plane is the presence of Textual unity and cohesiveness. But first,what is the nature of the ideational meaning made on the Expressionplane? Visual semioticians Floch (1986) and Thurlemann (1990) haveobserved a double layer of signification in pictures. They term the first levelas 'iconic' and the second as 'plastic'. Sonesson (1993: 325) explains that 'onthe iconic level, the picture is supposed to stand for some object recognizablefrom the ordinary perceptual lifeworld, while concurrently on the plasticlevel, simple qualities of the pictorial Expression serve to convey abstractconcepts' within the lifeworld as well. Lifeworld, according to Husserl, is the'world taken for granted'. To extend this rather crudely into SFL terms,lifeworld can be compared to the Context of Situation and Context ofCulture, the social reality in which the individual operates.

Doonan (1993: 15), working on picture books from a literary perspective,also recognizes the 'two modes of referring' in pictorial images. She simpli-fies 'Denotation' as the representation of an object in a particular Context ofculture. 'Exemplification', on the other hand, is the mode by which'abstracted notions, conditions and ideas' (1993: 15) are represented withinthat culture. This approach to the representation and composition of pic-torial semiotics is congruent to our proposed formulations in this paper,which draws expedientiy upon some of these ideas. Modifying the originalsense of denotation and connotation as proposed by Barthes (1977), theterms Denotative Value and Connotative Value are used to describe the two typesof ideational meanings made on the Expression and Content strata.

The Denotative Value is understood as the literal or iconic meaning. Forinstance, the denotative value of the colour red is confined to the perceptionand reference of the reddish hue. Saint-Martin (1990) observes that two

PRINT MEDIA 233

persons can look at one colour and yet see it differently. Hence, it must beadded that the use of denotative value is qualified with the acknowledgingof the reader's cultural-based subjectivities. This contrasts with Barthes's(1977) use of denotation as a rather non-Context-dependent Platonic ideal.In other words, the denotative value is understood in this paper as the literalbut Context-dependent meaning. Like Floch (1986) and Thurlemann's(1990) conception of the 'plastic' and Doonan's proposal of'Exemplifica-tion', the Connotative Value is the ideas and abstractions evoked from theliteral image. For instance, the connotative value of the colour red refers tothe abstract concepts which the colour evokes in the reader. Dependent onthe Context of culture, situation and co-text, the red hue could connoteantithetical ideas ranging from danger in a European Context to good for-tune in Chinese culture.

The Interpersonal meaning dominates when system choices on theExpression stratum generate Salience., in other words, when salience has acritical impetus. This salience can sometimes be achieved through contrastsin, for example, Size, Shape and/or Colour as mentioned above. The criticalimpetus of salience can be linked to the notion of'markedness' in Halliday's(1994) conceptions. The notion of'markedness' could be helpful to accountfor the meaning expansion on the grammar stratum as well as on the Expres-sion plane. Markedness in Halliday's (1994) original usage means to'stand out' as an atypical choice. The choices made in Typography for mosttexts, for example, are usually stereotypical options according to their genre.For instance, in the Context of a piece of formal academic writing such as adissertation, a particular selection of appropriate Typography is expected. Inaddition, because of the association of certain Typography with particulargenres, any departure from the convention or mismatch between Typog-raphy and genre would render those typoGraphical choices as 'marked'.This is consistent with Halliday's (1994) observation that there is an order ina clause which is usually expected in a particular clause type, for example,the nominal group functioning as Subject is usually the first item in a clausewhich has a declarative mood. When this order is not adhered to, the clauseis marked. A marked selection in Typography is similarly meaningful.

The notion of critical impetus is thus useful when included in systemicanalysis of both linguistic and multimodal discourse. The critical impetus isused to identify the environment whereby certain Interpersonal meaningsmay dominate through the notion of marked choices. In the same manner,Textual meanings are usually observed when the critical impetuses ofUnity and Cohesiveness in a text are in operation. For example, in a tapestrydesign, the system of Saturation and Hue in Colour and the geometricforms through the system of Shape operate to create unity and cohesion inthe text. As may be seen from this very preliminary discussion, however,further research is needed to understand the conditions under which certainmetafunctional orientations are realized through choices from the systemsoperating on the Expression strata. Provisional networks for these systemsare given in the next section of this paper.


Linguistic and visual system networks on the Expressionplane

The predominant linguistic and visual systems which operate on the Expres-sion stratum, respectively referred to as Typography, is displayed in Figure9.4. Given their metafunctional diversity, the systems are not classifiedaccording to metafunctional categories. As seen in Figure 9.4, meaning onthe Expression plane is made through the selections in Typography withinthe systems of Font and Layout. The system of Font in Typography hasthree sub-systems, Type, Size and Colour. Paradigmatic options are alsoavailable within each of the three sub-systems.

As the meaning potential of particular systems is theoretically infinite, it isnot possible to list all the possible options. Thus, the network represented inFigure 9.4 confines itself to a few common selections for the purposes ofexemplification. For instance, the system of typeface keeps expanding withnew font types being created. The options shown within this system inFigure 9.4 are merely the Typeface families, within which many other Type-faces are classified. For instance, the Typeface Times New Roman is categor-ized within the Roman family. The system of Size similarly contains toomany options to be listed here, and thus the option of 12 point is an exampleand the sign °° indicates the system's infinite potential. It is the ultimate cline.

As displayed in Figure 9.4, the system of Layout includes the systems ofSpacing and Justification. The system of Spacing has four sub-systems. Lead-ing is the spacing between lines on a page, which includes options for doubleto single spacing. Kerning, on the other hand, is the adjustment of spacebetween the letters of a word. The Internal Space refers to the spacebetween words and the system of Justification is the alignment of the sen-tences. Finally, the choices for Indentation allow a clearer demarcationbetween paragraphs and function primarily to signal a shift in direction ortext-type from the preceding lines.

The systems operating on the Expression plane for pictures on theGraphics stratum, namely Perspective and Form, are displayed in Figure9.5. Perspective, according to Doonan (1993: 34), is 'the way an artist con-trols space in the picture'. Perspective has two sub-systems: Deep Space (DS)and Point of View (PoV). DS portrays an illusion of a three-dimensionalworld through a two-dimensional image on a page thereby generating asense of illusionary depth. DS can be achieved through Contrasting Size,Converging Lines or Chiaroscuro. The use of Contrasting Size, forexample, in Picture A in Plate 9.2, shows that illusionary depth is created asthe play slide is represented as located further back in the picture world. In atwo-dimensional surface, the figure of the duck is shown to be larger thanthe slide. This interpretation, however, defies the hypothetical reader's cul-tural knowledge. Hence, in order to make sense and maintain relevance, thereader assumes that DS through Contrasting Size generates the impressionof a three-dimensional world. This interpretation, as opposed to a worldof enormous ducks, fits more congruently with the reader's world. The

PRINT MEDIA 235

Figure 9.4 System network for Typography

theoretical assumption behind this interpretation is consistent with Sperberand Wilson's (1986) theory of relevance in verbal communication, whichsuggests that their observations can be extended to visual communicationas well.

In Plate 9.2, the use of Converging Lines to produce DS is seen in PictureB taken from Satoshi Kitamura's (1986) When Sheep Cannot Sleep. The series


Figure 9.5 System network for Graphics

of converging vectors gives a sense of the illusionary depth and adds a senseof three-dimensionality into the picture world. Finally, Chiaroscuro is theapplication of light and shadows to create DS in Picture C. The example of

PRINT MEDIA 237

Picture A: Contrasting sizes.Reproduced from Wong (2000: 9)

Picture B: Converging lines.Reproduced from Kitamura (1986: 15)

Picture C: Chiaroscuro.Reproduced from Yee (1998: 13)

Picture D: Low tilt.Reproduced from Sallustio (1999a: 4-5)

Picture E: A high tilt.Reproduced from Kitamura (1986: 11)

Picture F: Close-up and medium shot.Reproduced from Sallustio (1999b: 24-25)

Plate 9.2 Perspective


the Singaporean Merlion statue shows how shading can suggest a sense ofthree-dimensionality on a two-dimensional plane.

PoV is the viewpoint through which the reader is presented with a scenein the picture. Following cinematography theory, Bordwell and Thompson(1997: 241) explain that there are systems available in a cinematic shotwhich determine the reader's entry into the story world. Two main systemsare Angle and Distance. Angle is the tilt at which the visual image is pre-sented. A high tilt may place the viewer in a somewhat voyeuristic position.This can be seen in the frame shown in Picture E in Plate 9.2, where thereader is 'situated' in the position of an intrusive outsider. A sense of alien-ation and detachment or feelings of superiority could result from a skilfuluse of the high tilt. Correspondingly, a low tilt may lead the reader to feeloverwhelmed, usually with the character positioned to be 'towering' over thereader. An example can be seen in Picture D, where the pile of toys isemphasized and the children are portrayed above the clutter. Finally, thesystem of Distance has the categories of Long Shot, Medium Shot and Close-up. Although these categories are relative, they are typically discernible, asdisplayed in Picture F, and have a powerful effect.

The system of Form displayed in Figure 9.5 contains four sub-systems,those of Colour, Shape, Line and Strokes. Colour, following Doonan (1993),operates through three sub-systems. Hue or pigment distinguishes the col-our across the spectrum, making it possible to discriminate, for instance,blue from purple. Tone 'is a measure of light and dark of an area regardlessof its colour, and its quality of a surface as measured purely by its position inthe scale between black and white' (Doonan, 1993: 30). Tone or shading canrender the effects of texture and lighting. Saturation refers to the purity of acolour. The primary colours such as red, yellow and blue are hues with thehighest level of intensity or saturation.

The system of Shape includes the options Geometric and regular or Non-Geometric and irregular. The selection of shapes adds to the multifariousmeaning made in the text. For instance, a picture composed of largelyregular shapes positioned horizontally or vertically could suggest stabilityand even a sense of rigidity. The system of Line 'creates contour, modelling,shading and a sign for movement. A contour puts a Line round objects andfigures and gives them individuality and character' (Doonan, 1993: 23).Lines such as those used to create varying tone could render the effect oflighting conditions. Finally, the system of Strokes in Graphics refers to theway in which colour is applied. Some common options available are Brush,Pencil, Paint and Crayon. Once again, these systems are not exhaustive, butrather they are presented to illustrate how systems on the Expression planecontribute to the overall meaning of the text.

Space of integration (Sol)

The Sol functions as the theoretical platform for discussion of the dynamicsin the interaction between language and visual images for meaning-making

PRINT MEDIA 239

in a multi-semiotic text. Sol topologically reflects the semantic multiplica-tion brought about by the interaction and integration in intersemiosisbetween the two semiotic resources. Thibault (2000: 362) explains that it is'on the basis of co-contextualizing relations that meaning is created'. Thispaper proposes contextualizing relations as the meaningful relationships that arepresent between two modalities. Intersemiosis is therefore a result of thecontextualizing relations between the two semiotic modalities.

One of two types of contextualizing relations can be found whenever twomodalities operate in a multimodal text. In cases where the meaning of onemodality seems to 'reflect' the meaning of the other through some type ofconvergence, the two resources share co-contextualizing relations. On the otherhand, in cases where the meaning of one modality seems to be at odds withor unrelated to the other, their semantic relationship is one that diverges.Here the resources share re-contextualizing relations. The implications of thesetwo contextualizing relations are apparent in the semantic expansion thatconsequently occurs with the co-deployment of language and visual images.

It may be helpful to differentiate between the nature of the interactionbetween the semiotic modalities and the extent or degree to which thelinguistic item contextualizes the meaning of the visual image. Both semioticmodalities can either co-contextualize or re-contextualize the other, regard-less of the degree of contextualization on each other. The nature of theinteraction between the two semiotic modalities thus refers to whetherthe two resources are co-contextualizing or re-contextualizing. Further tothis, the interaction between the semiotic resources is seen to be mutuallycontextualizing at every instantiation, as opposed to Barthes (1977: 26) con-ception of the visual image which 'illustrates the text. . . [or] the text loadsthe image'.

Cheong (1999; this volume) refers to intersemiosis between the two modal-ities as the Bidirectional Investment of Meaning. Cheong's (1999) analysis ofadvertisements as multimodal texts suggests that the degree of intercon-nectedness and the degree of interweaving of meaning between languageand visual images can be measured through a scale known as ContextualizationPropensity (GP). CP 'refers to the degree/extent to which the linguistic itemscontextualize the meaning of the visual images' (Cheong, 1999: 44). In otherwords, GP measures the strength of the influence the modalities exercise oneach other. Gheong (1999) also shows that CP in turn has a direct influenceon the Interpretative Space (IS) of the reader resulting in either a high or lowSemantic Effervescence (SE) of the text. For example, a multimodal text with ahigh CP will lead to a low IS thus resulting in a low SE. Essentially, Cheong's(1999) proposals provide us with the meta-language to look at the degree andextent of contextualization the two semiotic resources have on each otherand the implications of these contextualizing relations.

The focus here, however, is the nature of the interaction between the twosemiotic modalities. Understanding this phenomenon can contribute sig-nificantly to a clear understanding of the mechanisms at work on the Sol.For example, further expansion of meaning may occur on the Sol through


the process of Homospatiality in the Expression plane and Semiotic Meta-phor in the Content plane. The avenue by which this multifaceted seman-tic multiplication occurs on each plane is discussed in the followingsections. As will become evident in this discussion, further research is neededto understand the range of mechanisms through which semantic expansionstake place intersemiotically (see also O'Halloran, forthcoming).

Sol on the Expression plane

One mechanism which can result in a semantic expansion on the Expressionplane is Homospatiality. The term is adapted from Carroll's (2001: 355)conceptualization of 'disparate elements in one spatially bonded homogen-ous entity'. Carroll (2001) proposes the term for the analysis of visual meta-phors. However, its incorporation into IMM to describe the relatedphenomenon of two systems sharing the same spatial coordinates on theExpression plane appears to be useful. This integration of the two differ-ent semiotic systems where one superimposes on the other usually results insemantic multiplication on the Content plane, where the meaning made isreinforced or where new meanings are made.

An example of Homospatiality is shown in Picture A on Plate 9.3, wherethe linguistic text, 'Snaaap', realized through the system of Font on theTypography, shares the same spatial coordinates as the visual image realizedthrough the systems in Graphics of the word breaking into two. The mech-anism of Homospatiality reinforces the meaning of a strong force breakingan object into two with an accompanying 'image' of sound. Hence, anexpansion of meaning through reinforcement results from this process ofHomospatiality. Another example is found in Picture B in Plate 9.3 where thevisual image of the smoke emitted by the campfire functions simultaneouslyas the Typography for the word 'hot'. Thus, through the mechanism of

Plate 9.3 Homospatiality: Picture A reproduced from Sallustio (1999b: 4)

Picture A Picture B

PRINT MEDIA 241

Homospatiality, an intensified sense of heat and smoke from the fire is repre-sented. These extensions of the meaning stem from the intersemiosis on theExpression plane of the multimodal text, which engenders the meaningarising from choices in the Content plane.

Sol on the Content plane

Semiotic Metaphor is a mechanism, proposed by O'Halloran (1999a, 1999b,2003, forthcoming), which operates on the Content plane, more specif-ically in the Sol existing between the lexicogrammar of language and thevisual grammar. O'Halloran defines semiotic metaphor as an intersemioticprocess whereby a shift in the functional status of an element arisesthrough a shift between semiotic resources. That is, with a movementbetween semiotic codes, 'the new functional status of the element does notequate with its former status in the original semiotic or, alternatively, a newfunctional element is introduced in the new semiotic which previously didnot exist' (O'Halloran, 1999a: 348). O'Halloran gives two examples ofsemiotic metaphor which occurred in a lesson on trigonometry in a sec-ondary high school. In the first case, a process realized by the verb 'look' inthe teacher's oral linguistic statement 'and he looks down of course'becomes an entity in the form of a line segment in the visual diagram onthe blackboard (O'Halloran, 1999b: 24, forthcoming). O'Halloran explainsthat a second semiotic metaphor occurs in this lesson when the new entity ofa triangle is constructed visually in the mathematical diagram on theblackboard. This entity did not exist prior to the visual semiotic representa-tion of the trigonometric problem. Although originally proposed for theintersemiotic reconstrual of elements occurring across language, visualimages and mathematical symbolism in mathematical discourse, the notionof semiotic metaphor is productive in its extension to other semioticresources.

An example of Semiotic Metaphor is shown in Plate 9.4. The visual imageof the diamond is juxtaposed with the linguistic clause 'because he lovesme'. This association of the visual image of a diamond with the linguisticclause implies the gift of a diamond is an Expression of love. Here thedynamic process of 'love' is reconstrued as a fixed entity in the form of adiamond, and thus is an example of a semiotic metaphor. Indeed, it couldbe argued that diamonds (as gems, not cutting agents) are in themselvesalways semiotic metaphors. This may be true of all social symbols, wherepeople are encouraged to attach a range of complex and dynamic meaningsto fixed entities in the form of consumer goods. The new meanings arecultivated and circulated through the co-deployment of different modalitiesin the media. As such, advertisements can be seen to specialize in the cre-ation of semiotic metaphors.

O'Halloran (1999a) distinguishes between different forms of semioticmetaphor, parallel and divergent, which may be seen to function as oppositeends of a cline. A parallel Semiotic Metaphor has 'an expanded semantic field but


Plate 9.4 Semiotic Metaphor (reproduced from http://www.hearts-on-fire.com)

also one which is situated within the old' (1999a: 348). Although there couldbe redundant meanings due to overlaps, 'new layers of meaning are [essen-tially] simultaneously added to the original representation'. The reconstrualof elements in a divergent Semiotic Metaphor, however, is more far-reaching.Here 'the functional element is reconstrued into a new semantic field'(ibid.}. The metaphorical shift in meaning accompanying such divergentreconstruals is substantial as the functional element is literally relocated ina semantic field which is not typically intertextually related to the first.O'Halloran (1999a) explains that the types of semantic shifts involvedin divergent semiotic metaphors, however, gradually become naturalizedover time.

A possible by-product of the meaning made through parallel semioticmetaphors is semantic redundancies. These redundancies are realized whenthere is a duplication of the meaning made by the semiotic resources. Thesemeanings, though actualized when the modalities are independent, serve areinforcing function when the two systems combine in the Sol. A by-productof divergent semiotic metaphor, on the other hand, could be the surfacingof conflicting meanings. These conflicts or examples of 'ideological disjunc-tion' are a possible result 'of the complex, often intricate, relations of inter-functional solidarity among the various semiotic resource systems that areco-deployed' (Thibault, 2000: 321). However, the Sol usually brings about aharmonization of these disjunctions and conflicts 'in the service of thesemiotic project of this particular text' (ibid.}. In a multimodal text where themodalities share co-contextualizing relations, there is a stronger likelihoodfor parallel semiotic metaphors to arise, where the new meaning maderemains situated within the old. Divergent semiotic metaphors wherenew, previously unrealized meanings are being made through the processare more likely to emerge from a text where its modalities share re-contextualizing relations.

http://www.hearts-on-fire.com

PRINT MEDIA 243

Conclusion

As a meta-model, the IMM attempts to synthesize various research effortsby situating them on the strata, planes and metafunctional dimensions ofthe IMM where there is greater centrality and focus. For instance, the fieldof materiality and medium of resource is located within this larger theor-etical multi-semiotic model, in this case, across the communication planes.The IMM is designed to help unify diverse research efforts in the field bylocating their contributions into a single model, which takes into account thecomplexities of multimodal meaning-making.

However, some qualifications exist with respect to the IMM. The problemof addressing a dynamic phenomenon with a typological description andframework is a perennial quandary. Hence, the IMM may bear the criti-cism, like other frameworks, of being reductionist and even rigid in thecategorization of systems according to the metafunctions, despite the use-fulness of the metafunction as a principle of theoretical integration. Theseverity of this criticism, however, will be somewhat alleviated in the IMMwith the construction of a model that can reflect more effectively topologicalmeaning in dynamic environments such as those afforded by film and hyper-text. In addition, at this stage the categories in actuality are more fluid thancan be represented by clearly delineated and neat classifications of systemsin the model.

Apart from recognizing the fluidity of the classifications, it is useful tonote that each of the metafunctions may not be equally dominant on amultimodal page. O'Toole (1994) discusses the monofunctional tendenciesof certain schools of paintings, where a single metafunction may tend todominate in a certain work. Similarly, not all metafunctions are equallysalient in a multimodal text, despite the appearance of the equal topologicalspace allocated to each metafunction in the abstract theoretical constructionof the IMM. Hence, it is not surprising to find a particular metafunctionhaving a greater role in a certain multimodal texts.

O'Toole (1999) also comments that since only some options within thesystems in the matrix are selected in the construction of any one text, it isnot necessary to account for every system in the analysis of a text. Likewise,in the IMM, there are many systems used to describe and analyse a multi-modal text. However, not every single system needs to be accounted for inan analysis; rather, the model is to serve our purpose of understanding howmeaning is made in a multimodal text through the choices which have beenmade in the text.

Despite these possible weaknesses, a categorical framework for theanalysis of a multimodal text that pays attention to the meaning made onthe Expression plane as well as on the Space of Integration is helpful.IMM may be likened to a neat (although at this stage underequipped)toolbox. The toolbox contains concepts and a theoretical meta-language todescribe and account for phenomena which arise in the multimodal con-struction of meaning. Just as one does not use all the equipment in a


toolbox in any one instance, the analyst selects the tools most useful for theanalysis of the text. However, it is also realized that the IMM and theaccompanying conceptual apparatus are provisional and exploratory.There remains much work to be done in the theory and practice of multi-modal analysis.

Acknowledgements

Plate 9.1 and Picture A in Plate 9.2 are reproduced with kind permissionfrom SNP Panpac Pte Ltd, Singapore from the children's picturebookDominic Duck Goes to School (2000) written by Maeli Wong and illustrated byDon Low. I thank Zuraidah Jaffar for generously waiving the copyright feesfor reproducing these pictures.

Picture B and Picture E in Plate 9.2 are reproduced from When SheepCannot Sleep written and illustrated by Satoshi Kitamura with kind permis-sion from Andersen Press. Thanks also to Red Fox who currently publish thepaperback version of the book.

Picture C in Plate 9.2 is reproduced from the book Rhyming Round Singapore(Yee, 1998), written by Patrick Yee, Kathleen Chia and Linda Gan, Girl'sBrigade, Singapore. Thanks to Linda Gan for kindly granting permission toreproduce the picture.

Picture D in Plate 9.2 is reproduced from The Tidy-Up Race and PictureF in Plate 9.2 and Plate 9.3 are reproduced from Lightning and Thunder byE. Sallustio with kind permission from the Educational Publishing House(Singapore) with special thanks to Margaret Tan for her assistance.

Plate 9.4 is reproduced from the website http://www.hearts-on-fire.comwith kind permission from 'Hearts on Fire - The World's Most Perfectly CutDiamond'.

References

Baldry, A. P. (2000) (ed.) Multimodality and Multimediality in the Distance Learning Age.Campobasso, Italy: Palladino Editore.

Baldry, A. P. (this volume). Phase and transition type and instance: patterns in mediatexts as seen through a multimodal concordancer, 83—108.

Baldry, A. P. and Thibault, P. J. (2001) Towards Multimodal Corpora. In G. Astonand L. Burnard (eds), Corpora in the Description and Teaching of English, Bologna:GLUEB, 87-102.

Baldry, A. P. and Thibault, P. (forthcoming) Multimodal Transcription and Text.London: Equinox.

Barthes, R. (1977) Rhetoric of the image. In R. Barthes (S. Heath, ed. and trans.),Image-Music-Text. London: Fontana, 32-51.

Bohle, R. (1990) Publication Design for Editors. New Jersey: Prentice-Hall.Bordwell, D. and Thompson, K. (1997) Film Art: An Introduction. New York:

McGraw-Hill.Carroll, N. (2001) Beyond Aesthetics: Philosophical Essays. Cambridge: Cambridge Uni-

versity Press.

http://www.hearts-on-fire.com

PRINT MEDIA 245

Cheong, Y. Y (1999) Construing meaning in multi-semiotic texts: a systemic-functional approach. Unpublished masters thesis. National University ofSingapore.

Gheong, Y. Y (this volume) The construal of ideational meaning in print advertise-ments.

Doonan, J. (1993) Looking at Pictures in Picture Books. Exeter: Short Run Press.Felix, M. (1980) The Story of a Little Mouse Trapped in a Book. Lajolla, CA: The Green

Tiger Press.Floch, J.-M. (1986) Les Formes de I'empreinte. Perigueux: Pierre Fanlac.Gombrich, E. (1960) Art and Illusion. London: Phaidon Press.Halliday, M. A. K. (1978) Language as Social Semiotic. London: Edward

Arnold.Halliday, M. A. K. (1994) An Introduction to Functional Grammar (2nd edn). London:

Arnold.ledema, R. (2003) Multimodality, resemioticization: extending the analysis of dis-

course as a multi-semiotic practice. Visual Communication, 2(1): 29—57.Kitamura, S. (1986) When Sheep Cannot Sleep: The Counting Book. New York: Farrar

Straus Giroux.Kress, G. (2003) Literacy in the New Media Age. London: Routledge.Kress, G. and van Leeuwen, T. (1996) Reading Images: The Grammar of Visual Design.

London: Routledge.Kress, G. and van Leeuwen, T. (1998) Front page: (the critical) analysis of news-

paper layout. In A. Bell and P. Garrett (eds), Approaches to Media Discourse. Oxford:Blackwell, 186-219.


Kress, G., and van Leeuwen, T. (2002) Colour as a semiotic mode: notes for agrammar of colour. Visual Communication, 1(3): 343-368.

Lemke, J. L. (1998) Multiplying meaning: visual and verbal semiotics in scientifictext. InJ. R. Martin and R. Veel (eds), Reading Science: Critical and Functional Perspec-tives on Discourses of Science. London: Routledge, 87-113.

Lim, F. V (2002) The analysis of language and visual images — an integrative multi-semiotic approach. Unpublished masters thesis. National University of Singapore.

Lemke, J. L. (2002) Notes on multimedia and hypertext. Available online: http://www-personal.umich.edu./~jaylemke/

Martin. J. R. (1992) English Text: System and Structure. Amsterdam: Benjamins.Martin, J. R. and Rose, D. (2003) Working with Discourse: Meaning Beyond the Clause.

London: Continuum.O'Halloran, K. L. (1999a) Interdependence, interaction and metaphor in multi-

semiotic texts. Social Semiotics 9(3): 317-354.O'Halloran, K. L. (1999b) Towards a systemic-functional analysis of multi-

semiotic mathematics texts. Semiotica, 124(1/2): 1-29.O'Halloran, K. L. (2000) Classroom discourse in mathematics: a multi-semiotic

analysis. Linguistics and Education, 10(3): 359-388.O'Halloran, K. L. (2003) Intersemiosis in mathematics and science: grammatical

metaphor and semiotic metaphor. In L. Ravelli, A.-M. Simon-Vandenbergen andM. Taverniers (eds), Grammatical Metaphor: Views from Systemic Functional Linguistics.Amsterdam: Benjamins.

O'Halloran, K. L. (forthcoming). Mathematical Discourse: Language, Visual Images andMathematical Symbolism. London: Continuum.

http://www-personal.umich.edu./~jaylemke/

http://www-personal.umich.edu./~jaylemke/


O'Toole, M. (1994) The Language Of Displayed Art. London: Leicester UniversityPress.

O'Toole, M. (1995) A systemic-functional semiotics of art. In P. H. Fries andM. Gregory (eds), Discourse in Society: Systemic-Functional Perspectives: Meaning andChoice in Language: Studies for Michael Halliday (159—179). Norwood, NJ: Ablex,159-182.

O'Toole, M. (1999) Engaging with Art. [CD-ROM] Murdoch University, PerthWestern Australia.

Pang, K. M. A. (2000) Designing Children In Changing World, Changing Hopes: AMulti-semiotic Analysis of a Museum Exhibition. Unpublished Honours Disser-tation. National University of Singapore.

Pang, K. M. A. (this volume). Making history in From Colony to Nation: a multimodalanalysis of a museum exhibition in Singapore, 28—54.

Royce, T. (1998a) Intersemiosis on the Page: A Metafunctional Interpretation ofComposition in the Economist Magazine. In P. Joret and A. Remael (eds),Language and Beyond. Amsterdam: Rodopi, 157—176.

Royce, T. (1998b) Synergy on the Page: Exploring Intersemiotic Complementarityin Page-Based Multimodal Text. JASFL Occasional Papers, 1(1), 25-49.

Royce, T. (2002) Multimodality in the TESOL Classroom: Exploring Visual-VerbalSynergy. TESOL QUARTERLY, 36(2), 191-205.

Saint-Martin, F. (1990) Semiotics of Visual Language. Bloomington, IN: Indiana Uni-versity Press.

Sallustio, E. (1999a) The Tidy-Up Race. Singapore: The Educational PublishingHouse Pte Ltd.

Sallustio, E. (1999b) Lightning and Thunder. Singapore: The Educational PublishingHouse Pte Ltd.

Sardar, Z. and Van Loon, B. (2000) Introducing Media Studies. Cambridge: Icon Books.Sonesson, G. (1993) Pictorial semiotics, gestalt theory, and the ecology of percep-

tion. Semiotica 99 (3/4): 319-399.Sperber, D. and Wilson, D. (1986) Relevance, Communication and Cognition. Oxford:

Blackwell.Thibault, P. J. (2000) The multimodal transcription of a television advertisement:

theory and practice. In Baldry (ed.), Multimodality and Multimediality in the DistanceLearning Age, 311-384.

Thibault, P. J. (forthcoming) The theory and practice of multimodal analysis ofvideo texts. In Baldry and Thibault (forthcoming).

Thurlemann, F. (1990) Vom Blid Sum Raum. Beitrage zu einer Semiotischen Kumstwis-senchaft. Koln: DuMont.

van Leeuwen, T. (2000) Some notes on visual semiosis. Semiotica 129(1/4): 179—95.Wong M. (2000) Dominic Duck Goes to School. Singapore: SNP Education Pte Ltd.Yee, P. (1998) Rhyming Round Singapore. Singapore: Girl's Brigade Singapore.

Index

Aarseth, E. 132Advertisements (printed) 4, 83Alberts, B. et al. 4,197,217

see also ECBAnderson, B. 3Announcement 164-5, 171, 173, 175-82,

186Primary 165-7, 170, 173, 179-81,

183-7Secondary 165, 167, 170, 173, 180-1,

184-5Antze, P. and Lambek, M. 49Appraisal 32, 38-40, 42-3

see also language systemsarchitecture 11-27, 55-6, 66, 68, 72, 110Arnheim, R. 43, 208-9, 217Australian Broadcasting Commission

(ABC) 15

Baigrie,B. S. 196,217Bakhtin, M. 21Bal, M. 43Baldry, A. P. 1, 2, 3, 5, 29, 83, 84, 85, 89,

90,94, 106, 110, 111, 113, 118, 163,196, 203-4, 215-17, 219, 220, 228

Baldry, A. P. and Taylor, C. 84, 96, 99, 106Baldry, A. P. and Thibault, P. J. 5, 84, 98Barthes, R. 135, 153, 156, 164, 175, 180,

191-2, 209, 214, 217, 228, 232, 239Bastide, E 209,217Beetle 165, 171-2, 175Belcher, M. 31Bennelong Point 21Bennett, T. 31,40,43Bernstein, B. 91Betsky, A. 55Bi-directional investment of meaning 4,

164, 176-8, 188see also intersemiotic mechanisms

biology texts 4, 196-219see also Essential Cell Biology (ECB]

Bohle,R. 148,168Bordwell, D. and Thompson, K. 114, 115,

116, 117,238Bruns,A. 192buildings

see also Sydney Opera House 55—6, 60,66

Business Times 33

Callaghan, J. and McDonald, E. 1,110

Call-and-Visit Information 164-7, 170,174-6

camera work 86, 88-9, 93, 119, 125, 126,238

Capture 164, 176Centre of Visual Impact (C VI) 148,

231Cheong,YY. 163chiaroscuro 18,234Chinatown 4,110,113-27Christopher, N. 116chthonicity in buidings 21Chua, B. H. 48, 55Circular Quay, Sydney 21circulation path (in exhibitions) 40-4clarity and focus 119cluster 135-7Coffin, C. 41collocation 1colour 119,124,224,225

see also language systemscolour cohesion and contrast 68, 72, 124,

125, 126, 127see also language systems

Communist Party 38-54Comp.LoA see Complement to the Locus of

AttentionComplement to the Locus of Attention

165-97, 169, 170, 189compositional balance 124,125

248 INDEX

compositional (textual) meaningbuildings 2, 11-13, 15, 23-5, 26, 60-1,

66-8,71cities 61,66-7film 120-3, 124-7hypertext 141-5museum exhibitions 31, 41-4, 47-8schematic drawings 200, 206, 208-9statistical graphs 202-3

concordancer see Multi-modal Corpus Authoring(MCA) system

conjunction 11connectors in buidings 23Contextual Propensity (CP) 4, 164, 176-8,

186, 188-94, 239see also intersemiotic mechanisms

Contextualising Relations 223, 239see also intersemiosis

Cook, G. 83,175Cortazzi, M. and Jin, L. 49costume 124culture of consumerism 65, 77

Dale, O. J. 57Darwin, C. 197Dean, D. 36Dellora, D. 27Display 164-7, 170, 175-6, 186

congruent 165—7explicit 165-7implicit 165, 170incongruent 165, 170

DoonanJ. 232-8duration image 126Dyer, G. 187

eating spaces 16—18Eaton, M. 114,115,116-17Emblem 164-7, 170, 175-6English for Specific Purposes (or English for

Academic Purposes) 196, 215-16Enhancer 164-7, 170-1, 173, 175-7, 182-7Epson 165-8, 175, 188, 191-3Essential Cell Biology (ECB) 197, 204-14experiential meaning see representational

meaningexpression plane 226-38

see also language system, graphics andtypography

Fairclough, N. 38Fan car advertisement 85—105Fawlty Towers 18Focus 164, 176Fong, T. W. 57

Ford,B.J. 196,217film 109-30

see also multimodal frameworkframing 124, 125, 126From Colony to Nation 2, 33—54functionalism in architecture 15, 26functionalism in design 18

gaze 76, 84, 90-1, 105, 124, 125, 126, 127Generic Structure 164, 166-7, 170Generic Structure Potential (GSP) 163-5,

174-6, 194genre

advertisements (print) 4dynamic 95—6, 105-6film 116-17television advertisements 84,96, 105

Gestalt theory 225gesture 84, 106, 125, 126Given and New Information 164, 192, 194Goffinan, E. 18,27Goldman, R. 179Golf 164,166,171, 173-5,177,179,185-7grammatical metaphor 206-7graphics 222,234-8

see also expression plane, language system,graphics and typography

perspective 234form 238strokes 238

graphology 226Gregory, M. 84, 87, 89, 94, 112, 113Gregory, M. and Malcolm, K. 84, 94, 98Guess? 165, 175, 188-94Gwee, P. K. W. 68, 72

Haas, C. 197,217Hall,M. 31Halliday, M. A. K. 1, 4, 5, 27, 28-9, 32, 56,

84, 85, 87, 98, 110, 131, 133, 142,176, 184, 186, 196, 204, 206-7, 212,217-18,220,233

Halliday, M. A. K. and Hasan, R. 85, 164Halliday, M. A. K. and Martin, J. R. 28Harris, R. 29Hasan, R. 164,176Heisner,B. 114,117,127Heng, G. and Devan, J. 138Hernadi, P. 32-3Hirsch,E 116Hodge, R. and D'souza, W. 33homospatiality 5

see also intersemiotic mechanismsHooke, R. 198Hooper-Greenhill, E. 30,31,33

INDEX 249

Humphries, B. 21hypertext 4,26,131-59

ledema, R. 1,110Integrative Multisemiotic Model (IMM)

220-46see also intersemiosis, intersemiotic

mechanisms and multimodalframeworks

contrastive salience 231-3critical impetus 224, 232

salience 232-3textual unity 232cohesiveness 232saturation 233hue 233

connotative value 232—3denotative value 232—3homospatiality 223perceptual equity 228-9space of integration 223, 238system-metafunction fidelity 224,

232Item 134-5, 137, 141-59interpersonal (modal) meaning

buildings 11-13, 15, 17-19, 23-5, 61,65-8,71-2

cities 66film 119,120-3,124-7hypertext 141-55museum exhibitions 32, 38-40, 42-3,

47-8schematic drawings 199-200, 204-7,

209-10statistical graphs 202-3, 210-11television car advertisements 86, 90-91

Interpretive Space (IS) 4, 164, 176-8, 186,188-94,239

see also intersemiotic mechanismsintersemiosis

see also intersemiotic mechanismsadvertisements (print) 163—195biology text 196-219buildings 61,55-79children's picture books 220-246cities 55—79hypertext 4, 152-59Integrative Multisemiotic Model (IMM)

5,220-46film 109-130museum exhibitions 29-54multimodal concordancing 83-108scientific text 4, 203-5, 210-12, 214television advertisements 83-4, 87,

90-5, 95-106

intersemiotic mechanismsBi-directional investment of meaning 4,

4, 164, 176-8, 188,239conjunction/disjunction 92—3Contextual Propensity (CP) 4,164,

176-8, 186, 188-94,239homospatiality 5, 223, 240Integrative Multisemiotic Model (IMM)

220-46Interpretive Space (IS) 4, 164, 176-8,

186, 188-94,239RIM (Relation, Intersection,

Manifestation) 155-6Semantic Effervescence (SE) 4, 164,

176-8, 186, 188, 191-4,239semiotic metaphor 5, 61, 71, 241sychronization 90-1,93Visual Metaphor 4, 164, 168-9,176,240

intertextuality 3, 21, 23, 68intertextual motif 124,125

JanovyJ.Jr. 197,218Jayapal, M. 55Jewitt, C. 1Johns, A. 4,215-16,218Justification 164, 176

Kaplan, E. A. 116Kavanagh, G. 31Keung, J. (57)kinesics 85-95,119Knight, D. 204,218Kok, K. C.A. 157Kress, G. 1, 61, 74, 132, 196, 203-4, 215,

218Kress, G. and van Leeuwen, T. 1,4, 5, 29,

31,44,84,91, 110, 131, 132, 153,163-6, 169, 192-3, 196, 203-4, 218,220, 224, 227-8, 230

Kress, G. et al. 1,29,215,218Kronberg Castle, Helsingor 23Krutnik, F. 114,116Kuhn, T. S. 204,218

Landow, G. 131,132,135language systems

see also intersemiosisAppraisal 32, 38-40, 42-3, 174Colour 68, 72, 141, 152, 173, 234Font 142-3, 148, 150, 152, 155, 173,

234Modality 11Mood 11Position 165Prominence 141, 169, 173

250 INDEX

Salience 165, 169, 173, 177-8, 181Scale 141, 173Size 141, 173

Latour, B. 215,218Lead 164-5, 167, 170, 175-89, 191Lee, T. H. 47Lemke J. L. 1, 4, 5, 29-30, 47, 110,

132, 163, 194, 196-8, 201, 204, 211,215,218,221,229

Lexia 135-7, 141-59lighting 74,119,124,127Lim, B. L. L. 132LoA see Locus of AttentionLocus of Attention 165-71, 178-81,

185-9, 191, 193logical meaningelaboration 185enhancement 185expansion 184extension 185film 120-3

Lombardo, L. 84Liu, T. K. 57Lynch, K. 55, 67-8Lynch, M. 196,215,218Lynch, M. and Woolgar, S. 215,218

Ml 165, 169-71, 173-5, 180-5McDonald, S. 28Mclnnes, D. 110McMurryJ. and Castellion, M. E. 198,

218makeup 124Maroevic, I. 30Marriott Hotel (Singapore) 3,56,61, 66-8,

71-7Martin, J. R. 1,31,32,36,220Martinec, R. 110mathematical and statistical graphs 4MCA see Multi-modal Corpus Authoring (MCA)

systemmetafunctional organization text 84, 85,

86-95,98-9, 110-11, 113, 127meta-language 222metaphor see visual metaphor, semiotic

metaphor and intersemiosisMinistry of Education (MOE) Singapore

4, 131, 136, 137-57Modality see language systemsMood see language systemsMoore, M. 132movement-action-event 86, 88-93, 125,

126multimodal communicative competence

216

multimodal frameworkssee also intersemiosis and intersemiotic

mechanismsarchitecture 68, 77advertisements (print) 4,163-95buildings 2, 60, 62-3, 77children's picture books 220-46cities 3, 58-9, 77film 3,114-18,120-3hypertext 4, 133-7, 143museum exhibitions 3, 28, 34-5schematic drawings 4, 199—201statistical graphs 4, 201-3television car advertisements 85—95see also phase and transition

multimodal software see Multimodal CorpusAuthoring (MCA) system

multimodal concordancer see MultimodalCorpus Authoring (MCA) system

multimodal transcription 3, 83—95, 96,106, 110-11, 112-13

Multimodal Corpus Authoring (MCA) system3, 27, 83-4, 85, 95-106, 110-11, 127

Mumford, L. 55museum exhibitions 3, 28—54Myers, G. 196-7, 199, 215, 218

National Heritage Board (Singapore) NHB47,49

Neal, A. G. 49Noble, S. 72

on-screen space 124,125,126,127O'DonneU, M. 85O'Halloran, K. L. 1, 2, 3-4, 5, 29, 41, 110,

148, 163, 188, 196-7, 201-3, 212, 215,218-19,221,228,241

O'Halloran, K. L. andjudd, K. 98, 106,110

opacity 23OsborneJ. 197-8,219Orchard Road (Singapore) 3, 55-7, 61,

65-8, 71-2, 76-7O'Toole, M. 1,2, 3, 4, 11-3, 16, 27, 29, 30,

32, 44, 55-6, 60-1, 65, 67-8, 72, 77,84, 110, 111-12, 118, 131, 133, 134,142, 143, 145, 153, 163, 169, 173,188-9, 197, 199-200, 203, 208-9,219,227,243

Ove Arup and Partners 16

Palmer, R. B. 116Pavesi, M. and Baldry, A. P. 85,105,216,

219Pearse, S. M. 30

INDEX 251

Peirce, C. S. 207-8, 216 n.2, 219perspective 55, 124, 126, 234

see also graphicsphase 3, 84, 85-95, 95-106, 110-11,

112-13phonology 226photography 44—8Phillips, D. 46Piastra, M. 105Pike, B. 55Place, J. 117Polanski, R. 4, 110, 114, 116, 119, 125,

126-7, 127-8, 127-8 n.lpolitics of buildings 19Pompidou Centre 19Preziosi, D. 3, 55, 60-1Price, D. 44, 46Price, D. and Wells, L. 46proxemics 19, 125, 126, 127PuruShotam, N. S. 47Purves,W. K. 197,219

Ravelli,L. 29,30,31,33reading path 148-52, 157, 168, 178, 184,

187,203-5,210-11,230reflectivity 23representational (Experiential) meaning

advertisements (print) 164—7, 170-1,173-7, 185-7, 192-3

buildings 2, 12-13, 15, 60, 70-2, 74film 94-5,104,119,120-3hypertext 145-6, 148-159museum exhibitions 29, 36-40, 44-8schematic drawings 200-1,206-10statistical graphs 201-3, 211-12television advertisements 89-93, 95-104

rhythm 90, 119Rogers, R. 19roof-tiles 17-18Royce, T. 1,215,219,221

Saarinen 18, 21Safeyaton, A. 56schematic drawings 4, 199-201, 204-10scientific texts

see biology texts, schematic drawings, andstatistical graphs

Scott, P. and Jewitt, C. 215,219sculpture 110Seidler, H. 17,27Semantic Effervescence (SE) 4, 164,

176-8, 186, 188, 191-4,239see also intersemiotic mechanisms

semiotic metaphor 5, 41-3, 212, 214, 241see also intersemiotic mechanisms

setting 124, 125, 126Singapore History Museum (SHM) 3,

28-54Singapore Master Plan 56-7Singapore Ministry of Education (MOE) see

Ministry of EducationSmith, C. S. 31,46software see also Multimodal Corpus Authoring

(MCA) systemAdobe Premiere 4, 83-4, 113HyperContext Web 105OCP 98Systemics 1.0 111video editing tools 109,110,111,113,

118-27Wordsmith 98

soundtrack 84, 85-95, 112, 122-3space 40-4Stables, K. 117statistical graphs 201-3, 210-14Stern, R. A. M. 55Swales, J. 215,219Sydney Harbour 24Sydney Harbour Bridge 21Sydney Opera House 2,11,14—27Sydney Symphony Orchestra 15SykesJ. 15,16,24,27systemic functional frameworks see

intersemiotic mechanisms, languagesystems, multimodal frameworks andvisual images

system network 225

Tag 164-6, 170, 174-6TaggJ. 46Tan, J. H. 57Tan, S. 56, 57Taylor, C. and Baldry, A. P. 84television advertisements 85—106, 112-13texture 17textual meaning see compositional meaningtheatre 16The Straits Times 33, 67, 138-40Thibault, P.J. 1, 5, 29, 44, 46, 83, 84, 86,

87,90,92,93,94, 102, 104, 110,112-13, 118, 208, 214-15, 219, 221,230

This Week Singapore 64Tilling, L. 217n.4, 219topological and typological meaning

198-9, 201-2topological grammar 194Towne, R. 114transition 3, 65, 84, 92-95, 95-106, 110,

113,118

252 INDEX

Tuman, M. C. 131TuskaJ. 114,116typography 222,234-237

see also expression planefont 234layout 234spacing 234

URA Annual Report (Singapore) 5 6—7,61,66-7

Unsworth, L. 1,132UtzonJ. 2, 11, 15, 16, 17, 19, 21, 23, 24,

27

van Leeuwen, T. 110, 228-9Ventola, E., et al. 1,110Vergo, P. 31,36vertically in buildings 21video texts 3, 110-11

see also television advertisements and filmvisual images

see also intersemiosis, intersemioticmechanisms and multimodalframeworks

advertisements (print) 3,163—95biology texts 4,196-219

film 3-4, 109-30hypertext 141-59Integrative Multisemiotic Model (IMM)

220-246museum exhibitions 44—8paintings 110,111-12,118scientific drawings 4,196—219software for analysis see multimodal

softwaretelevision advertisements 83—106

Vasta, N. 84Visual Metaphor 4,164,168-9,176,

240see also intersemiotic mechanisms

Voytilla, S. 116

web 136-7Wee, O.K. 163,168Wee, C. J. W-L 47, 48-9Wernick, A. 187White, P. R. R. 173-4Whyte,W. 61,65

Zago, S. 94Zammit, K. and Callow, J. 1Zar,J. H. 198,219