16.1.02.van den boomen _ metaphor _configurations

8/14/2019 16.1.02.Van Den Boomen _ Metaphor _Configurations

1/23


2/23


3/23


4/23

The problem is that the interace tools are made rom the very

same material as the output products, and both are simultaneously

displayed on the same plane: signs and symbols on the screen. We

cannot read a text document without a word-processing program,and we have no access to our e-mail without the specic interace

o an e-mail program. As Lev Manovich states in his seminal work,

The Language o New Media: [T]he choice o a particular interace is

motivated by a works content to such degree that it can no longer

be thought o as a separate level. Content and interace merge into

one entity, and no longer can be taken apart.4 He shows how these

digital mergers o interace and content employ concepts taken

rom three media domains: print, cinema, and humancomputer in-

teraction (HCI). According to Manovich, this interacial assemblageis not conned to the computer as such; it extends to what he calls a

cultural interace, a general humancomputerculture interace

by which we interact with culture at large.5

The Swallowing Screen

The screen is the plane where the three instances o print, cin-

ema, and HCI come together, and this is probably the reason why

Manovich considers the screen the key element o the modern

interace.6 This is a substantial claim, and Manovich goes even ur-ther by narrowing down the screen to cinema: Rather than being

merely one cultural language among others, cinema is now becom-

ing the cultural interace, a toolbox or all cultural communication,

overtaking the printed word.7

While this might hold true or visual culture at large, it certainly

does not or a PC. Its interace not only consists o a screen, but also

o a keyboard and mouse. These mediating devices even correspond

neatly to Manovichs triad o metaphorical domains: the keyboard

interaces print; the mouse, HCI; and the screen mediates graphic

and cinematographic representations. The screen does indeed re-

mediate cinematographic operations (raming, projecting, zoom-

ing, montage, and so on), but it also remediates print (in the orm

o les, letters, documents, onts) and clickable HCI (cursor move-

ments, selecting, clicking). The three interacial devices are insepa-

4. Lev Manovich, The Language o New Media (Cambridge, Mass.: MIT Press, 2001),

p. 67.5. Ibid., p. 70.

6. Ibid., p. 68.

7. Ibid., p. 86, emphasis in original.

36 Confgurations


5/23

rable and indispensable8 or PC operations, and there is no reason to

privilege one above the other. The screen reassembles all interacial

processes, translating and returning them as visual entities on a fat

visual plane. Hence the screen is just a graphical interace within abroader interace.

Manovich has his roots in lm studies, and this might explain his

inclination to overrate and reiy the screen. However, the tendency

to conceive the screen as pars pro toto or computers, contemporary

media, and contemporary culture in general is not limited to lm

scholars.9 The screen appears to be a general, dominant visual meta-

phor10 or computing, in ordinary speech as well as in new media

studies. This could be explained by the ability o the GUI to swallow

up all other components in its visual representations, thus renderingirrelevant what remains invisible. What you see is what you get, and

thats all there is to get. At least, that is what the GUI suggests.11

Iconic Condensation

Hence the GUI is an interace within an interace, with a strong

tendency to swallow up the other components. And it does not end

here, since the GUI is in itsel a nested compound, consisting o

relatively autonomous smaller GUIs: operating-system interaces,

application-program interaces, and helper-program interacesallwith their own buttons, icons, and other operation displays. Since

these tools are all visually represented, they are subject to the engul-

8. Although we might dispense with the mouse when we learn how to use keyboard

combinations to navigate the cursor. And when suering rom repetitive strain injury

we could use a voice-recognition sotware interace in order to circumvent or at least

minimize the use o mouse and keyboard. But dropping any interace or manipulating

text and cursor would make the computer inoperable.

9. C. Sherry Turkles metaphorical book title,Lie on the Screen: Identity in the Age o theInternet(New York: Simon & Schuster, 1995). In Steven JohnsonsInterace Culture: How

New Technology Transorms the Way We Create and Communicate (San Francisco: Harper,

1997), the interace equals the screen with its visual metaphors. Even in Jay David

Bolter and Richard GrusinsRemediation: Understanding New Media (Cambridge, Mass.:

MIT Press, 1999), the basic principles o mediationimmediacy and hypermediacy

are both dened as styles o visual representation (p. 272).

10. More precisely, this is a metonymical trope, a synecdoche, as it takes a proximal

partthe screento stand in or the whole, namely, a computer or visual culture.

11. Contrary to a so-called command-line interace (MS-DOS, Unix, Linux), where the

keyboard is the dominant interace device and metaphor/metonym. This interace iswaiting or the user to type specic commands in order to let the machine do some-

thing. You dont see a visual approximation o what you will get; you see an empty

space that has to be lled with your knowledge o commands. Here, what you write is

what you get.

Van den Boomen / Interacing by Iconic Metaphors 37


6/23

ing mechanism o any visual representation: in the very act o trans-

lating into visuals, they devour all other components and modes. By

showing, they hide; by translating, they substitute; by representing,

they reiy.This is especially the case with computer icons. These small pic-

tures are, in act, nothing but shortcuts to specic sotware com-

mands in order to yield some result. However, in our daily comput-

ing praxis, this is rendered invisible wherever possible. Instead o

reerring to specic sotware commands, they cloak this, pretending

that they reer to places on a computer (My Documents, My Net-

work, Mailbox), to les, or to an application program.

Sometimes it is not even clear whether an icon reers to a place, a

le, or a program. Take the mailbox icon, or instance. What does itrepresent or the ordinary user? Well, my e-mail o course, would

be the typical response. But what exactly is implied in the concep-

tual shortcut, my e-mail? Does it reer to a specic program run-

ning? Or to a place on your computer, the mailbox, where your mail

resides? Or to specic les, sent to you as e-mail messages? Or per-

haps to all these notions combined? Usually, when your e-mail just

works as you expect it should, these questions are irrelevant. Who

cares? But sometimes these questions pop up, as in the ollowing

story, which shows how these dierent possible reerences may con-verge or diverge and conuse users.

Somewhere around 1997 a riend o mine had problems with her brand new

cable connection. She asked me to have a look at her computer. When she

clicked on the mail icon an empty inbox showed up.

See, no mail. And Im sure I have mail; I orwarded some mails rom my

oce. That man who did the installation yesterday said everything was work-

ingJust click and go, maam. But where is my mail?

Okay, I said, First, are you sure you are online?She pointed vaguely at the wires, still hanging chaotically all over the

place, swirling to her desk. I hoped they were okay, yet, I meant the sotware

connection. There was a network icon on her desktop, represented by a tele-

phone connecting two computers.12 And ortunately it appeared to be cor-

rectly congured; a double-click established the connection with her Internet

12. Over the years, the display o the connection icon has been changed. While it

started with a plain telephone device, it evolved into the more appropriate telephone

modem, and nally into an abstract device, standing or any kind o connection, en-

abled by a telephone modem, cable modem, or wireless network card. Average Internet

users nowadays would perhaps not even understand the telephone as a network-con-

nection icon, being not amiliar anymore with dial-up accounts. A telephone icon

nowadays usually stands or another application: voice over IP.

38 Confgurations


7/23


8/23

process13 is very strong. In act, it is part and parcel o the very unc-

tion o icons, since desktop icons are shorthand or complex ma-

chine processes. This is the kernel o our contemporary graphical

user interaces: translating machine processes into human-readablestable signs, and translating user action into machine-readable pro-

cesses. Icons can only carry out their signiying andexecuting job by

concealing the involved complexities, that is, by representing stable

entities.

In the case o my riend, the mail icon necessarily hides the

complex nested processes it reers to: executing the mail program,

including its congurations or a particular Internet connection, a

particular mail account, and particular incoming and outgoing mail

servers located at a particular Internet service provider. Instead, itrepresents a specic result o the process (received mail) located at a

specic place (the mail box). The same holds true or the telephone

icon: it conceals that it is reerring to the execution o the Internet-

connection program, including its congurations and settings; in-

stead, it represents the contiguous telephone device.14

In short, both icons reer to machine processes to be executed, and

not to a specic state, place, or thing. At the same time, they have

to conceal this and represent stable, ontologized entities in order to

provide a human-readable sign. This concealment goes urther thanjust non-representing; it is an active process that involves specic

labor against representation. I propose to call this depresentation.

We could then say that the icons on our desktops do their work by

representing an ontologized entity, while depresenting the proce-

dural and material complexity. This is the way icons manage com-

puter complexity, the task we have delegated to them. And this is

why we are seduced, indeed compelled, to take icons literally, at in-

terace value.

The process oontologizingis constitutive or interacial computericons, and probably or any humancomputer interaction. Some

13. The analytical distinction betweenprocess and state is elementary or programmers

and sotware designers, but less so or semioticians, media scholars, and social scien-

tists. Though structuralists distinguish between diachronical and synchronical analyses

the latter pertaining more to states and structures, the ormer more to processes o

historical transormation o these statesthis usually yields to dierent studies with

dierent methodologies. The explicit terminological distinction o state and pro-

cess within structures or transormations seems to be absent.14. The telephone icon, in act, enacts an already orgotten displacement. We learned

not to take the telephone icon literally, since we know it stands or using the tele-

phone modem and not the telephone device itsel. In that sense, the telephone icon is

not a metaphor (based on resemblance), but a metonym (based on proximity).

40 Confgurations


9/23

scholars would call this substantializing,15 yet I consider substan-

tializing (to treat as i a substance), reiying (to treat as i a thing),

and anthropomorphizing (to treat as i a human being) as particu-

lar instances o ontologizing. Although ontologizing implies dep-resentation, reduction, and xation, it should be noted that it is a

productive epistemological act o classiying, able to create objects

o knowledge and intervention. In the act o ontologizing, dynamic

properties (e.g., the prize o a commodity, behavior o people, out-

put o a computer) are abbreviated, congealed, and represented as

something stable (as thing, human being, state, or place). These

ontologized entities need not necessarily emerge as isolated; they

may imply or evoke their own external connections. When thus in-

tegrated in a broader discursive ormation, this may result in thehistorical creation o new concepts along axes o knowledge, power,

and ethics.16

Signs: Iconicity and Indexicality

I have argued that by the simultaneous enactment o representa-

tion and depresentation, the computer icon tends to ontologize. It

seems to implode as a sign, condensating reerence, reerent, and

meaning. To urther disentangle this ontologizing and condensating

mechanism, we need to turn to the question, what kind o signs arecomputer icons? Would a theory o the sign be able to explain their

icontologythat strange ontologizing capacity o icons? In order

to fesh out these questions, I will take up C. S. Peirces theory o

the sign. Peirces semiotics seems typically apt to solve the riddle o

the icon, not only because it explicitly addresses iconicity, but also

because his triadic sign analysis enables a connection to the world

outside language.17

What I loosely called reerence, reerent, and meaning echoes

aintly the three Peircian ingredients comprising a sign: namely,

15. See, or example, Werner Rammert, Relations that Constitute Technology and

Media that Make a Dierence: Toward a Social Pragmatic Theory o Technicization,

Society or Philosophy and Technology4:3 (1999).

16. Michel Foucault, The Archeology o Knowledge (London: Tavistock, 1972); Ian Hacking,

Historical Ontology(Cambridge, Mass.: Harvard University Press, 2002).

17. Contrary to Saussures structuralist semiology, in which only two ingredients o the

sign are distinguished: signifer(material embodiment o the sign) and signifed(mental

concept, produced by the dierences in the chain o signiers, arbitrarily connected toa signier). In this linguistic conception o the sign, any notion o reerring to some-

thing outside language is disabled. While this precludes the pitalls o claiming a natu-

ral correspondence between words and things, it also excludes any conceptualization

o possible relations to a world outside human language.



10/23

something articulating 1) a quality, enabling reerence to 2) some-

thing else, and thus producing 3) meaning. As Peirce puts it: A

sign is something which stands or another thing to a mind.18 Peirce

mostly used the terms representamen (perceptual material quality o thesign, the signier), object(thing or phenomenon reerred to, the re-

erence), and interpretant(mental eect, thought, the signied).

A sign, according to Peirce, can only unction as a sign in the ull

dynamic trinity o representamen, object, and interpretant. These

three determine one another in a recursive, nested manner; the third

element (interpretant) always implies the other two, and the second

element (object reerred to) always implies a relation with the rst

(representamen). For example, the sign triad o a red trac light

would be made up by the representameno redness,19 by the object

o a busy crossroad, and by the interpretant o the thought, or meaning:

You are supposed to stop here in order to let the other trac pass.

Firstness, Secondness, and Thirdness

That the Peircian sign comes threeold is no coincidence. The idea

o a rst, second, and third level is ubiquitous in Peirces work, and

he analyzed all kinds o phenomena (signs, logic, science) in terms

o triads. This resulted in a general classication scheme o three ba-sic ontological categories that he called Firstness, Secondness,

and Thirdness. Peirce was aware o his strange penchant or this

numbered triad:

This sort o notion is as distasteul to me as to anybody; and or years, I endeav-

ored to pooh-pooh and reute it; but it long ago conquered me completely.

Disagreeable as it is to attribute such meaning to numbers, and to a triad above

all, it is as true as it is disagreeable. The ideas o Firstness, Secondness, and Third-

ness are simple enough. . . . Firstness is the mode o being o that which is such

as it is, positively and without reerence to anything else. Secondness is the

mode o being o that which is such as it is, with respect to a second but re-

gardless o any third. Thirdness is the mode o being o that which is such as

it is, in bringing a second and third into relation to each other.20

18. Charles Sanders Peirce, O Logic as a Study o Signs, in Writings o Charles S. Peirce,

vol. 3, ed. Peirce Edition Project (Bloomington: Indiana University Press, 1986), p. 380.

See also Robert Marty, 76 Denitions o The Sign by C. S. Peirce. http://www.cspeirce.

com/menu/library/rsources/76des/76des.htm.

19. But also o light and the particular thingness o the trac light. Signs may

combine several instances o Firstness.

20. Charles Sanders Peirce, A Letter to Lady Welby, in Collected Papers o Charles Peirce,

vol. 8, ed. A. W. Burks (Cambridge, Mass.: Belknap Press, 195866), p. 328.

42 Confgurations


11/23

Firstness is thus the level o qualities as such; Secondness, the level

o relations; and Thirdness, the level o general laws integrating rela-

tions in a system.

The beauty o the scheme is that the categories can be reiteratedat any level. Thus, while a sign consists o instances o Firstness

(signier), Secondness (signier relating to an object), and Third-

ness (signied), its Secondnessits possible ways o relating to an

objectis in itsel also threeold. This is Peirces most well-known

triad, based on the three basic relations a sign can have to its ob-

ject: iconic, indexical, and symbolic. The icon (Firstness, o internal

qualities) reers to its object by resemblance or analogyor example,

a portrait o a person, a trac sign with a picture o a hollow in the

road, or the map o a country. The index (Secondness, o external re-lations) reers to its object by an existential or causal connectionor

example, smoke indicating re, a ngerprint identiying a person,

a sundial indicating time. The symbol (Thirdness, o general rules)

reers to its object arbitrarily, by habitor conventionor example,

the fag o a country, the logo o a company, or the alphabet or

phonemes.

The triad can also be reiterated on the level o Thirdness: the in-

terpretants can take on the mode o (First, quality) apossibility, (Sec-

ond, relation) act, or (Third, systematic law) rule/reason. Applied tothe interpretants o trac signs, the sign or a parking lot signies a

possibility, the sign or a hollow in the road signies a act, and the

sign or one-way trac signies a rule. For all reiterated instances o

First, Second, and Third, the principle o recursiveness holds: a Third

always implies a Second and a First, and a Second always implies two

Firsts. In act, just as in calculus: 3 = 2 + 1, 2 = 1 + 1, 1 = 1.21

Peirces triadic recursive classication scheme provides a vocabu-

lary to dissect and understand the working o signs at several inter-

twined levels, as it is scalable and dynamical. It gives an account ohow a sign may be composed by other signs; how it may reer to a

single object or a compound o objects, and how it may do so in di-

erent ways. It particularly accounts or nestedness: in every symbol

(Third) there is also something indexical (Second) and something

iconic (First). For example, a trac sign is always a symbol, as it is

taken up in a system o rules and conventions about orms and col-

ors, but it may be indexically reerring to a hollow in the road, and

21. And since 4 or more can always be decomposed in combinations o 3, 2, and 1, 3 is

always enough, as Peirce rmly stated in A Guess at the Riddle, in Collected Papers o

Charles Peirce, vol. 8, ed. A. W. Burks (Cambridge, Mass.: Belknap Press, 195866), pp.

355367.



12/23

it may do so by an iconic image resembling a hollow. Besides, signs

may changeicons can become symbols; or example, the iconic

picture o Che Guevara becomes a symbol when printed on a T-

shirt, signiying not so much the person Guevara, but radical letistsympathies. Likewise, symbols can be reiterated as an index; a yel-

low letter M is a symbol reerring to the brand name McDonalds,

but as a sign on a pole, it is indexical or the nearby McDonalds

outlet. In short, the scheme accounts or semiotic transormations;

interpretants may break loose and become new signiers, invoking

new objects and interpretants, new signs, with other relations and

rules.

Indexical Symbols with Virtual ObjectsPeirces dynamic triad may be able to account or the interaction

between state (Firstness), process (Secondness), and system (Third-

ness) at work in computing semiosis, especially when it comes to

computer icons. At rst sight, these icons seem to be no Peircian

icons at all, because they are more like symbols. Ater all, they are

arbitrarily assigned; what a specic icon stands ora le, a pro-

gramwe have to learn, nding out by collateral experience. On an-

other level, however, they appear as other kinds o signs. The icons

display small graphics reerring to their object, which can be donein three ways: based on resemblance (iconic), on existential relation

(indexical), or on convention (symbolic). The mailbox icon would

then be a Peircian icon, since its relation to its displayed object is

based on a particular resemblance, a similarity between postal mail

and e-mail. But the connection icon, which depicts a telephone de-

vice connected to a computer, would be more an indexical sign than

an icon, since its representation is not based on resemblance, but on

an existential relation. And the desktop icon or Microsots Internet

Explorer, the branded stylized e (or the icon or Mozillas Fireoxbrowser, displaying a ox curled around a globe), would be a Peircian

symbol, since the image is completely arbitrary and conventional.

From one perspective then, all computer icons can be seen as sym-

bolic signs, and rom another perspective, they can be analyzed as

icons, indices, or symbols. The latter concerns the object as repre-

sented by and in the sign; the ormer situates the object elsewhere,

outside the sign.

In act, these are two dierent kinds o objects. Peirce called the

rst one the immediate objectthe object as represented by the sign

and the second one the dynamical objectthe object in the real

44 Confgurations


13/23

world, that is, as interpreted instance. This object cannot be expressed

by the sign, it can only be indicated, and it has to be learned.22

This dierentiation between objects enables us to fesh out how

computer icons work. As we have seen, desktop icons materially re-er to an act o executing machine code. From this perspective, all

desktop icons are indexical signs: they reer to existential, physical

chains o causation, to machine processes to be executed. Their dy-

namical object is thus code, but two kinds o code: indexically it

reers to machine code, and symbolically it reers to human-read-

able code (e.g., mail, le, program). Hence computer icons are

Peircian indices (reerring to the dynamical object o machine code),

wrapped in Peircian symbols (reerring to the dynamical object o

human code)they are only contingently iconical (when represent-ing the symbol by means o pictorial resemblance). However, in

their signiying practice, they completely inverse this: they appear

as primarily iconic. While computer icons are almost never genuine

Peircian icons, they all exhibit what I have called icontology

reied iconicity. They do so by equating and substituting the sign

with its immediate object o reerence, thus negating its indexical

reerence to the dynamical object. In other words, they enact their

iconicity by hiding their indexicality.

Again, we can see a Peircian triad shimmering through: the im-mediate object as internal Firstness, the dynamical object as rela-

tional Secondness. In order to understand the strange double dy-

namical object o computer icons (machine code and human code)

I propose, in line with Peircian triads, a third kind o object: a uture

object, a not yet actualized objectan object to become, in short, a

virtual object. This object is not represented in the sign, neither is

it completely covered with the indexically executed machine code.

The virtual object reers to what might be actualized by this code,

reerring to something that might be done with it by user actioncombined with machine action. It is not just your e-mail program,

it is that you might have mail and can etch it; it is not just your

browser, it is the world wide web, opened up or you; it is where

dynamical object, interpretant, and action are inextricably inter-

mingled. It is true Peircian Thirdness: organized by systematic rules,

produced by nested Firstness and Secondness, a possible congura-

tion o actualrelations in a complex system.

22. Peirce speaks o the Dynamical Object, which, rom the nature o things, the Sign

cannotexpress, which it can only indicate and leave the interpreter to nd out by col-

lateral experience; see C. S. Peirce, A Letter to William James, in The Essential Peirce:

Selected Philosophical Writings, vol. 2, ed. Peirce Edition Project (Bloomington: Indiana

University Press, 1998), p. 498.



14/23

Signs as Tools: Ready-to-Hand and Present-at-Hand

The proposed extension with a virtual object touches on a mech-

anism that Peirces theory o the sign does not address explicitly. It

concerns the moment that the sign becomes a tool: not somethingto be read and interpreted by a human being, but a thing in the

hands o a user with which things can be done. Heideggers distinc-

tion between tools ready-to-hand (Zuhanden) and tools present-at-

hand (Vorhanden) can be o help here.23 Heidegger points out how

tools exist Um-zu (in order to); their being reers to a work to be

done, in a system o other equipment and work. A tool we use is

ready-to-hand, as it aligns with the activity in our hands and the goal

in our minds. It exists seamlessly with its reerence, its Um-zu, and

does not call or urther refectionwe do not think about a ham-mer when we need it, we just reach or it and use it. But when a

tool does not unction according to its Um-zuor example, when it

breaks downit suddenly appears as a strange separate object. Then

it is no longer ready-to-hand, butpresent-at-hand, open to questions

and knowledge about what is the matter with it. We have to recon-

sider our engagement with the object, repair it, replace it, or change

our goal altogether.

Desktop icons cannot break down as an object like a hammer

might, but the modes o being as ready-to-hand or present-at-handare still appropriate here. Icons are ready-to-hand when they yield

the result we expect rom them. We then routinely take the icons

or granted; we neither think about them as tools nor as signs: they

appear as sign-and-tool24 together in their iconic ready-to-handiness.

The unconscious eyehand coordination makes us even orget that

they are connected to our hands and our clicking action; the screen

seems to be working on its own, by its own icontology. But when

the icon does not yield the expected result, it is no longer ready-to-

hand; it decomposes in a tool and a sign part, now both present-at-hand, raising questions about whether the toolequipment chain

is broken somewhere, or whether the sign was misinterpreted. In

Peircian terms, this raises questions about its reerence to the object,

which seemed to be unproblematically iconic when ready-to-hand,

but exposed its indexicality when present-at-hand.

From this perspective, the sudden turn o the desktop icon rom

an integrated ready-to-hand signtool into a decomposed present-at-

23. Martin Heidegger, Sein und Zeit (Tbingen: Max Niemeyer Verlag, 1979), pp.

6683.

24. Ibid., p. 79. Not be conused with Heideggers notion oZeigzeug, which pertains to

signs as tools, um zu zeigen (tools to reer).

46 Confgurations


15/23

hand object is not a disturbing inconvenience, but an opportunity

an opportunity to investigate taken-or-granted actions and notions,

an opportunity to raise questions and learn something about the

way digital equipment works, how it reers to other equipment andcode, how it represents and ontologizes in iconicity, concealing its

indexicality, how it can ool you, and how you can counter this. As

Terry Winograd and Fernando Flores put it: Breakdowns serve an

extremely important cognitive unction, revealing to us the nature

o our practices and equipment, making them present-to-hand to

us, perhaps or the rst time. In this sense they unction in a posi-

tive rather than negative way.25

Metaphor: Conceptual and MaterialThe vocabulary o ready-to-hand and present-at-hand enables us

to view the computer icon as semiotically and ontologically taken

up in an operational system, where a perpetual signtoolmachine

interaction is at work. Neither Peirce nor Heidegger, however, gave a

ull account o how signs may become tools or vice versa, let alone

how the digital signtool condensation comes about. How can

something be sign and tool in one? Signs are about reerencing and

signiying; tools are about doing and transorming. How can such

an ontological gap be bridged? My claim is that this is achieved bymetaphors.

Metaphors are special signs: they condensate two reerences by in-

corporating qualities rom the one into the other. As such, they not

just reer, but they also qualiy; they qualiy by transerring quali-

ties rom one domain to another.26 This transerence o qualities can

be done by words, but equally so by other media orms (images,

sounds) or even objects; they may stand or and qualiy something

else by blending their own qualities with the qualities o the signi-

ed. Hence metaphors are vehicles o transerence, notably trans-erence between conceptually and semantically dierent domains,

but also between ontologically dierent domainsrom words to

objects or vice versa, rom sound to images or vice versa, rom digits

to images or vice versa, rom signs to tools or vice versa.

In order to track down these principles o transerence, I will ad-

dress below the so-called conceptual theory o metaphor to see what

it can tell us about digital iconicity and transerence. Subsequently,

25. Terry Winograd and Fernando Flores, eds., Understanding Computers and Cognition

(Norwood, N.J.: Ablex Publishing, 1985), pp. 7778.

26. The notion o qualication and objects-as-metaphor comes rom Samuel Gutten-

plan, Objects o Metaphor(Oxord: Claridon Press, 2005).



16/23

I will turn to a theory o material metaphor in order to nally close

the circle o signtoolmachine oscillation.

Conceptual MetaphorGeorge Lako and Mark Johnsons theory o metaphor is based

entirely on a conceptual transerence paradigm.27 Metaphor is con-

ceived o as the transerence o concept x taken rom source-domain

X and transported to target-domain Y, with the result that concepty

is understood in terms ox.28 Metaphors are dened as cross-domain

mappings, sets o conceptual correspondences29 across conceptual

domains, which can be abbreviated in the ormula: Y is X, or target

domain is source domain.30

We could apply this scheme on the mailbox icon. The underlyingregulating conceptual metaphor is then e-mail is postal mail, and

the cross-domain mapping looks like this:

Source domain: postal mail Target domain: e-mail

postbox inbox o mail program

letters, packets messages, attachments

sending and receiving send or get mail button

sorting, disposing distribution to olders, deleting

[postal distribution system] [mail-server network at ISPs]

[delivery by postman] [connecting to mail-server; etch mail]

However, here we encounter some problems. The last two correspon-

dences are indeed at work in the transerence, but these are also ex-

actly what is hidden in the iconic metaphor. They are and are notpart

o the conceptual mapping; they are what I have called depre-

sented, while the other correspondences are represented as such on

the interace.

Note that this depresentation does not coincide with Lako and

Johnsons notion o downplayed or hidden parts o the domains.

27. Dubbed by George Lako as the contemporary theory o metaphor, but also

known as the conceptual theory o metaphor, or the cognitive theory o metaphor.

See Lako, The Contemporary Theory o Metaphor, in Metaphor and Thought, ed.

Andrew Ortony (Cambridge: Cambridge University Press, 1993), pp. 202251.

28. George Lako, and Mark Johnson,Metaphors We Live By(Chicago: University o

Chicago Press, 1980), p. 5.

29. This correspondence is not conned to resemblance or similarity. While most other

theories o metaphor hold that metaphor is a trope o resemblance, Lako and John-son (ibid., p. 151) claim that the perception o similarity can actually be producedby

conceptual metaphor. Moreover, they conceive metonymy, synecdoche, and symbols

as special instances o metaphorical concepts.

30. Lako, Contemporary Theory (above, n. 27), p. 207.

48 Confgurations


17/23

Hidden or unused parts in the metaphor e-mail is postal mail

would be stamps in the source domain, and viruses in the target

domain. These aspects remain unused in the conceptual transer-

ence, they have no cross-domain correspondence, and they are notnecessary to establish the metaphorical concept o e-mail. Yet the

aspects postal distribution system and delivery by postman in

the source domain do have a correspondence in the target domain.

Moreover, these correspondences are indispensable or a successul

transerence. But as such, they are not represented in the icon me-

diating between the source and target domains. Their operation is

delegated to hidden processes, residing in a black box o sotware

and machinery.

We may conclude that the contribution o Lako and Johnsonstheory to the understanding o computer icons remains at the level

o the graphical user interace. It can provide an account o how

the mailbox icon connects and condensates the source domain o

postal mail with the target domain o e-mail, but the model has no

room or the material, indexical operations involved in the transer-

ence: a connection with an Internet service provider, a mail client

consulting a mail server, and the subsequent transport o mail to the

users PC. In schematic orm, it shows an input and output mecha-

nism represented on the screen interace, and a loop mediated bythe metaphorical mailbox icon, which depresents the indexical re-

erences in the black box (Fig. 1).

Usually, when everything works smoothly, this is no problem, the

signtools are so ready-to-hand that we can aord to orget them

as tools and equipment and read them as signs. The user can take

them at interace value. However, reading and understanding the

icon is actually not enough to let it do its job; while this suces or

linguistic or visual metaphors, it will not do or the desktop icon.

The desktop icon needs action to complete its signiying task suc-cessullynot only tool-action by the user, who must at least click

on the icon, but especially action inside the machine and the net-

work. In that sense, the icon is a two-aced Janus: one side is aced

toward the user, who must be able to read, understand, and operate

the icon; and the other is aced toward sotware and machine pro-

cesses. But the user is only able to read the icon as ar as the other

side is concealed and depresentedblack boxed. And this is exactly

what the icon does by its icontology: equating the sign with its im-

mediate object, eacing its indexical relations with its dynamical

and virtual object.

Lako and Johnsons theory o metaphor explains this partially

by ocusing on the cognitive understanding and reading o the icon,



18/23


19/23

in the orm o colored pixels on a screenfickering signiers31

as Katherine Hayles calls thembut it also has a materiality as a

tool that is able to invoke material transormations inside the ma-

chine and the network. It is this latter materiality that is activelysuppressed and depresented in conceptual metaphors.

Hayles proposed the concept o material metaphor in order to

include this notion o materiality.32 She dened material metaphor

as the instance o metaphor where the transerence does not take

place between words or concepts, but between signs and material

apparatus. With this notion, we may be able to open up the black

box in the signtoolmachine network invoked by computer icons,

since it ocuses on transerence rom sign to physical artiacta

transerence that implies more than just the transerence o concep-tual meaning or qualication.

The materiality o a material metaphor does not reside in the ma-

teriality o the sign itselater all, all signs are material, and meta-

phors may take any material orm provided by any medium: linguis-

tic, visual, acoustic, physical objects,33 and so on. The materiality o

a material metaphor resides somewhere else: in the change in a state

o aairs in the material world, invoked by a material execution o

the metaphor. The material metaphor has this possibility o material

execution built in, along with its conceptual signiying power.The material execution is nowadays usually enacted by comput-

ers, but Hayles notes that the occurrence o material metaphor is

not conned to digital machines:

the transer takes place not between one word to another but rather between

a symbol (more properly, a network o symbols) and material apparatus. This

kind o trac, as old as the human species, is becoming increasingly impor-

tant as the symbol-processing machines we call computers are hooked into

networks in which they are seamlessly integrated with apparatus that can ac-tually do things in the world, rom the sensors and actuators o mobile robots

to the semiotic-material machinery that changes the numbers in bank ac-

counts.34

31. N. Katherine Hayles, Virtual Bodies and Flickering Signiers, October66 (1993):

6991.

32. N. Katherine Hayles, Writing Machines (Cambridge, Mass.: MIT Press, 2002).

33. Physical objects as metaphors qualiying something else are, or example, textile

swatches or a suit, or a weaving tree in a storm or a state o mind; see Guttenplan,

Objects o Metaphor(above, n. 26).

34. Hayles, Writing Machines, (above, n. 32), p. 22.



20/23

The numbers in bank accounts are good examples o material meta-

phors that are not necessarily digitally mediated (though, o course,

always computed, in the sense o calculated). These numbers do

not just reer conceptuallyto your money resources, but also materi-allychange the symbols and you really do get materially richer or

poorer.35 Numbers as such are not material, but their banked trans-

erence eects surely are.

Still, we could think o material metaphors that are also crudely

material as a sign, in the sense o being a physical object. Speed

bumps on the road, or instance, are material objects signiying that

you have to slow down, and they enorce this when executed by

traversing them. Visible surveillance cameras are also such double

material metaphors, as they are material objects saying, Watch out,you will be watched, and they do so indeed when invoked. A ma-

terial metaphor can take on any modalitytextual, visual, acous-

tic, perormative, physical and can nestle itsel in any medium or

device as a semioticoperational connection between a sign and a

material event.

Hence a material metaphor does not just signiy, it is also able to

evoke acts in the material world. In other words, it condenses two

reerences, but in another way than plainly linguistic metaphors.

The dierence is that material metaphor blends a conceptual anda material reerence, not two conceptual reerences. It exploits the

condensation o iconicity and indexicality in one signtool. Its in-

stantiation oten takes the orm o a switch or button, as with desk-

top icons, which are visual signs and operational buttons in one. We

might also think o other symbolicoperational signs such as hyper-

links on a web page (conceptually reerring and materially transer-

ring les in one) or a re alarm that automatically connects to the

re department (indicating re and evoking its suppression).

In act, any button or switch can become a material metaphorwhen its signied includes, yet goes beyond, its instrumental tool-

being. Material metaphors thus create two kinds o surpluses, de-

pending on the point o view: a surplus o qualication, on top o its

instrumental indexical unction; and a surplus o material-physical

action, on top o its symbolic representation. In their two-way di-

rection, material metaphors unction as switches between symbolic-

semiotic systems (sign systems, languages) and material systems

35. Specic numbers (as on bank accounts, but also as school grades, telephone num-

bers, or IP numbers) can also be conceived as Peircian signs, connecting their numeric

representamen to specic dynamical objects (respectively, money resources, school per-

ormance, telephone device, hardware device).

52 Confgurations


21/23

(the Internet, banking systems, telecommunication networks, sur-

veillance circuits, but also nondigital networks such as equipment,

electricity, and transportation inrastructures).

The ongoing prolieration o intermediations, induced by thecombination o electricity and digitality, results in more and more

materialsemiotic, technosocial blends such as data-based riend-

ship networks, online communities, virtual companies, and other

dispersed though networked constructions. This can all be seen as

evoked by the productive labor o material metaphors. In this situa-

tion, where material tools become more and more sign systems, and

sign systems become more and more tools, we are not just transer-

ring messages anymore, neither are we just doing things: we are do-

ing things in the world and with the world by transerring messages,thus extending the signtool machinery.

Hacking the Black Box

Compared to the notion o conceptual metaphor, which mediates

between conceptual source and target-domains, the material meta-

phor ocuses on other kinds o transerence. It oten presupposes

and parasitizes on the transerence between conceptual source and

target, but most important is its transer rom material orce into

event. While the conceptual metaphor o e-mail is postal mail canexplain what happens inside the head o the sign-interpreter, the

material metaphor leads us rom the hands o the tool-user through

and behind the interace, inside the black box o digital machinery,

inside the material domain o reerencing to dynamical and virtual

objects.

This domain consists o sotware at dierent levels (client pro-

grams, server programs, computer languages, network protocols,

commands, data packets), but also o hardware (PCs, ISP-servers,

cables, modems, routers,switches). It is a network o equipment inwhich all devices reer and transer to one another. This machinery

is not just a neutral, enabling means or executing the sotware; it

thoroughly rames our way o thinking and doing things, in align-

ing our heads and hands with data objects and virtual objects.

We humans are practically blind in this domain o digital ma-

chinery. We have delegated the relevant intentional acts to code,

which we can execute but not grasp directly. We are simply not

equipped with a perceptual or cognitive apparatus to read the digi-

tal, to read patterns o numbers and iner their meaning. However,

we do have mediated access to the digital, because machinic calcula-

tions and translations can be ontologized and represented to us as

readable signsthat is, as material metaphors. We are able to read



22/23


23/23

a material metaphor is an epistemological hacking tool, enabling us

to hack the black boxes o metaphor, code, and machinery.

AcknowledgmentsWith special thanks to Charles Forceville, Anneke Smelik, Piet

Zweers, and the members o the Metaphor Club or their critical

and inspiring comments. A shorter version o this essay has been

published as Interacing by Material Metaphors: How Your Mailbox

May Fool You, inDigital Material: Tracing New Media in Everyday Lie

and Technology, ed. Marianne van den Boomen et al. (Amsterdam:

Amsterdam University Press, 2009), pp. 245257.
