A thesis project presented to The School of Graduate Studies Nova Scotia College of Art & Design: NSCAD University

by Ian Grivoisin partial fulfillment of the requirements for the Master of Design

ImmersIve experIences &narratIve Interfaces

Copyright © 2012 Ian Grivois. All rights reserved. No part of this book may be used or reproduced in any manner whatsoever without written permission, except in the case of brief quotations embodied in critical articles or reviews.

DEDICATION

This is for my parents, thank you for everything; I cannot begin to contain it in a list.

Thank you to the faculty of nscad: Marlene Ivey, Christopher Kaltenbach, Michael LeBlanc, and Rudi Meyer. I have been exposed to wealth of ideas and given new perspectives on art and design practice. I have benefited from your encouragement, erudition, and flexibility in allowing me to find my own way. You have given me more confidence in my design process, skills in writing, and thinking. May my future practice reflect and make you proud of what you have generously shared. Thank you to David Clark who through generously sharing his time and ideas was an informal teacher and an inspiration. I feel lucky to have to studied art and design in the quirky and culturally rich community of nscad University, surrounded by the living history of Halifax. There is a wonderful support network of staff and student union who made everything this year flow smoothly. As well, thank you to the workshop teachers: Karen Jans and David Peters, and to the Dalhousie School of Architecture for our participation in the DesignLab with and Maria Elisa Navarro. The brief and intense experiences you provided expanded my understanding of design practice.

I am very grateful to Athabasca University for their generosity and dedication to professional development and the opportunity to enrich my life and professional practice through this course of study.

To my classmates, thank you for such inspiration and generosity. The journey was immeasurably more rewarding because of travelling with you. As well, thanks to all my friends and robots for your kindness and support.

A thesis project presented to The School of Graduate Studies Nova Scotia College of Art & Design: NSCAD University

by Ian Grivois

in partial fulfillment of the requirements for the Master of Design

ImmersIve experIences &narratIve Interfaces

Story EXPErIENCE SENSES

1 Abstract

3 Introduction

7 Method & Methodology

13 Interdisciplinary Discovery

19 Aesthetic Interaction & Craft

23 Sensing Poetry

27 Film and the Language of Emotion

31 Learning from Storytellers

37 The Bow & Arrow of Psychology

43 Mobile Light

51 Cell Light

57 Story Light & Story Spaces

71 The Affordances of Imagination

81 Slideshow Zoom

89 Narrative Interfaces & Experience Sketches

115 Conclusion

119 References

123 Image Credits

DéNOUMENT

RESEARCH APPROACH

BACKGROUND

POETIC INvESTIGATIONS

DESIGN ExPLORATIONS

cOntents

abstract

What storytelling methods in the collective memory of film, literature and fine arts can be

uncovered to enrich the practice of design in the digital medium? Each of these disciplines can craft evocative and meaningful experiences, although in the digital domain there is often a focus on content and function that forgets the more qualitative aspects of how these elements are experienced. A more poetic functionalism is discovered through an interdisciplinary inquiry into each unique storytelling vocabulary. Through action research methodology and exploratory design projects, the psychology of motivation, optimal experience and perception is probed. This thesis examines the storytelling qualities of light, digital devices, and interfaces, with the ultimate goal of finding ways to create interactive story spaces that visitors discover and illuminate with the light of their own interest.

Keywords: Narrative, story, storytelling, interface, interaction, immersion, art, craft, design, critical design, film, poetic, experience, digital medium, multimedia

1

IntrOductIOn

all too often in the digital medium, content and function are presented without adequate qualitative

consideration of how it will be experienced. In this context, how can we communicate the value of creative practice to design situations for people to craft their own meaningful experiences of a subject matter? Storytelling is one way to do this, and each medium has its unique storytelling forms. Through interdisciplinary research, what can be uncovered about successful narrative practices that will enrich design for the digital medium, and how can we sketch experiences and craft narrative interfaces that will inform and delight?

The cyclical process of planning, acting, observing and reflecting—all essential qualities of the action research method—will guide this research to explore a variety of disciplines to uncover their narrative forms. This is necessary because the digital medium is still quite new and has its own limitations and affordances yet to be discovered, and the depth of understanding found in established practices still has much to enrich and teach.

This thesis looks to film as an applied media aesthetic that literally and figuratively focuses a lens on the experience and flow of relationships over time through a unique language of emotion. Adjunctive to that, psychology offers a foundation of understanding on which to base more broad human responses, and creative writing offers traditional understanding of the narrative hook and the pull of plot lines to create heightened experiences. The abilities of craft, art, and design to clarify, interpret and intensify (Zettle, 2011) will be utilized to find a deeper poetic understanding. In recounting these research findings, the various disciplines will constantly be placed in the context of design for interaction and the digital medium.

Practice is a way of knowing. Hence, part of this research will involve explorations through design, and reflection through recounting of findings. This thesis embraces the poetic qualities of light and electronic devices as theatres; explores how a narrative can be a unifying principle in designing for the digital medium; and shows, as film practice has discovered, that “what is depicted at any moment should contribute to the over-all storytelling” (Mascelli, 1965, p. 54).

An undercurrent of this research will be identifying the value that art and design practice can bring to finding responses for the problems posed. Ultimately, this research focuses on creating situations for meaningful experience in the digital medium, reinforcing the role of art and design as a culture-forming activity in an evolving milieu.

Art is necessary in order that man should be able to recognize and change the world. But art is also necessary by virtue of the magic inherent in it. (Fischer, 1971, p. 23)

2 / 3

research apprOach

Based on Crank Device for Manipulating Wings, Leonardo da Vinci

methOd & methOdOlOgy

Natural forms are continually modified during growth by their environment. (Munari, 1966, p. 166)

Simply by trying things out you will become involved in an absorbing game (the ancient Japanese word for art was asobi, which also means game). (Munari, 1966, p. 184)

let us consider how best to explore the territory of this topic that reaches across the borders of large

and deep disciplines, each with their own knowledge and discourses. One looks for a methodology that will demonstrate both rigor and research-based critical thought but will also fully engage the aspects of art and design practice that speak with their own language of form—those communicative vocabularies of a sense-experience. As Cal Swann (2002) acknowledges, “visual form is a form of knowledge” (p. 52), and one can expand that idea to include the communicative forms for each of the senses. As well as the language of the sense-experience, it is also necessary to find a research practice that makes allowance for the more illogical aspects of the creative practice, such as the often messy process of enabling an informed intuition to creatively address design needs.

Many of the current theories regarding creative practice find the scientific method lacking when it principally looks for underlying patterns and for universal and repeatable truths, and has the positivist belief that all can be known. Like a cut flower, the role of intuition, emotion, and creativity can wither under the hot lights of logic when they are removed from the holistic synthesis with their situation (Swann, 2002, p. 55). Dissection may not, therefore, be the best way to study living patterns and transformations. Instead, the

method of research that most resonates here is action research, which is a branch of social research employed by sciences that are akin to art and design practice (because they deal with people and systems), such as sociology, psychology, anthropology, ecology, and ethnography.

In Visualizing Research: A Guide to the Research Process in Art and Design, Gray and Malins (2004) describe the practice of action research as follows.

Originating in group dynamics and educational research, the definition of action research varies with time, place and setting; however, action research is situational—intervening, diagnosing and solving a problem in a specific real-world context, for example a school (McKernan, 1998). It requires the co-operation of the ‘inhabitants’/participants of the potential action context, and is self-evaluative with modifications ongoing, where the application of the results is part of the methodology. Its aim is ultimately to improve practice in some way. (p. 74)

One of the possible challenges of this approach is that, even though this thesis question is situated as a proposal for change in a social community, there has not yet been an opportunity for collaborative equity. While there is a community of research peers and thesis supervisors, the main inhabitant of the research ecosystem is the solitary researcher wishing to deepen his own design practice. Even though there is opportunity for feedback and advice, the Master’s level researcher addresses his or her own research question rather than one situated in the domain of a professional client. Hence, the action research process—planning, acting, observing and reflecting—is employed internally

6 / 7

A visualization of a design process overlaid the action research process

as a reflective process, as opposed to sociology where the action research encompasses change from within a larger community. In the end, it is the act of recounting the journey of growth and applying the lessons learned that benefit a larger social community.

This leads to the question of how best to relate this knowledge that Nigel Cross (2001) so eloquently describes as a designerly way of knowing—a “knowledge inherent in the activity of designing, gained through engaging in and reflecting on that activity” (p. 54). Literacy is the ability to read and write, and if art and design employ the language of visual form for communicative purposes, then they must also be able to “read” and “write” through visual form, although it may be an intuitive and emotional understanding they convey rather than a conscious or verbal one. Cal Swann (2002) states, “visual literacy is the same as verbal or audio (music) literacy” (p. 52). However, just because one is able to listen to music (“read” sound), it is not a given that one is also able to perform it (“write” sound). So too, it is with the language of visual form, where the best way to recount knowledge is perhaps to demonstrate it visually, as in “show, don’t tell.” We can look to cinema to illustrate this idea. visual language is often used to show cause and effect, to move the story forward, or create a characterization, just as an actor more often shows emotions like rage with facial expressions, tone of voice, and body language than verbally naming the emotion, “I’m angry.”

Good writing is supposed to evoke sensation in the reader, not the fact that it’s raining, but the feeling of being rained upon. (Doctorow as cited in Stein, 1995, p. 8)

Here we can see the beauty of using action research, and its compatibility with art and design practice becomes apparent. In terms of the four stages of action research, exploration of the thesis question identifies “needs” or the terrain for exploration (plan). It is through the practice of art and design (act) and the recounting of that practice (observe & reflect) that the reflective practitioner is able to perform mid-stream evaluation. This cyclical process will begin to have a formative influence on the practice itself, which is in accord with the action research goals of affecting change in a situational context.

It is this process of learning through doing that will unfold a great deal of richness in the art and design research practice. Ideas will be proposed, attempted, observed and reflected upon, but most of all, they will be unfolded through demonstration. Gray and Malins (2004) point out the masterful example of Leonardo Davinci as a visual researcher. A wide-ranging and eager intellect is displayed with images and text on the pages of his notebooks. His “notion of art—‘arte’—meant skill, while that of science—‘scienta’—meant knowledge, and he considered both to be interdependent” (p. 93). For Leonardo, it is the act of sketching, visualizing, and prototyping that were a way

8 / 9

of visually thinking, as for “designers, the process of building, prototyping and trying things is the decision-making process” (Raney & Jacoby, 2010, p. 36). To further support this idea, the designer and author, Bill Buxton (2007), convincingly demonstrates in Sketching User Experiences: Getting the design right and the right design that sketching and prototyping is an integral part of the thinking process for design exploration. Displays of visual thinking will be a part of recounting this research journey but also evidence of the rigor and process of the visual thinking itself. Writing and the practice of art and design will inform each other.

To wrap up this section on methodology, it is worth mentioning again that this research proposal is flying across many broad and deep disciplines in order to discover and employ the formal languages they use to “write” for sense-experience. It is not possible in the scope of this research to delineate all of the vocabularies used in these disciplines. Instead, the researcher will be like a bowerbird that dives in and retrieves what is needed to make a compelling nest demonstration.

Native to Australia and Papua New Guinea, the male bowerbird builds a thatched structure as a mating display and will arrange colourful and coordinated collections around this bower.

10 / 11

InterdIscIplInary dIscOvery

most art is a type of expressive representation or an abstraction that can represent abstract qualities.

These representations are formed into a composition through an assembly of design elements. For example, the basic design elements of two-dimensional visual design are “lines, shapes, textures, values, and colours that have been arranged to create a unified whole” (Stewart, 2002, chap. 1, p. 10). Each of the senses has an expressive formal language that opens potential channels of communication. This assertion of languages of form based on sense-experience is the foundation upon which this thesis will build and the reason why an interdisciplinary research approach was chosen.

The expressive potential of the digital medium encompasses forms of sense-experience much broader than the boundaries of traditional design practice. The digital medium can be textual, visual, spatial, time-based, and auditory. The digital medium can also be haptic, as it is experienced through devices, and often relies on “astute simulation that plays off our senses and our memory” (Manzini, 1986, p. 23). A simple example is the expected action and response of elements like pushing virtual buttons, but it is further explored in the spatial qualities of information architecture, 3D virtual reality, and the simulated depth of trompe l’oeil visuals and scanned artifacts. Hence, study of the digital medium requires a deeper interdisciplinary understanding of these emotive vocabularies to realize its potentiality.

In their article “Digital Humanism,” the authors Traub and Lipkin (2001) make a case for what they call the creative interlocutor. This individual would become “a navigator of associative trails of thought and resource, who enables others to freely and creatively manage their human interests” (para. 4). The authors describe

how the Internet has dissolved boundaries, creating “a commonality of human expression that crosses disciplines” (para. 2).

This focus on humanism is reinforced by Janet H. Murray (2008) in her prophetic work Hamlet on the Holodeck: The Future of Narrative in Cyberspace. This book eloquently brought the richness of a humanities education and literary theory into an envisioning process for the affordances possible in the digital medium: procedural, participatory, encyclopedic,

Alfred Hitchcock on an Apple Inc. “Think different” billboard

12 / 13

and spatial (Murray, 1997). She continues in a recent book, Inventing the Medium, to outline three principles, explaining that everything coded on a computer belongs to “the digital medium” with its own unique affordances, that all work in the digital medium helps to tune and refine that same medium, and that “when we expand the meaning-making conventions that make up human culture, we expand our ability to understand the world and to connect with one another” (Murray, 2011, para. 4).

Linda Leung, a senior lecturer for the Institute for Interactive Media and Learning at the University of Technology in Sydney, has been working to broaden interdisciplinary knowledge, as is evident in the book she edited, Digital Experience Design: Ideas, Industries, Interaction (2008). Writing in collaboration with specialists in their fields, Leung explores topics such as pedagogy, film, fashion, psychology, and sound. She uses an effective metaphor in her introduction to demonstrate the need for this kind of exploration: “Consulting other communities of practice allows a chair designer to apply techniques for working with leather from the making of saddles, and similarly a web designer to learn from a film-maker about the creation of user experiences” (Leung, 2008, p. 10). She extends the metaphor of leather, wondering about the material understandings of a fashion designer, a shoemaker, an automobile designer, or a bookbinder.

François Truffaut, a renowned film director himself, asked Alfred Hithcock about his process when adapting a novel into a film.

What I do is to read a story only once, and if I like the basic idea, I just forget all about the book and start to create cinema. Today I would be unable to tell you the story of Daphne du Maurier’s The Birds. I read it only once, and very quickly at that. (Coe, 2011, para. 1)

In an article in The Gaurdian about this topic, the journalist Jonathon Coe (2011) pieced together the reason for this almost cursory treatment of source material. Truffaut teased out agreement from Hitchcock by stating “a masterpiece is something that has

already found its perfection of form, its definitive form” (para. 4). Hitchcock responded, “To really convey that in cinematic terms, the language of the camera for the written word, one would have to make a six- to ten-hour film. Otherwise, it won’t be any good” (para. 4). In a way, this interdisciplinary research is similar. It is both drawing from inspiration from “definitive forms” and realizing these ideas through the unique perspective of a single medium.

From these examples, it is evident how much interdisciplinary practice, or translations between them, can be a source of inspiration for the digital medium. One of the fascinating things about sense-experience and hidden languages of form is that they can be layered on top of each other to create deeper meaning where, in filmic terms, all choices revolve around the dramatic need, or what the main character wants and needs (Lancaster, 2011). Speaking of Donald A. Norman’s work in his book Emotional Design, the authors of “Aesthetic Interaction: A Framework” describe what the best works of the digital medium should aspire to:

…if an artifact is to give continued pleasure, two components are required: the skill of the designer in providing a powerful, rich, and compelling experience, and the skill of the user to detect this richness. He notes that works in the fields of art, music, and literature that have stood the test of time are rich and deep so that there is something new to be encountered on each experience with such ’classics.’ (Locher, Overbeeke & Wensveen, 2010, p. 79)

[Above & opposite] The Weather Project, Olafur Eliasson, 2003, installed in Turbine Hall at the Tate Modern, London, UK.

Olafur Eliasson is interested in making space feel tangible. With fog, a large semi-circular disc, and a mirror on the ceiling, he created a unique space for the Tate Modern in which the audience discovers themselves through reflection.

14 / 15

backgrOund

[Top, left] Tree of life, 1854

[Top, right] Side chair, c. 1880

[Left] Trustees’ Desk, 19th century

[Right] Spool Box and Cover, c. 1825-40

aesthetIc InteractIOn & craft

Anyone who uses a properly designed object feels the presence of an artist who has worked for him, bettering his living conditions and encouraging him to develop his taste and sense of beauty. (Munari, 1966, p. 26)

the practice of design can be a political act, as design artifacts can communicate the values and

beliefs of their makers. These aesthetics of use can elevate a conversation in that, even if not immediately appreciated or valued, carefully crafted and beautiful work is respectful of the viewer’s intelligence, sensitivity, and ability to rise to the level of understanding and appreciation. Indeed, if the artist or designer understands the values and beliefs embodied in their work, the very practice of their craft can take on the role of an advocate or cultural activist.

This is inline with the Shaker maxim “don’t make something unless it is both necessary and useful; but if it is both necessary and useful, don’t hesitate to make it beautiful,” that acknowledges beauty as a natural part of necessity and use. The renowned beauty and utility of Shaker artifacts is that they are sincere embodiments of their religious lifestyle, culture, and community values. It is not just the artifacts themselves but the beliefs communicated through them that are beautiful. Simply put, there is a sense of inherent joy in beholding and using them.

Csikszentmihayli and Robinson (1990) performed empirical studies on the reactions of art viewers and found that they sought an interaction with art “not because they expect a result or reward after the activity is concluded, but because they enjoyed what they are doing to the extent that experiencing the activity became its own reward” (pp. 6-7). They described this as an “autotelic experience, that is, one that contains

’Tis the gift to be simple, ’tis the gift to be free ’Tis the gift to come down where we ought to be, And when we find ourselves in the place just right, ’Twill be in the valley of love and delight. When true simplicity is gain’d, To bow and to bend we shan’t be asham’d, To turn, turn will be our delight, Till by turning, turning we come ’round right.

~ The Shaker Hymn

18 / 19

In this Japanese traditional craft technique, individual lengths of thread are dyed at intervals along the woof and weft. The pattern is revealed once the cloth is woven.

Perhaps this is analogous to the craft of the digital medium, where the threads of code manifest their patterns through the weaving of the program.

the goal in itself” in which the viewer had an immersive flow experience. “Flow” is a term the authors use “to describe the deep involvement in and effortless progression of the activity” (pp. 6-7). This aesthetic experience of use and sense of flow hold the same for user-interactions. In a study of aesthetic interaction, or resonant interaction, the flow experience was characterized by “terms such as fun, surprise, delight, engagement, and rewarding” (Locher, Overbeeke & Wensveen, 2010, pp. 70-71).

Should work created in the digital medium not still demonstrate the warmth and touch of its maker? Can the hand of the maker be an invisible guide for interactors, moving them seamlessly through patterns of use? This could be described as feedforward (anticipating use) and feedback (responding and reinforcing use), framed by the idea that “the origins of pattern are inextricably sewn into the fabric of use” (Yanagi, 1989, p. 117). Considered in these terms, a successful pattern design for an experience is when the embodied values of the maker match the needs of the interactor for necessity, usefulness, and beauty, whether those “needs” be conscious or as yet undiscovered.

Dr. Jakob Nielson is a well-known pundit who had a formative influence on the early days of the Internet as it emerged from a primarily textual medium expanding to offer rich media capacities. Through his website, Alertbox: Current Issues in Web Usability, he wielded an influence that focused primarily on necessity and use with little regard for beauty. As early as 1998, Michael D. Levi took Nielson’s usability principles and reframed them in terms of Shaker principles in the article A Shaker Approach to Website Design (2008). Many of these ideas around clarity, organization, and beauty still ring true for current website design practice.

These ideas of how form can communicate values, autotelic experience, patterns of use and aesthetics of use, and how beauty can humanize function, all offer rich potential for craftsmanship in the digital medium. They reveal the underlying ability of art and design practice—as displayed in the title of the 2008 biopic of renowned illustrator and designer, Milton Glaser—to Inform and Delight.

20 / 21

Evening light [Detail], Ian Grivois (2004)

sensIng pOetry

In this reverberation, the poetic image will have a sonority of being. (Bachelard, 1994, p. xii)

In The Poetics of Space, Gaston Bachelard (1994) takes the reader on a philosophic and poetic tour through a

place at once strange and familiar. In this magical visit to the house of our childhood, the contained universe of our first experiences, he calls out the poetic meanings of spaces, objects and images, and explores how they formatively resonate and reverberate with inhabitants (Bachelard, 1994).

In this realm, the tropes of literary figures of speech—simile, metaphor, onomatopoeia, personification, oxymoron, paradox, hyperbole, and anthropomorphism—offer frames within which we may transform our perspective of everyday experience into the potential for the poetic: “Connection to life gives art its vitality… For art to have meaning, commonplace experience must be distilled, re-examined, or transformed” (Stewart, 2002, chap. 9, p. 1). Let us examine some of the ways that these figures of speech can be used in the practice of art and design.

The following passage on industrial design from Hertzian Tales (2005) by Anthony Dunne is enlightening, and it can be equally be applied to the digital medium. This is similar to an idea discussed earlier, where once the demands of necessity and use have been addressed, there is room for more.

The most difficult challenges for designers of electronic objects now lie not in technical and semiotic functionality, where optimal levels of performance are already attainable, but in the realms of metaphysics, poetry, and aesthetics, where little research has been carried out. (Dunne, 2005, p. 20)

In a way, the work of Anthony Dunne, later joined by Fiona Raby, has evolved to embody a paradox. Typically, functional objects could be mapped as to how close they fell to the poles of science or poetics. Dunne and Raby’s work bends these poles together and eliminates their functional utility until they exist in a magical space “somewhere between reality and the impossible.” Their function becomes one of inspiring critical thought (Biennale Internationale Design, 2010). They have carved out a unique space for what they call Critical Design.

Critical Design uses speculative design proposals to challenge narrow assumptions, preconceptions and givens about the role products play in everyday life. It is more of an attitude than anything else, a position rather than a method. There are many people doing this who have never heard of the term critical design and who have their own way of describing what they do. Naming it Critical Design is simply a useful way of making this activity more visible and subject to discussion and debate. Its opposite is affirmative design: design that reinforces the status quo. (Dunne & Raby, 2011, para. 1)

A paradox can be a sophisticated tool to opening space for thought. Like a Zen koan, it prevents the mind from grasping at polar dualities and forces exploration of the much more vast territory in between. Soetsu Yanagi (1989) explores this idea of non-duality and the connection between Buddhist practice and creativity in the chapter “The Buddhist Idea of Beauty” in The Unknown Craftsman: A Japanese Insight into Beauty. Paradoxes like Picasso’s statement “art is a lie that tells the truth” (Quote DB, 2012) or Yogi Berra’s “half the lies they tell about me aren’t true” (Brainy Quote,

22 / 23

2012) humorously arrive succinctly at a deeper truth. Paradox can also be used to fill out the middle territory of ideas mentioned in the Method and Methodology of this thesis. For example, in taking something apart, you lose an important part. The oppositional or contrasting thought inherent in this method is excellent for expanding and exploring ideas, as logical contradiction forces one to step back and have a more open mind to see the meaning. This could be used in the digital medium: exploration vs. security, internal vs. external, suggestion vs. explanation, etc.

Another literary device that can be a conceptual frame for exploration is metaphor, which is “understanding and experiencing one kind of thing in terms of another” (Lakoff & Johnson, 2003, p. 5). In Metaphors We Live By, Lakoff and Johnson (2003) propose that metaphor is fundamental to human thinking and how we make meaning of our world. They describe it as an “experiential synthesis.” Mark Meadows (2003) explains the agency of a metaphor thus: “By placing a metaphor between two ideas, the reader gets a whole new picture. A metaphor is another kind of lens. A metaphor adds information by comparison” (p. 30).

By encoding these messages in literary devices, there is a risk that they will not be understood, but the opposite is also true. Having to work at making connections can reveal a deeper sense of meaning achieved through the act of participation. The audience has to fill in the blanks in their understanding; they are presented with simple puzzles and must engage with them.

One artist who successfully works at a deep level of encoded experience is Olafur Eliasson. He has discovered a rich artistic practice through exploring the nature of perception. One can “read” his work as employing these literary concepts and the form of sense-experience itself to place the perceiver in ambiguous situations where they are forced to objectively examine the nature of the experience they are having. He is not concerned that the interpretation is highly subjective, as he believes that people all still have similar experiences. In the following passage, Eliasson

describes his work for a garden scent tunnel, which was a humorous play on the name of the town close to the project.

In 2000, I created a smell tunnel for a botanical garden close to Bielefeld, Germany, in a small town called Gütersloh (smell tunnel). I had to engage a blacksmith, a construction draughtsman, and maybe most important of all, a landscape gardener who specialized in the scents of plants, because I don’t really know anything about plants. But I had an idea of what I want people to experience when walking through this tunnel. The people working in this garden don’t consider it an art project. And the people who eventually visit is don’t care whether it’s art or not. The experience doesn’t change I hope… For every experience there is a set of rules or conditions, and these conditions can be set by me or by the spectator or by other people. (Grynsztejn, Birnbaum & Speaks, 2007, p. 29)

The poetic activity of critical design, paradox, metaphor, and encoded sense-experience are all similar in that they can affect thought. Thus, how “we think is implicated in what we are able to think and what we are able to think about” (Hills, 2011, 5.1 para. 4). Literary devices are rich with potential ways to analyze subject matter as well as rich expressive forms of communication in themselves. They are also a fun and playful way to explore where we live, in keeping with the example of Bachelard’s house.

Olafur Eliasson’s Dufttunnel (Scent Tunnel), 2000, a project for the Autostadt in Wolfsburg, Germany

24 / 25

Audrey Hepburn in Roman Holiday [Photo] Ian Grivois, 2010

fIlm and the language Of emOtIOn

film has developed a very rich language of emotive form. Through the use of cinematography, music,

sound design, editing, sets, lighting, plot story development, and the compelling force of characters, the director has an orchestra of elements with which to shape the emotions of the audience. In the best films, the flow of multichannel messages can communicate a great deal of information, and it is the dramatic pull of storytelling that unifies this stream of images and events.

In his presentation at the 2008 TED conference, the director J. J. Abrahms talked about the process of discovering what he would talk about there. He remembered a box that he kept on his office shelf and, professing that he was generally not a packrat, he began to wonder about it: why did he keep it and why had he never opened it?

The box was from a magic store that he went to as a child with his grandfather. The humble cardboard box front was dominated by a large black question mark with the name dropped out in white: Tannen’s Magic Mystery Box.

He realized the box brought back warm memories of his beloved grandfather and the experiences they shared, but also memories of his young self so fascinated by understanding the wonder of how things worked. He came to understand that the box represented his fascination with mystery and how, in a way, the expectation of discovering what was in the box was more enjoyable him than what was likely inside.

He went on to use the metaphor of that magic box to show the dramatic value of withholding information, that “mystery is the catalyst for imagination.” As well, he described how, like his memory of his grandfather in the story of the box, the central dramatic need in cinema

is character driven. He went on, moving at a frenetic pace through clips and ideas, to demonstrate some of the dramatic hooks employed in the cinematic form.

In a way, the theatre is a type of mystery box that the audience sits inside and collectively participates in creating magic. The lights are low, the seats do not allow for much movement, and people are asked to turn off cell phones, all in an effort to enhance the concentration that the audience collectively focuses upon the screen. In Hamlet on the Holodeck, Janet Murray (1997) beautifully describes this state that she later characterizes as an “immersive trance” (p. 199).

Narrative is also a threshold experience. As we know from the work of child psychiatrist D. W. Winnicott, all sustained make-believe experiences, from children’s play to Shakespearean theater, evoke the same magical feelings as a baby’s first teddy bear because they are “transitional objects.” The teddy bear provides comfort because the child projects upon it both his memories of the soothing mother and his sense of himself as a small being who can be cuddled and hugged. But though it embodies these strong subjective elements, the teddy bear is also a real object with a physical presence outside of anything the child imagines about it. To the baby it has a richly ambiguous psychological location, shimmering with emotion but definitely not a hallucination. A good story serves the same purpose for adults, giving us something safely outside ourselves (because it is made up by someone else) upon which we can project our feelings. Stories evoke our deepest fears and desires because they inhabit this magical borderland. (Murray, 1997, pp. 99-100)

26 / 27

An author who has been successful in drawing techniques from filmic storytelling and semantics is Scott McCloud. In his books Understanding Comics: The Invisible Art and Making Comics: Storytelling Secrets of Comics, Manga and Graphic Novels, McCloud (1994 & 2006) demonstrates a deft ability to explain ideas visually. One example is the cinematic editing techniques dealing with image juxtapositions in scene cuts, such as moment to moment, action to action, subject to subject, scene to scene, aspect to aspect, and non-sequitor (McCloud, 1994). Comics express themselves as a time-based medium (like film) by how they can use the turn of the page to create surprise. As

well, the way that a cinematographer structures images to tell a story over time is also similar to comics by establishing shots for scenes, over-the-shoulder shots to show relationships, close-ups for drama, etc.

In 1965, Joseph Mascelli’s book The Five C’s of Cinematography outlined key principles during what turned out to be a formative time for the industry. The five techniques he describes (camera angles, continuity, cutting, close-ups, and composition) are still part of the vocabulary of cinematography that is seen in almost every Tv show and movie. Macscelli explains that there are two types of editing in film: continuity editing, which prioritizes the logical flow of the narrative, and

[Above] The wide-screen format proportions (16:9) and the evocative use of mise en scéne, meaning a visual storytelling or all of the aspects that go into the image on the frame, make this image reminiscent of the look of film. [Photo] Ian Grivois, 2012

complexity editing, which probes the depth of a scene by attempting to trigger an empathetic response.

Continuity editing focuses on the visual telling of a scene where cause and effect logically build. It also deals with creating a logical mental space in the mind of the viewer. In terms of an action scene, it can be simplified into four basic stages: an environment is established, a situation is initiated, the action happens, and there is some kind of reaction. Moreover, while continuity editing is about objective plot details, complexity editing is more about expanding a subjective awareness of the moment.

Sound can also be roughly mapped to these two editing approaches. Continuous sounds are called diegetic and deal with literal sounds, whereas non-diegetic sound is more in the subjective realm of music and feelings. Having this subjective and objective division helps to show how film and sound can so powerfully create a narrative trance. However, in practice,

nothing divides absolutely clearly into distinct categories.

The reader may be wondering how these ideas apply to the digital medium. In many ways, the view of the Internet as a textual medium lingers in the minds of those who create digital content, but even text is a visual texture (squint your eyes at this text block and it turns into a grey colour) and thus subject to two-dimensional design considerations. The importance of other channels of sense-experience that are available on the Internet also cannot be denied. In addition to the direct experience of media such as movies or slideshows, the process of moving through the rhizomes

of links on the Internet is a spatial progression through time. Hence, this movement can employ the storytelling techniques of other time-based arts, such as the deep filmic understanding for cutting and juxtaposing images and events.

Hillman Curtis is a web designer who has also become an accomplished filmmaker. Through his book Hillman Curtis on Creating Short Films for the Web (2005) and his website, he outlines with language and a mental process very familiar to designers his progression into this new discipline. He has developed his own filmic language that can be very moving. For example, he created video portraits of people who are not performing any overt action or smiling. You only sense it is video when they blink or you notice their breathing. Situating this video portrait on a webpage as if it is a photograph and part of a static layout composition can be quite arresting. What an intriguing approach this would be for a biography page.

The further one travels into the expressive territory of film, the more impressive the techniques for creating emotional depth become. Each of the many disciplines involved in the process of crafting the cinematic experience is unified under the vision of the director. But while no one director could take total credit for all of the creativity displayed in a film, he or she is responsible for tying together a body of knowledge and practice much greater then his or her own. So, too, the designer of digital media must understand the depth and breadth of expressive forms available in the electronic mystery box in order to realize his or her potential.

[Left] Scott McCloud (1993, p. 74) visually explains the basic methods of cinematic editing and transition

28 / 29

learnIng frOm stOrytellers

according to Janet Murray (1997), stories are a “magical borderland” (p. 100), a liminal space

where our internal thoughts, dreams, and feelings are mapped onto external realities. Stories embody our past memories, the way we make sense of current experiences and recount them to others, and the way we imagine our futures. Stories are a synecdoche for living in that, as simulations, they are at once representative of both small parts and the whole of our experience. When you reach to pin down where storytelling sits, you find that it does not sit well in any one medium. Whispered in your ear by a close friend, dashing across the silver screen, or kept on the pages of old love letters in hidden boxes, stories formlessly abound, and yet, as ephemeral and boundless as they appear, they do in fact have a grammar of form. They employ the vocabulary of sense-experience and flow out across the maps of our human geographies—fractionally small journeys with beginnings, experiences, and endings that rest within greater journeys of a lifetime.

Sol Stein was the editor and publisher for a number of outstanding authors and playwrites of the 20th century. He is also an acclaimed Broadway-produced playwright, novelist, and screenwriter. In Stein on Writing (1995), he argues that the best stories are driven by characterization more than the plot. He outlines five ways to create characterization, as follows.

1. Through physical attributes. 2. With clothing or manner of wearing clothing. 3. Through psychological attributes and mannerisms. 4. Through actions. 5. In dialogue. (p. 56)

These insights reveal how compelling stories are driven through a focus on people. Some of the ideas could also be extended to an interface design. Character is revealed in the look and feel of the interface, which come from an attention to detail and craft. Character is also revealed through the mannerisms of the interaction and interface elements, through how the actions of the interaction satisfy the users’ goals, and through writing that hooks and engages the interest of the observer.

Journalists have encapsulated some of the grammar of effective storytelling. Of course, these can be applied across other media than just writing. Simply by answering the questions who? what? when? where? and how?, we can paint rich pictures. Frank Rose (2011) is a contributing editor for Wired magazine and author of The Art of Immersion (2011). He applied the use of the four “W’s” (and an “H”) in how he recounted the progression of ideas. Rather than just dealing with an idea and citing its source, he first creates a round picture of a person that includes character, situation, why the ideas came about, and how they became important. This creates compelling ideas buoyed by many characters.

Another impressive journalistic storytelling force is Ira Glass, the host of the radio and Tv show This American Life and winner of an Emmy and a Peabody. Glass (2009) talks about the value of anecdotes in his work. There is an element of engagement that hooks someone’s interest, often in the form of a question. These questions do not have to be overt but can arise in the mind of the audience. Why is the house suddenly so quiet? These questions are balanced by moments of reflection that reward an audience for following the narrative motion. The reflective moments reveal greater themes, release the lingering tension of questions, and

30 / 31

prove that the story is going somewhere. As a narrative progresses, it will flip between these moments of motion and reflection.

In his masterful unraveling and telling of the origins of the American sub-prime housing crisis and subsequent market collapse, Glass takes the role of neophyte journalist learning along with the listeners as he interviews experts and people affected (Episode 355: “The Giant Pool of Money”). It is a compelling character-driven story. In the following passage, he recounts some of the techniques used to a class of journalists:

At that point in the story nobody turns off the radio, even though the facts are incredibly banal. It’s because you can feel the motion of these facts. One thing is leading to the next, leading to the next. You can tell that this is going somewhere, that this is a story with a destination…Narrative is not about logic. It’s not about reason. It’s not an argument. It’s not the pyramid structure of writing. It’s entirely about motion. So you say, “And this happened, and this happened, and this happened.” You can’t turn away from it. The way that we’re built to engage with a narrative, in the proairetic code. We feel a sense of suspense simply through the accretion of action, and understanding that lets you manipulate people’s attention in an incredibly useful way. (Columbia Journalism School, 2009, 50:05 min.)

The proairetic code he is referring to is a method of building tension and mystery, much like J. J. Abrams talked about with his mystery box. Glass (2009) recommends simply “starting the narrative motion going, letting the dream begin…we just let the action pull people in.” The term proairetic comes from Roland

Barthes’ description of five codes (hermeneutic, proairetic, semantic, symbolic, and cultural) in S/Z: An Essay (1974). In his book, Barthes deconstructs in minute detail the dramatic progression of a short story by Balzac.

Indeed, Balzac is a prime example of rich storytelling techniques that can be found in literature. Plain text running across the page may not use a lot of visual language in its story telling, but reading alone can still craft deeply immersive experiences. The basic elements of story—character, setting, conflict, plot, and theme—emerge in endless permutations. It is worth considering some of the different twists that can be used in developing a plot in that they would also be possible approaches to apply to the digital medium.

One fascinating plot method that is often used by television and movies is the two-goal plot (Shawning, 2004). Characters start with one goal, often characterized as a false goal, and then midway through the story there is a dramatic shift into a completely new direction. When you watch an episode of the Simpsons cartoon, it is mechanically evident. At precisely half-way into a 30-minute episode, the plot will take a dramatic turn by switching character point of view or conflict. A deep sense of satisfaction—and often humour—can be achieved at the end of the story where, just as the current goal is resolving, the former reemerges for resolution or to cause a reversal of assumptions. This bait and switch, (i.e., withholding resolution and releasing surprise, or reversal of expectations) is a common element of humour and dramatic exposition.

The act of winding, kinking, knotting, tying together, drawing tension tight and releasing slack are all part of progression along the thread-lines of a story. The well-known dramatic arch of Freytag’s plot line—exposition,

1. Story arc2. Experience cycle

(Shedroff, 2001)3. Adoption process

(Duarte, 2010)4. Project process

(Duarte, 2010)5. Inspiring change

(Duarte, 2010)6. Sales cycle

(Duarte, 2010)7. Computer program

(Meadows, 2003)

32 / 33

The story metaphorically rests on the tongue of frog storyteller, by Martha Arquero (1980)

Linear and episodic is probably the plot convention that we’re most familiar with. Events progress with a logical system of cause and effect, or conflict and resolution. It traces the metaphor of a journey, ideally full of many surprises and learning experiences.

Elliptical plots end with a reference to the beginning. They offer a heightened sense of resolution and can reveal unseen plot relationships.

A popular approach for interactive stories is the idea of an expanding tree of choices, or routes branching to multiple endings. Creating a narrative with many endings can be a complex undertaking; a more manageable alternative is to have branching choices that converge.

Two parallel plots can progress and inform each other until they meet at the end. Often, one point of view is maintained for the entire chapter so as not to confuse the reader.

Sub-plots enrich the reading experience with multiple layers of meaning. As well, the networked model of the digital medium seems ideally suited to having stories within stories, where one story can frame smaller ones. This is a similar idea to a magazine article that has a supporting sidebar of information. It is somewhat out of the regular flow but enriches the understanding of the main ideas.

Meadows (2003) identified Edgar Allan Poe’s unique approach to plotting: Start at the climax of the story and reveal how the story arrived there in the falling action. This is often seen in action stories now.

Writers reinforce the reader’s feeling of continuity, setting up significant events with foreshadowing. In an interactive story, people have the ability to easily jump forward and backward in narratives. Indeed, it is the ability to choose the speed and order of progression through a narrative that is one of the primary differences between digital storytelling and the cinema.

PLotS LInES and othEr PattErnS

rising action, climax, falling action and denouement—can not only be mapped onto many different activities but can follow endless weavings. The map is a good analogy for how one moves through the Internet, as Mark Stephan Meadows explained in Pause and Effect: The Art of Interactive Narrative (2003). The street is controlled by traffic lights, signs indicating one-way paths and other directions for behavior; so, too, is the Internet. These rules guide how one can travel, but not necessarily the routes one takes to arrive at the destination.

The word webpage not only references the physicality of a book’s page but also refers to the stories contained therein. We should consider each page of a website as an episode of a larger story we are telling rather than a container we fill. In this way, we will begin to more deeply understand the process of moving through time in the digital medium as an experience and not just a stack of static pages. All too often we only see the page and forget the story. Space telescopes in and out, through the expansion and contraction of nested menus. Search engines and reorganized database archives create new associative stories. The Internet becomes an immense cultural embodiment of the branching neural networks of the individual minds using and contributing to it. The interface itself is really a series of small stories that we read and traverse.

Nonetheless, it is not always easy to uncover and use storytelling techniques to deepen immersive experience. In their chapter “What’s the Story? Harnessing the Power of Storytelling in Film for Experience Design,” Drago, Leung, and Ward (2008) talk of conditions that risk becoming story killers.

Arguably, the potential of the story (even the simplest ones) in web design has not been realized because of the emphasis on pragmatism, task completion, conventions and standards which prioritize the fulfillment of expectations and often demonize the unexpected. (p. 40)

The grammar of storytelling offers a way to distill and transform our experiences. For the creative interlocutor, it offers rich territory for discovery. One can take on the role of a narrator who guides the visitor through a website’s content and the story embodied in its interface. However, not only are we creating this magical borderland for the audience, we are also endeavoring to draw an unexpected story space in the design process itself. We search to overcome the resistance to these more poetic means.

34 / 35

A door knocker in Spain [Photo] Ian Grivois, 2006

the bOW & arrOW Of psychOlOgy

Interactive designs that encourage their audience to play with them often leave users feeling that the designers have genuinely taken their feelings into account. (Bramston, 2010, p. 130)

Our goal is to unlock the design methods of the experiences that audiences find deep, rewarding,

and immersive. One of the bows on which we can draw the arrows of our ideas is psychology—specifically the psychology of games, learning, interaction, and optimal experience. Two of the keys to the nature of the digital medium are that it is procedural and participatory (Murray, 2011). Just as an interface telegraphs what is possible, seeks to engage an audience, and responds to their activity, the interactor begins to write his or her own story of an experience. Perhaps a better understanding of psychology will ensure that their cursor arrow will find the right target.

Control, or the sense of our own agency, that interaction provides is its own reward. It is the simple delight of call and response. Piaget (as cited in Csikszentmihályi, 2006) argues that “one of the first stages of development in children (his Sensorimotor Stages 2 & 3) is the ‘pleasure of being the cause,’ of not being motivated by imitation or enculturation but rather by a more innate, animal joy of power” (p. 125). The most effective interactive experiences tune in to this delight of control and reward the interactor with a response that causes delight. The devices of Apple Inc. tend to be excellent examples of this kind of interaction. This is seen in the “swoosh” sound of emails taking off from an iPhone and the jewel-like interface icons glowing with tactile visual cues of materials that we are familiar with in the physical world.

Psychology’s understanding of motivation taps deep and often subconscious urges. In The Art of Immersion, Frank Rose (2011) exposes many of these psychological underpinnings.

‘The dopamine system operates below the cortex,’ says [Dr. Kent C.] Berridge. ‘It’s activated by simple Pavlovian learning, not so much by the cognitive understand.’ Think of it as an extremely sophisticated pattern recognition system that functions beneath the level of conscious thought. We’re able to act on it, but if we experience any knowledge of the pattern, it comes to us as a feeling, a hunch, rather than anything that we can explain. This learning is an emotional response—one that has nothing to do with logic or linear thought. Which is what makes it so powerful. (p. 267)

Rose also interviewed Raj Paharia, a game designer, who drew insights from our innate hunter and gather instincts as to why we love having goals, collecting, moving up levels, and completing sets. Through these ideas, Rose (2011) identified highly motivating “incentive systems” (p. 176). He then concluded that “anything that invites us to participate and promises some sort of reward can become a game—including, as participatory media proliferate, storytelling itself” (p. 273).

However, it is not only the idea of collecting and rewarding that is intriguing to us, but also our natural desire to search and master our environment. We are spurred by the delights of discovery, uncovering the secrets of the mysterious and solving puzzles. As Joak Panksepp (as cited in vincente, 2012, para. 1)—a distinguished psychologist, psychobiologist,

36 / 37

and neuroscientist—teaches, “seeking” is the “granddaddy” of our emotional systems. Rose (2011) concurs:

This emotional system… makes animals intensely interested in exploring their world and leads them to become excited when they are about to get what they desire… Panksepp identifies our fixation on chance results with a behavior that’s fundamental to animal survival, foraging—or, as he ultimately dubbed it, “seeking.” (p. 272)

Knowledge and understanding of an environment would be essential for a hunter who needs to understand the paths which game travel, how to avoid danger, and where resources can be found. The essential characteristic of yearning to achieve intimate knowledge of an environment is essentially the desire to explore, to test, and to understand it. This undoubtedly translates to how we explore the digital medium.

Whether digital or virtual, any new technological space prompts us to immediately begin to look for the rules that govern what is possible. There is an element of seeking to understand, learning, and memory. Mnemomic practices of forming associations and pattern reinforcement are all types of games that help people to find deeper meaning and motivation to remember. Another fascinating concept that Rose (2011) uncovered was found in the work of the neuroscientist Demis Hassabis, who argues the concept of neural mirroring. Hassabis noticed that the neurons involved in watching and thinking about an activity were the same ones that lit up when actively doing it.

That’s when he saw the missing questions. If the reconstructive camp is right—if our brains snap together memories as needed—why wouldn’t the same brain mechanisms that go into reconstructing a memory also be involved in constructing new ones? In other words—what if memory and imagination came from the same place? (p.285)

This reinforced Rose’s (2011) proposition that “like games, stories are rehearsals for life” (p. 7).

To further explore the role of creating meaningful experiences we can look to Mihaly Csikszentmihalyi (as cited in Leung, 2008), a psychologist who has done a great deal to benefit our understanding of immersive “flow” experiences. The flow state, “a detached state of consciousness” (p. 39) lost in a moment of activity and enjoyment, has certain conditions that could help inform work in the digital medium. The flow experience is promoted by a lack of distraction, challenges that match skill level, and engagement of conscious attention that requires concentration (Leung, 2008). Csikszentmihalyi’s ideas of flow, or optimal arousal, are achieved through a balance between high challenge and high skill application.

A key point to pull out of Csikszentmihalyi’s ideas on creating flow is about creating an environment for an audience that does not have a lot of distractions. A clear example of such an environment is movie theatres where dampened environmental qualities help the audience forget themselves and project into the movie’s story. Csikszentimihalyi talks about how the mind has a limited capacity to interpret sensations in any given moment. When a specific experience

Csikszentmihalyi’s (2008) diagram above demonstrated the conditions of optimal skill and challenge that contribute to a flow state.

demands a high amount of our cognitive load, requiring our skill and concentration, the mind blocks out other sensory input. The sensation is one of pleasurable absorption, immersed in a flow, and it even transcends our consciousness of time passing (Csikszentimihalyi, 2008). This has applications in interface design. Choices about what occurs on the screen should help people attain focus, and support their dramatic needs.

One method creating the conditions for a flow experience is found in understanding psychological closure. Our minds are so attuned to patterns that they often fill in the gaps to form the perception of a whole. If not immediately seen, the experience of closure can often snap into realization and create a pleasurable sensation of suddenly making a connection or finding meaning. The formal practices of art and design are exceptional mediums for using psychological closure.

Freed of the necessity to provide every detail, the artist can convey an idea by using suggestion, rather than description. When the viewer completes the image in his or her mind, it is often more memorable than a more explicit image. (Stewart, 2002, chap. 3, p. 7)

This implies walking a fine balance where there is enough information to interest and engage, but not too much to be simplistic and lack challenge. In some cases, it is desirable to reduce as much as possible the potential for misunderstanding, such as a map for a fire exit route. However, when our goal is optimal arousal and engagement, appropriately using mystery is powerful technique. When we understand the skill level of our audience and match it with a suitable challenge,

we are showing respect for their intelligence and ability. We provide opportunities that will allow our audience the “pleasure of being the cause.”

In finding this delicate balance, it is important to consider the intentional fallacy, which is the “assumption that an author’s declared or supposed intention in writing a work is the proper basis for deciding on the meaning and the value of that work” (Baldick, 2008, para .1). Meaning can change depending on context and individual interpretation once it leaves the control of the author. D. H. Lawrence (1923) sums up this idea as: “Never trust the artist. Trust the tale” (as cited in Baldick, 2008, para .1).

We often aim to please an imaginary average audience, but that really does not exist. Instead, it is more useful to pull back and judge more objectively, we can use the broader human responses found in psychology and sociology as a guide for how an audience might respond. The author Alan Moore (2010) writes: “understanding these broad human responses is much more useful as a tool toward creating… than any consideration of an ‘average audience member’ would be” (p. 12).

Here are some examples of how these broad human responses are applied. It is through an understanding of the visual dynamics of Gestalt psychology that the designer harnesses the power of visual language to communicate. Similarly, it is through an understanding of the psychology of cinema as an “emotion machine” (Tan, 2009) that talented filmmakers can so skillfully touch audiences. Finally, the psychology of games—somewhat touched upon here—turns people into mice hunting cheese in the virtual mazes of computer games.

The above Kanizsa triangle employs the Gestalt principle of psychological closure, whereby the viewer sees two triangles that are not explicitly outlined.

38 / 39

The sculpture of Eros at Piccadilly Circus, London, creates a mysterious silhouette against the evening sky

Sciences like psychology and sociology can help explain how media affect us, but these academic disciplines are not really the creative source. The most effective designers, game programmers, filmmakers and artists are not necessarily well studied in science. Rather, it is through the innate formal knowledge of their specific disciplines—which has a culture and knowledge of formal means within the discipline—that the ability to conjure a desired emotional response emerges. When practical experience and the accumulated craft-wisdom of a discipline is combined with the spark of a talented individual, the risk of the intentional fallacy is somewhat transcended. The creator is able to design for an experience that will roughly translate along broad human responses to an expected outcome.

However, creative activity is not a place to look for formulas and certainties; the most successful designs of a creator are still those deemed so by an audience. As well, testing and the iterative refinement that is part of the process of development can help mitigate the chance of failure.

To pull the many different arrows of thought presented here to one bow string, the reader will notice that the common line of these ideas is that the study and practice of creative disciplines is also the study and practice of the psychology of perception. As creative communicators, we all strive to pierce the centre of our target. We strive to find the optimal tension and aim to navigate the wind-tossed spaces between each other, to arrive at touching the hearts and emotions of others. Not only a hunter and gather, but also a cupid seeking to make people fall in love with our stories. We take our best shot.

40 / 41

pOetIc InvestIgatIOns

Sketches of a projecting mobile

mObIle lIght

a mobile draws itself in space with motion, reconfiguring with a breath of air, surprising and

somewhat life-like in the unpredictable patterns it follows. What if stories were to reconfigure themselves in this manner, creating new associations and poetic reveries? Perhaps this is what happens in our memories naturally. A mobile is an old idea but it has the potential to reassert a new poetic functionalism. Along the path of exploration, research parallels will be found. As well, we will discover how ideas in one context can find their true form in another.

A whimsical idea was spun in the mind of Bruno Munari. Born in 1907 in Italy, he was an Italian artist, designer, and teacher, with a very playful nature. He created many delightful mobiles with a simple means that he called “useless machines.” Watching a mobile turn can inspire one to drift off into daydreams… What if the old technology of a Munari design was married with the modern affordances of technology?

A mobile spins under the blank canvas of the ceiling; small projectors pointing to the ceiling could be part of the design that would not be obvious until the lights were low, until the eyes adjusted. If the light was gentle, adjustable, it could inhabit the liminal spaces in our bedroom between dreaming and waking, and not dominate. Indeed, ideas often seem to emerge unexpectedly from the dark just like this. In the beginning they are often subtle and not fully formed—easily lost like a previous night’s dream. We could program this projector through our computer or mobile device, which would connect with Wi-Fi to automatically upload. It could project an image of your child laughing, deliver a surprise text message from travelling grandparents, or a silent video of blowing candles at your first birthday. The mobile turns and

Bruno Munari mobiles

44 / 45

A visualization of the projecting mobile and an interface for uploading material to database

A prototype and testing of the mobile that projected circles of dots

46 / 47

[Left] Stills from Telematic Dreaming (1992), Paul Sermon

[Above] A sketch of the mobile over workspaces

suddenly your own child is laughing next to your first birthday cake; one wonders if the dreams projecting inside your head each night will be much different than those projecting inside your bedroom.

Amidst these revolving flights of fancy, an idea for a projecting mobile emerges. Designs are visualized and sketched. Materials and means are considered. Prototypes evolve. All the while, research continues, feedback is received, the original idea grows… and research might turn up similar initiatives.

For example, Paul Sermon’s delightful art project, Telematic Dreaming (1992), charts a similar space in the bedroom. Here, rather than the ceiling as the surface being painted by light, it is the bed. Two beds in separate locations record in real time their inhabitant and project him or her onto the bed of their mirrored location. In celibate playfulness, people are able to touch across a great physical distance the likeness of another, touch only with their eyes.

It turns out this is not the first attempt to claim the ceiling over the bed as a creative projection space, and this will not involve the use of mirrors. In the Renaissance, the ceiling was fair game for frescos to climb up the walls and carve out new illusionistic spaces; in modern times this has not so much been the case, except perhaps in the form of a beach poster over the dentist’s chair. The Philips’ Research and Development department created a wonderful exploratory design project called the Nebula. A ceiling projector linked to an Internet database of selected content can be manipulated by your sleeping positions in the bed. Certain body positions were even keyed to activate a video game played out on the ceiling between the couple (Kyffin & Gardien, 2009).

The Nebula project was actually used by Philips as an

example of a failure, because it did not reach the target for return on investment. Indeed, they found that the vast majority of research and development traditionally fails to achieve this standard. However, Phillips turned the “failure” in a success using their concept of design-led innovation and an innovation matrix. They proposed that their research projects did not lead to direct returns because their ideas had not yet had time (in this matrix of innovative ideas) to influence and define new business practices. Those new practices would lead to new markets where the idea would flourish. They pointed, as an example, to the Internet, which was originally designed as an army communication device and later developed into the World Wide Web. The Nebula would transform and flourish for Philips, as a new business model designing for health care. Philips Ambient Experience, as of April 2012, has been implemented in over 300 hospitals worldwide.

Another kind of creative evolution can happen in the work place. Bill Buxton (2007), talks about the importance of passively sharing your ideas among coworkers. He advocates displaying them publicly on a bulletin board where they become quiet seed-ideas that can take root in unexpected ways. What if there was a projecting mobile over each desk? Workers could share the name of the blues song playing in their headphone sound-bubble, an installation artwork by Janet Cardiff that was inspiring their work, or a photo of their cat sleeping in a sink. The images would move and interact with their cubicle neighbors, ephemeral dogs chasing the neighbor cat across the ceiling.

A visitor to the space might not see a monotonous grid of workspaces simmering under flat fluorescent lighting, but workspaces glowing internally from task lighting and computer screens, and a dimly lit

48 / 49

ceiling dancing with subtle interactions of images and emotions. Each worker’s mobile exists as a silent visual reminder for them to find a moment during the day to pause and lean back in their chairs and reclaim their imaginative spark and creative inspiration. The playful story space claimed by the slow moving mobiles might also give the worker pause to look away from the hypnotic gaze of their computer screens.

A personal projection device might one day be small enough to carry with us as in a pen or phone. On the spur of the moment, we can project and share our inspirations into the nearest dark corner, like the current street art practice of temporary laser graffiti tagging.

This design investigation of a mobile with projection capabilities began as a way to explore adding new layers of emotional depth, playfulness, and personalization onto a classic idea. As the project progressed, the ideas projected onto a mental ceiling began to interact with those of others that appeared adjacent—such as those of Bruno Munari, Paul Sermon, Philips’ Research and Development, and Bill Buxton—the project evolved into an exploration of how ideas exert an influence upon each other and might even change their environment to create conditions in which new ideas can thrive.

[Below] © Philips Design, Nebula: design concept creating a virtual/physical experience for bedtime

[Opposite top] © Philips Ambient Experience: Ambient Experience in emergency department for children

[Opposite bottom] © Philips Ambient Experience in hospital cath lab

50 / 51

cell lIght

an artist or designer can reframe what we take for granted to show a new way of seeing. This is the

art of making strange, perhaps a delightful experience in itself, as seeing the familiar anew can also offer a critique of our choices taken for granted. Such a critical practice can cause us to examine the impact of our choices on ourselves, our impact on the environment, and to reexamine our values in light of our fragility.

The cell is the building block of the organism. What is the “cell” phone the building block of? We usually talk of our phones ringing, or having to answer the phone, rather than receiving a call or having to answer a call from someone. It sounds as if the device has a life of its own, your phone is calling. Are “smart” phones really intelligent, or do they even really en-hance our intelligence?

[Unexpected Interruption] The muffled sound of a doorbell was coming from her purse. She reached in and emerged with a phone jingling a tassel and bell, “Hi-yah! Not bad. What are you doing?” Her companion flashed a look of resignation and went back to his dinner.

These intentionally naïve questions are expanded by the psychologist Sherry Turkle (2011), who feels that our devices are teaching us a bad habit: how to be “alone together.” The amount of life-mediation that a computing device affords is proportional to the demands that it places on us to use it. She encourages us to consciously make the effort to relight the candle of personal communication.

Computing devices have made an extraordinary evolution from the room-sized sentries of punch card machines to the black boxes of personal computers hidden under desks and jacked into with screens and keyboards. The electromagnetic spark hidden inside the computer is beginning reveal itself. Screen and computer merge into one device. Screen glass on a

52 / 53

spirit of generosity built-in where they wire-lessly balance their battery charges, lifting the weakest from lifelessness.

For it is an obvious fact that a paper is more fragile than plastic, glass, or steel. In-deed the white paper looks a lot like a burial shroud and has a certain solemnity. But per-haps that delicacy would engender feelings of care from their owners, who would not casually throw away their phone when a new model emerged.

Perhaps the paper skin could be renewed in a way that was more respectful of the en-vironment than the traditional smart phones. Paper, after all, is what we are used to wrap-ping things in. Think of the green paper hid-ing the plump mandarin orange or the silken mulberry fibers in the paper diffusing a Japa-nese lantern. Simple materials can evoke complex and rewarding emotions.

Certainly a paper phone would be im-practical, but considering the reality of such a device can help us peel back the wrappings of our assumptions about the use of technol-ogy. We can make sure that we are raising a lantern that illuminates our communal needs rather than arbitrarily following the tendency of an electronic spark to make us be “alone together.” Making the familiar become strange can help us gain critical distance and new understanding.

mobile or tablet device now spreads to the

very edge; and devices themselves seem to

glow and pulse with a life of their own. In the

dim light, our phone now casts its internal

light on our faces. The cell phone becomes a

shining beacon that spotlights us in a crowd

and declares our connection to the com-

puter’s spark.Is it possible to find the candle of human

warmth in the spark of an electronic box?

First, we might look at the materials. Plastic,

metal, and glass are not warm materials like

paper. Then we could look at the behavior.

Rather than using light to send our minds

and voices to a distant space, perhaps we

could use it to bring us back to the moment.

A phone made out of paper could begin to

glow and flicker when it sensed another pa-

per phone was near. In their candle-like be-

havior, they could have an invisible dialogue

like a wind passing through the room. Their

glow would signal a like-minded person, one

who also was a member of the paper phone

community. Rather than sending our atten-

tion elsewhere, the paper phone would sig-

nal a sympathetic mind was near.

When paper phones are piled up togeth-

er, perhaps they could pulse as if they were

having a dinner conversation, and serve as

role models for human behavior. Perhaps

these communal phones could even have a

54 / 55

A group of paper cell phones have a communal love-in

56 / 57

A visualization of information and its embodiment, in the form of cube device

each day is regulated by the timeless rhythm of the rising and setting sun. In the modern age, we

extend the waking story of our day with electricity, with electromagnetic currents flickering in containers where once there was only the possibility of flame. Rather than just illuminating our daily activity with these lights, it is also intriguing to ponder how these lights are used to tell stories, and to consider the metaphoric symbols of these illuminated story-spaces.

There is warm community to be found around a campfire, which can so hypnotize us with its dancing light. The crackling embers, the smoke, and the heat resonate deeply within our shared human history. Perhaps a campfire also uniquely warms us to conversation. As well, one finds warmth in the amber light of candles, lanterns, and incandescent bulbs. This is in marked contrast to the often-cool light now emerging from our televisions, computer screens, and mobile devices; or the greenish light cast from fluorescents over-illuminating our office cubicles.

As for cinema, it is hard to classify the character of the luminous path travelled by a projector’s light as it descends to the screen. Perhaps it has as much varying character of emotion as the light that passes daily through the windows of our houses. The window’s light and shape is projected across the room. That light draws volumes, pinpointing highlights and casting shadows in our living spaces. How many memories do we cherish that are lit by the windows of our kitchens, living rooms, and bedrooms?

However, rather than a dark room that is illuminated by light spilling from windows, the cinema is always lit from within. The isolation tank of cinema, with its darkness and etiquette, all contribute to deepening our concentration so that we can project ourselves down the path of cinematic light and into the story on the

stOry lIght & stOry spaces58 / 59

A visualization of information revealing its underlying forms

screen. In his emotionally impressionistic film, The Tree of Life (2011), director Terrence Malick often takes the audience back to a zero point of cinematic light with a luminous rainbow flame shimmering on screen. It hovers in a dark void. It is a reset, a return to the origin of story.

Between black and white you can find shades of gray, but also by the prismatic separation of white you arrive at the rainbow. The author and designer Kenya Hara argues in White (2009) that the colour does not exist at all; what we know as “white” always has a subtle hue. Perhaps, too, there is no absolute darkness as our eyes invariably see meteors crossing our retinas or the mind invents its own visions and diversions.

As the cliché goes, the eyes are the windows of the soul, but one wonders if there exists another flame inside us. Perhaps it is a flame of inspiration or of emotion that illuminates our thoughts, memories, and dreams, or a light projected onto a stage, as director Wim Wenders delineated in his 2011 documentary, Pina. In movies, and perhaps in our own heads as well, this light defines a story space in which we are both observers and participants. In the space of a theatre, the audience and the performance exist in the same space. Electronic devices are like personal theatres where we can begin the performance on demand.

Those modern electronic devices (screens, phones, and tablets) seem almost to be made of light as they extend to the edges of their containers. They are glowing boxes that capture our attention. It is intriguing to consider what lies beneath the screens. What if that lightening in a bottle happened to illuminate a physical idea that then cast its shadow like a silhouette, or a shadow puppet, onto the back of the screen? What if, when you are looking at an ordinary Google search screen, the subject of your search began moving like a shadow play behind the surface of your screen?

One way to examine the mediated experience we have through our devices is to consider more carefully the idea of a box. For if architecture is about enclosing air, then perhaps digital devices are about enclosing attention. These device-boxes are really story spaces, much like pre-cinema machines. Before taking on its

Still from Tree of Life

Still from Pina

A papercut is used to visualize how the screen could be used to animate a shadow play, a second story beneath that of the story of the information.

60 / 61

A visualization of information lit by the internal light of human interest

current lascivious meaning, a peepshow was a mystery box that travelling performers would carry around. For a price, one could satisfy their curiosity and gaze into an internal display or watch a primitive slideshow. From the three-dimensional renderings of a victorian stereoscope to the whirling animation of a zoetrope, we have long been fascinated and delighted by looking into story spaces. The sense of wonder embodied by rudimentary mechanical devices or our natural delight in toys and games, are all familiar starting points for exploring how we mediate the digital medium with electronic boxes.

During the Renaissance in Europe, there was a tradition of the cabinet of curiosities, or wonder cabinets, which seems to reflect upon this idea of boxes. These were basically collections of unusual and eclectic objects displayed in finely crafted displays or exhibition rooms. The abundance and unexpected juxtapositions of objects found in these collections can be compared to plethora delivered through visual database results, like those seen in Google Image search results. The intricacies and delights of discovery found through use-experience of a well-made cabinet of curiosity also reflects the playful joy that a finely crafted website interface can provide.

Just as a cabinet of wonder reflects the personality and tastes of its creator, there are similar reflections of self in the digital medium. As we move through the Internet we leave evidence of our passing, which is either intentionally created or otherwise. This evidence of our identity is locked inside theses digital boxes. But what if we could turn the box inside out? What if the device itself became a physical embodiment of our digital presence, rather than just a container for the ephemeral sparks? Perhaps the sudden physicality of our virtual presence would offer a moment of critical reflection about the nature of our digital activities.

This question of self in a digital context is intriguing. A box has a sense of mystery about it; likewise, other people are infinitely mysterious. How much soul can you really see by looking into the windows of the eyes? Perhaps our head is a box of sorts. We can imagine it as a peepshow; and when you look in, you realize you are looking into a small room with a mirror. You look

Engraving from Ferrante Imperato, Dell’Historia Naturale, Naples 1599

Cabinet of Curiosities, Domenico Remps, c. 1690s

62 / 63

A visualization of a virtual personality and presence made into a physical device

in the mirror and realize that you are actually looking through the eyes of a painting. It is not just that one eye of this painting really does follow you as you look around, but you are actually seeing through a beautiful representation inside the box, because a painting is a representation of a personality. The beauty of self could be seen through the eyes of Sandro Bottichelli’s venus, or perhaps this internal vision is through the perspective of Amedeo Modigliani’s representations of ideal of beauty. What if we really could look into people’s eyes and see the avatar they construct for themselves, or their true representation, or even oneself as perceived by them?

One of the great affordances of the digital medium is a sense of agency, that the interactors can direct their own movements and be guided by the light of their own interests. The interface designer is, in a way, helping people to exercise their own capacity. There is a little boost to the ego when an action is performed and the expected response actuated. Increasingly, the digital medium teaches the interactor how to construct within the medium itself. We gain confidence in ourselves, become capable and “handy” around the digital house, and perhaps even begin to think our greater capacity in the digital medium as reaching an adulthood of sorts. In a way, the digital medium is an experience constructed by our own attention. It is a shimmering reflection of us, our light and perhaps some of our darkness. What if we could open these devices and see our own flame inside?

So much of our story-spaces are in the domain of some kind of light. It is worthwhile to consider the qualities of this light, just as it is worthwhile to consider metaphors of light and the story space it defines. In so doing, the storytellers of the digital medium may be better able to craft for those immersive experiences that deeply move others, and to find the light of inspiration shining in the windows of our own personal theatres.

various representations of personality gaze out of paintings

A model zoetrope becomes a representation of self and time. A zoetrope is a pre-cinema device that simulates the impression of animation when looking through the slots because of our mind’s capacity for psychological closure and persistence of motion.

64 / 65

66 / 67

68 / 69

Inside, the face of a painting is revealed. You see through a “reflection” of yourself, while looking through an eye of the painting.

70 / 71

Ingres found inspiration looking back in history while Delacroix charged forward into the unknown.

[Far left] Detail of Bathing Woman (Baigneuse de Valpincon), Jean-Auguste-Dominique Ingres, 1806

[Left] Detail of Liberty Leading the People (July 28th 1830), Eugène Delacroix, 1830

“Brasília was one of the modern movement’s most extraordinary achievements, and as such was despised by the situationists” (Sadler, 1998)

[Far left] A situationist psychogeographic map: Life continues to be free and easy, Guy Debord, 1959 (Sadler, 1998)

[Left] A model of the city, Brasília [Photo] Ian Grivois, 2010

the affOrdances Of ImagInatIOn

What is possible? This is a question that rests at the heart of every revolution—the knowledge

that there can be something more, something better. This yearning struggle against the status quo appears in many social movements throughout history. By looking at these historic patterns, particularly in the French intellectual movement called Situationist International (1957-1972), we will find methods to reexamine our current life and visually test these on common Internet memes. The contrast of “virtual reality” is rich territory to explore.

The struggle between looking back in history for inspiration and cutting free for new inspiration can be seen in a conflict from art history that took shape in the 1800s. One pole of thought can be seen in the art of Jean-Auguste-Dominique Ingres (1780-1867), who was considered a neo-classicist. The clarity of his precise drawing style matched his rationality of his art that dealt the classic themes of painting: portraits, allegory, and history. His antithesis took shape in the form of Eugene Delacroix (1798-1863). Besides his innovations in developing colour theory for painting, Delacroix was a rebellious romantic. Rather than the clean and rational line of Ingres, passion, movement and dramatic themes characterize his work. His painting of Liberty leading the people captured the romantic aspirations of the French revolution, regardless of its violent conclusions.

This idea of revolutionary frustrations building to a crisis point found an echo in France during May of 1968. Wildcat union strikes and student riots created a brief social explosion, stemming from the Situationist International led by Guy Debord. This movement itself originated from two sources: the French Lettrists, a movement similar to Dada and Surrealism, and the Imaginist Bauhaus, a reactionary Italian movement

revolting against the rigid ideas of modernism. Just like the dichotomy between Ingres and Delecroix, the ideas of rational modernism and the Situationist International form a counter-point. Debord was a Marxist theorist, writer, and filmmaker whose book Society of the Spectacle (1967) outlined a path to reclaiming creative awareness. He outlined his goals by writing, “Of all the affairs we participate in, with or without interest, the groping search for a new way of life is only the aspect still impassioning” (Debord, 1955, para. 1). The movement contended that corporate and state interests had hijacked society and that lived-experience was exchanged for the influence of a seductive spectacle. Desire had been transmuted into consumption.

The Situtationist International movement advocated two techniques for people to rediscover their passion and personal awareness. These were the détournement and dérive, which translate as “diversion” and “drift.”

Détournement was a way of taking popular culture and subverting it through satire. Popular culture is re-imagined by subversive juxtapositions. “The discoveries of modern poetry regarding the analogical structure of images demonstrate that when two objects are brought together, no matter how far apart their original contexts may be, a relationship is always formed” (Debord & Wolman, 1956, para. 5).

The dérive was a way of re-imagining the city, a wander with no goals other than to follow what attracts and to avoid what repulses. It was designed to create a new picture of the environment we live in. The same motivation is seen in paths of desire, which are routes defined by people cutting corners when sidewalks do not follow the most direct route.

The new psychogeographies originating from these wanders and techniques of diverting corporate visuals

72 / 73

resulted in artistic works that subverted social norms. The ideas of the Situationists would appear in the contemporary work of Marshall McLuhan and later in the culture-jamming designs of Adbusters magazine.

The dérive and détournement are methods that can be applied to modern concerns. Like the modern city, the Internet is a type of psychogeography that embodies our desires. Also like the modern city, the Internet is just as susceptible to the corrupting forces of corporations, seducing users with a spectacle that replaces lived-experience. This is even more evident when we examine how computer users interact: physically isolated yet connected through devices. Surfing becomes a type of wander, though we perhaps have lost our consciousness of this process. Ubiquitous computing is now taken for granted. The memes, “those behavioral and cultural systems learned through experience and imitation” (K. Barber [Ed.], 2007), are so natural that we no longer notice them. Metaphoric icons of the desktop have become part of our natural cognitive flow to accomplish tasks on the computer.

We are told that this is a ‘metaphor’ for a ‘desktop.’ But I have never personally seen a desktop where pointing at a lower piece of paper makes it jump to the top, or where placing a sheet of paper on top of a file folder caused the folder to gobble it up; I do not believe such desks exist; and I do not think I would want one if it did.

~ Ted Nelson (as cited in Meadows, 2003, p. 32)

So let us employ the methods of the dérive and détournement on the memes of the Internet. By bringing these icons of the virtual world along on a wander through the real world, we can re-imagine our environment in a new context. We will look at five different aspects of computer-mediated interaction.Through the Situationist techniques of détournement and dérive, we are able to retake our imaginative landscape. We can detour together from the expected path. Corporate interests, a cluttered visual culture, and warning signs are subverted as agents that deaden our minds. Instead they become playful material to daily renew our sights seen and sites travelled. Debord’s idea that the Situationist’s practices would be like a potlatch (1955), giving gifts to others, reveals the affordances of imagination.

The metaphors of a computer adopt spatial and visual cues that communicate what the programs offer. When an arrow turns into a finger, we know that we can click. By keying into our real world sense-memories, we know what direction to go and where to click.

74 / 75

Google is the dominant search engine on the Internet. For decades, the company’s site has had an “I Feel Lucky” button for surfers who are confident that the first search result is the one they want—it sends you directly to the result rather than showing all options. The “I Feel Lucky” button is the virtual equivalent of leap-before-you-look. Here, luck and searching take on a new meaning through a new context.

With the simple addition of a close button, we seem able to clean the clutter of our real environment just as easily as our virtual. The ubiquitous warning signs and billboard advertisements can be closed with a touch… if only it was so easy.

76 / 77

Facebook, the dominant social destination for nearly one billion people globally, is a site where people connect and share. Facebook’s “like” button records and shares our interests. They are votes, reinforced paths of attention, but here it is the camera that records approval rather than a corporation.

Email is now one of the most common ways that we communicate with each other. It has in many respects supplanted the personality, character, and warmth of the written letter. Here we use a simple sight gag to throw this idea into relief. One wishes there was an equivalent to the

“beware of dog” sign for spam.

78 / 79

desIgn explOratIOns

slIdeshOW ZOOm

When the eye encounters something it does not expect, it will linger, consciously or unconsciously

trying to unravel a visual puzzle. One such puzzle is tricking the eye with an optical illusion of three-dimensional space on a flat surface. This technique of trompe-l’œil has been seen since Roman frescos. Dealing with this illusionistic space, or at least referencing it, is one of the fundamental preoccupations of art and design practice and helps a viewer to project him- or herself onto the stage of a flat surface or screen. It even gives psychological cues as to how an interface should respond, such as knowing a button should be pushed.

Through considering these spatial cues, we explore a new way to form relationships over time, with the goal of creating a more immersive storyline for viewers. Rather than the typical way of thinking about slideshows, which tend to move in a horizontal orientation of next and previous, we explore Google Maps’ navigation that dives into the picture plane. As well, we will investigate using cinematic techniques to help slides better relate to each other. The following ideas will help inform the conception of this interface.

It is important to consider the question of resolution, or degree of detail, when dealing with zooming and magnification. For example, a digital photograph on the computer has only a finite amount of information in the file. Zoom out, and you will start to lose visual information as the picture shrinks; zoom in and, after the maximum resolution is reached, the image will start to blur into pixel patterns. With fractal images you do not have this issue. Since fractals are determined by math and can be infinitely travelled into with each new magnification level, you always have the same resolution of detail.

[Right] Examples of the typical horizontal navigation patterns of a slideshow interface

[Opposite] Screen capture of the Google Maps interface

SPATIAL CUES

Light and shadow

Overlapping

Converging lines, perspective

Atmospheric colour and contrast: Objects receding into the atmosphere become lighter, take on the colour of the atmosphere (often bluish), and have lower contrast

Relative size and position: distant objects tend to be comparatively smaller and closer to the horizon

82 / 83

Frank Rose compares the vast encyclopedic nature of the Internet with a fractal, and proposes that this quality is changing the way stories are told (2011). He uses the example of George Lucas who developed his Star Wars environments, characters, and storylines with much greater detail than would ever be shown in the movies. This provided avid fans ample opportunity to explore to the magnification level of their interest. This technique of over-developing the resolution of your story is a rich starting point to remember. It helps the storyteller to find those poignant characterizations that bring characters to life and evoke places that seem familiar.

To tie these two ideas of travelling in depth with frac-tal-like resolution, we can look at the 1968 short film The Powers of Ten by Ray and Charles Eames. It begins with a couple in a New York park having a picnic and zooms out to the infinite limits of space and imagination. From there we travel back to the park scene and zoom into the very cells of the man’s hand. We travel to the limits of our imagination again, but this time into the energy of an atom. The distances that the Eames’ film described were dramatic, jumping magnitudes by a factor of ten. This matched the scientific theme of the film but is not easy for us to relate to. We can look to cinematic technique as a way to deal with our own familiar space.

A generation later, in the 1992 documentary Visions of Light: The Art of Cinematography, William Fraker relates a compelling anecdote about his cinematography work in Rosemary’s Baby with the director Roman Polanski. They crafted a scene where the actress Ruth Gordon was partially obscured by a doorframe when she answers the phone. Later, Fraker observed the theatre audience craning their heads to the side as if trying to see around that corner. This idea of obscuring is a useful dramatic technique to pique curiosity.

[Opposite] Screen captures from http://www.powersof10.com, a web version of the Eames’ movie

[Right] Examples of the infinite mathematical space of fractals

84 / 85

In an interface, a little mystery and challenge can help to create more engagement for the interactor—one of the conditions in creating a “flow” state as discussed earlier. The interface designer must balance the use of this technique so as not to cause frustration or disorientation (unless getting lost is part of the desired experience). Many of our favorite experiences as travellers have a sense of freedom and wandering. We end up at a place that was not planned and therefore is completely unexpected. There is no reason why this could not happen online as well. But, this is somewhat wandering from our topic.

The idea of obscuring, also commonly called blocking, can be used to create interest but also to create depth. There are many conventions in cinematography about how to maintain a logical spatial orientation for an audience. To create dynamic compositions in space, it is useful to consider the qualities of the different performance stages. In a theatre, the space tends to be shallow and wide, which supports horizontal and diagonal movement. In television, which is often the most similar in proportion to computer screens, the screen is narrower. Television tends to support dynamics in depth, and blocking is useful to indicate depth and spatial relationships. You see this in the very common over-the-shoulder shot, where the back shoulder and the head of a character block part of the screen and frame the face of the person being spoken to.

Often, all that is necessary is visibly assigning the planes of a foreground, middle ground, and background. These “grounds” can exist in space as if they were floating, stacked upon each other in depth. Another way to reinforce this sensation of depth is to have the plane in focus sharp and the other planes blurred.

[Above] Plans for Zhang Heng’s water driven armilliary sphere

[Top & top opposite] Su Song (c. 1020-1101). Song dynasty star maps.

The example slides of this design investigation apply the above ideas. The context for applying the ideas is the story of Zhang Heng, a scientist and artist of ancient China. The interface will zoom into the space of his timeline and accomplishments but will also zoom into the illusionistic space of the computer screen. Overlapping planes, size relationships, and selective focus are applied to give a greater sense of story. Seeing vaguely what is coming and what has passed is also like the writer’s technique of foreshadowing and flashback. They tie each frame of the slideshow together in a relationship that is different than the typically unrelated images.

Another cinematic technique applied to this slideshow deals with transitions between images. A film editor will often use a visual motif to smooth the idea of a transition from one scene to another. In the case of this slideshow, there is the repeating motif of a circle that visually echoes from slide to slide and telescopes into the picture plane. In film, this is called a match or graphic cut.

Even though this is a slideshow of still images and not an animated video or 3D space, this interface considers the progression of time and relationships formed over time. There is a sense of procession and storytelling as the viewer progresses.

Where can the exploration go from here? Due to the need of limiting the scope of this thesis, the research does not look at virtual reality, immersive gaming experiences, or motion graphics. They are related but are large topics that are already known to have storytelling elements. As if reflecting pre-cinema techniques, this interface exploration deals more with the relationships between still images rather than speeding up to the point of motion. Motion design over

time or virtual space will have its own vocabulary of storytelling.

One area that offers a lot of potential for interface design and storytelling is the use of parallax, which is somewhat at play in this slideshow when the slides are flipped through quickly. Imagine a bus as it travels down a neighborhood road. The passengers gaze out their windows; they notice that the trees on the boulevard move past faster than the houses. The houses pass faster than the backyard garages, peeking out with the passing of each driveway. Space is drawn through a parallax motion and by other blocking techniques mentioned here. Recent innovations in programming for the web, such as html5, css3 and Javascript libraries, have increasingly made it easier to design for these kinds of motions and transitions. We now travel through stories with the mouse scroll wheel or by hand gestures moving across a touch screen.

Indeed, the potential of current web programming languages suggests that moving transitions, rather than distinct still pages, will increasingly animate the web. Along with the other ideas explored here, these new affordances will help people to step across the window plane of their screens and project themselves into the story-spaces of the digital medium. The mature ability of cinematic technique to visually tell stories over time offers boundless potential to inform this new relationship.

Zhang Heng on a Chinese postage stamp

86 / 87

[Right] A mock-up of the zooming slideshow within an hypothetical interface for the International Herald Tribune

88 / 89

narratIve Interfaces & experIence sketches

the process of gathering ideas through research and formulating new ones often feels more like

stockpiling resources and planning routes on the map than actually undertaking an expedition. It is the doing, the making, the action, which forms another type of knowledge and experience. It tests the researcher’s foresight, stores of ideas and resources, and ability to deal with the unique problems encountered along the way. Both reflection and action is needed to arrive at a new understanding of practice. Here, we explore creating a narrative interface and will also generate small sketches of experiences that are yet merely ideas not yet refined or tested. These could be considered the paths not chosen for this expedition.

The main exploration we will undertake is a mock-up of an interface dealing with the topic of crime in Brazil. It will test a framework of ideas that an interactive designer can use to more deeply tap into the points of engagement offered by storytelling, and it will also attempt to synthesize much of the background research conducted. various experience sketches will act as points for future exploration and talking points on the designer’s role in crafting for an audience. In the process, we will do a contextual review of some other practitioners who are creating the conditions for rich experiences in the digital medium.

Sol Stein (1995) writes about the intentions he most commonly encounters as an editor. In the editing process, he uncovers something that is useful for us to consider regarding the experience of an audience and our role in designing for them.

The four most common [intentions] I’ve heard are ‘I am expressing myself’, ‘I have something to say’, ‘I want to be loved by readers’ and ‘I need money.’ Those are all the occasional outcome of the correct intention, which is to provide the reader with an experience that is superior to the experiences that the reader encounters in everyday life. If the reader is also rewarded with insights, it is not always the result of the writer’s wisdom but of the writer’s ability to create the conditions that enable pleasure to edify. (p. 3)

As an interaction designer, it is the practice of our stagecraft that enables the boundaries between the audience and the story performance to blur. We can help bring focus onto the story in a way that heightens an audience’s sense of intensity and awareness, allowing them to walk across the threshold and feel part of the story. The basic definition of narrative is the experience of relationships over time. Zettl (2011) describes these ways of experiencing as: “duration, change, causality, reoccurring phenomena, cycles, rhythms, and motion” (p. 232). We often think of webpages as distinct waypoints or destinations, but the digital medium is truly more of a procession. If the movement of that procession is compelling enough, then an audience will feel drawn to be a part of it. We need more consideration for the experience of relationships over time when designing for the digital medium so that we can optimize conditions for an audience to find their own reward. Brad Johnson (2012), founder of the design studio Second Story, has a poetic view of this.

90 / 91

The evolution of media means the story no longer flows in one direction, from the one to the many. Through a framework of possibility that visitors use to weave their own story, the narrative is only visible in hindsight—when their path is revealed—the path that was their history, their story—that is the second story. (para. 2)

Terra Incognita is a design firm that works extensively with museums and cultural institutions. In 2004, their principal, Bart Marable, presented what they called an “Integrated Experience Model” at the Museums and the Web conference. This model grew out of their experience answering the question of what a museum’s online exhibitions should be like to address the different needs and interests of a broad audience. They identified three distinct layers of information that could also be points of entry into an online exhibit: experience, exhibit, and research. We could visualize these in a pyramid, with experience at the top and research as the foundation. The top experience level is entertaining, immersive, linear in structure, low in content, high in emotional engagement, and limited in room for growth ([diagram], para. 5). The second exhibit level has these qualities blend with those of the level underneath. The foundation is the research level that has a reference application, is non-linear, has deep content, is visitor-structured, and has continued growth ([diagram], para. 5). Each of these three levels has multiple

“connecting storylines” (para. 17) that allow someone to seamlessly move between the different types of content. The Integrated Experience Model moves from a narrative-like experience, though an exhibit that allows greater choice, to a deep archive of rich and specific details.

Marable’s model can be slightly remapped to provide a new perspective on developing narrative interfaces: the top of the pyramid as story, the middle layer as interaction, and the bottom as database. We can now begin to view the pyramid as a type of sightline. Standing at the top of the pyramid, we can look down through the perspective of the story. The story becomes the unifying concept from which choices about the interaction are made. Looking through the story and the interaction, we now have a way to order the database, the minutiae of content.

This pyramid analogy orders the content from the top, from where the point of view converges into a single perspective. We can enrich the story and interaction levels by shifting this perspective point. Richard Saul Wurman, a designer and early pioneer of the Internet who coined the phrase information architecture, argues that there are only five organizing principles and uses the acronym latch to order them: Location, Alphabet, Time, Category, and Hierarchy. These ordering principles can be applied as different perspective points through which we can view the same data. Shifting perspectives

is a way to see one collection of information through new stories and new interfaces. It will enrich and enlarge both the interaction and story layers. latch can be also be applied as unique visual navigation systems for the same site/data.

You can see an example of this use of multiple perspectives in the website Cowbird (cowbird.com), a social website where people share stories through images, text, and sound. Permutations of these ordering principles are used to create multiple ways to view and discover stories from the database. Each ordering principle also creates a unique type of interface interaction. Jonathon Harris, Cowbird’s creator, and uses

“computer science, statistics, storytelling, and visual art as tools” (Harris, 2012, para. 1) in many of his projects. Throughout the websites you can see examples that fit with the pyramid idea and ordering principles discussed here. We see this in storytelling projects ranging from a website that maps the Internet for the use of emotional

“I feel…” words to a “portrait of happiness” formed through interviews with people he met while travelling around Bhutan. Harris believes in technology but is on a mission to humanize its use, and his work is delightful.

[Above] Cowbird.com is a communal storytelling project created by the artist Johnathon Harris. You can reorganize the database by stories, people, places, topics, timeline, sagas, or serendipity.

92 / 93

So far, we have looked at how the intentions of a creator can be different than the “second story” that visitors create on their own. We have also considered the importance of the digital medium as a temporal experience, and investigated an integrated experience model to see how viewing interaction and data through the perspective of a story can be a unifying principle. We have seen examples of how the ordering principles of latch can shift perspectives and thereby reveal new stories and new ways of interacting with data. Let us return to the idea of experience sketches.

A sketch is a plan, a way of thinking, a process. It is a translation of ideas into a medium, but not necessarily their final medium. As such, a sketch is not completely formed and somewhat messy and, because we generally can see this, a sketch has less chance of being harshly judged because we know it is a work in progress. It is a work that is open to evolving through feedback and comments. Let us sketch some ideas about sound and interaction.

[Images on this page] Photojournalist and filmmaker, Dianne Whelan, traveled with the Canadian Army in March 2007 to document the most extensive patrol of the Canadian arctic in 50 years. Whelan worked with Jeremy Mendes and Whallop Creative to make a narrative interface to tell this story for NFB/Interactive. The interface is based around a sound-track narration and is enhanced by videos, maps, animations, large screen images and a variety of navigation methods that allow the interactor to reorder the story and dynamically move through the time-line.

http://thisland.nfb.ca/#/thisland

SOUND SKETCHES

Film editing techniques show that sound can bridge transitions of shots and scenes. A drummer in a marching band is being interviewed indoors, and before cutting to an outdoor scene of him strutting and performing on the street, you hear music and clacking boots. Sound can even be a conceptual bridge, like in a Hitchcock movie where a woman opens her mouth to scream and the camera cuts to a low view of a car

tires squealing out of a parking lot. So, if you are progressing through a story using sound and interaction, why not allow the buttons to telegraph what is coming with a sound transition?

Janet Cardiff and George Bures Miller are installation artists who craft with soundscapes. In the work Paradise Institute (2001), the audience climbs into a small shed with a hyper-perspective model of a movie theatre

inside. On a tiny screen and through headphones, they observe a short film, but there is constant play with real space and cinematic space. As you watch the story, you hear short interruptions from audience members next to you, but they are not real people actually sitting next to you, only the voices of a pre-recorded audience. They make comments about the film or ask if you want popcorn, but in reality that is only the voice of the artist in the headphones. There is even a mysterious second story developing that is a different plot line than the

Dark Pool, Janet Cardiff & George Bures-Miller, 1990-1997, Sculpture and Installations. [Photo] Larry Qualls

audience conversation and movie narrative. The listener is informed by a voice from behind, “It’s all arranged. He’ll meet us between shows.” This idea of commentary and playing with real and screen space offers rich territory to explore in the digital medium.

Perhaps one reason that audio is not used more commonly than subtle button clicks is that bad sound is much harder for an audience to tolerate than bad visuals. An interface could be an exploration where the interactor must perform task dictated by voice, though it would be better as an exploratory exercise than being like the obstructive annoyance of automated telephone operators. Regardless, it is clear that sound and music have the capacity to touch human emotion directly.

In the digital medium, besides video clips that are very much a cinematic familiarity, an audio track can be used as a narrative backbone onto which a variety of visual stills and interactions can be overlaid. Many of the well-crafted sites on NFB/Interactive <http://www.nfb.ca/interactive> employ this technique. We all have a rich history of just listening to stories, from our parents reading to us before we could read, to a grandfather relating lived experiences from another age. Just like when reading a novel, our mind has great capacity to draw its own pictures and imaginary realities from audio.

Like film, a sound clip has a set duration. So, an interactive interface that assists the ability to navigate and control the user’s progress would take advantage of one of the great affordances of digital medium: being able to set one’s own pace.

94 / 95

INTERACTION SKETCHES

Novelty, charm, ingenuity, surprise, mystery, discoverability, a big idea, small goals, rewards—all of these qualities can go into the making of a successful interactive experience. The idea of an Easter egg hunt has been an inside joke with programmers for years. Indeed, it has become a meme for hidden surprises that can be discovered almost by accident or are known and shared by an experienced community. For example, graphic designers professionally rely on their layout software; it is expensive, serious, meant to do a job, but it can suddenly crack a smile and reveal the human side of the programmers that built it.

Quark Express, once the dominant print layout software, had a little alien that would march out and

“zap” an item you deleted when you used the “power” delete key combination. Rather than just hitting the delete key itself, you would hold down the shift, option, and command key to initiate the sounds of the little alien walking out onto the screen and the oscillating sound of its laser atomizing your digital trash. Adobe Illustrator often puts Easter eggs into their about pages. By holding down the option key when you follow the menu path “Illustrator/About Illustrator…” you will find an alternate or “optional” screen.

This notion of hidden Easter eggs could be used on a much grander scale, as a model for a type of interaction. These interactions can arise from the concept of the story and even embody the story itself. In a zoological interface, one could discover the ability to look inside the animals with an x-ray panel or, by touching the screen of our iPad on a movie about a certain Renaissance painter plays, we could realize the ability to look through an antique lens and examine Mona Lisa’s smile more closely. We could even teach a website visitor that hitting certain keystroke combinations

when they see a symbol will reveal hidden layers of information, perhaps edifying or merely anecdotal and amusing.

This type of interaction can extend into the ways of interacting with an interface. Instead of following the usual mode of moving about the Internet (e.g., by clicking an underlined word or phase), what if we communicated interactivity using different references? From our real world, we know that handles pull out drawers, knobs open doors, and switches turn lights on and off. With relative ease, modern interface designers can use programming script libraries to enact these kinds of interactions. Retrieving information then becomes a layered experience of discovery that can be unified under the perspective of a story. The interaction and the story become ways in which to uncover, order, and reorder the wealth of the database resting underneath.

[Above] A detail of the Quark Express power-delete alien Easter egg

[Top row] With a downward thumb swipe, a fabric screen or drawer comes down to reveal a hidden layer of information on the iPhone. There are many such interactions that are part of the iOS touch-screen interface by Apple Inc.

[Right] A sketch for a digital interface that uses the tactile reference of a dial.

[Left, column] Second Story, along with partners Donna Lawrence Productions and Gallagher & Associates, created an immersive exhibition for Coca-Cola around the idea of a vault containing their secret to their formula. Second Story (2012) describes how they used the idea of discovery and finding Easter eggs as part of the exhibit experience. In other words, it is a heuristic process of discovery, problem-solving, and learning, all is based on direct experience.

[Far left, column] Here are two examples of Easter eggs as interactive surprises created by programmers.

[Far left, top two] The browser screen performs the action itself when you do a Google search for

“do a barrel roll.”

[Far left, bottom two] The “About Illustrator...” screen and its “optional” version

96 / 97

MOCK-UPS

Through these movements, transitions, and reactions, the digital medium increasingly reveals itself as a medium dealing with time. What if an interface subtly addressed this very quality of time passing? We could use the ancient technology (in Internet terms) of an ani-mated gif as a repeating background pattern of lightly coloured diagonal lines. Gif animations can have any time duration. What if, over the process of, say, ten min-utes, the colour very subtly changes, and a page slowly ages from rose to orange to green, as if from the passing of the seasons. It could happen so slowly as to be hardly perceptible. The effect of navigating through and spend-ing time reading the pages of this site would perhaps be one of a shimmering presence of light and colour, and it would feel subtly more alive, more inhabited by a spirit of light and nature, than a static page.

To mock-up this idea, we can look at a hypothetical interface for the Kröller-Müller Museum in the Netherlands. It is a unique experiential museum in itself. Nestled in the countryside among farmland, the museum is a destination location that you can drive to, but not right up to. Instead, you must park and then proceed by bicycle through the verdant estate grounds leading to the museum proper. The building has many windows exposing the natural landscape from inside, so the outside is ever-present while you view an important collection of impressionist and post-impressionist masterpieces.

The Kröller-Müller Museum also has a modern sculp-ture garden and many of the installations play with the natural forms of the landscape. Indeed, the logo reflects this unique pairing of art and nature by incorporating subtle colour references to nature and the seasons. This interface mock-up is more whimsical than practical—it uses shapes of river-washed stones and diagonal lines to play with graphic space—but it should suffice to show the concept.

[Above] This display of Chinese terracotta horses demonstrates how the Kröller-Müller Museum merges interior and exterior spaces. [Photo] Ian Grivois, 2006

NARRATIvE INTERFACE MOCK-UP:

Kröller-Müller MuseuM

How do you subtly demonstrate the feeling of being at a museum (which uniquely blends the living environment with art) and how do you demonstrate on a website how the passing of seasons joins with the aesthetic experience?

98 / 99

This mock-up for the Kröller-Müller Museum website demonstrates a subtle movement of colour in the background (top of the interface) over an extended period of time. [Photo] Ian Grivois, 2006

100 / 101

The second exploration with an interface mock-up is on the topic of crime in Brazil. This is meant to be a journalistic piece that incorporates original photographs and compelling stories based on characterizations. The mock-up is layered with interaction, music and spoken word, which has to be imagined here in book form. The compelling story and visuals intend to evoke emotion and interest in the reader. As with the diagram at the beginning of this chapter, each episode contributes to a larger story. As well, fulfilling the role of a mock-up, these interface images should hint at what a fully developed interactive piece would be like.

Examples of the National Geographic magazine’s use of type and image and some of the interactive websites of PBS and CBC News were inspirational to the researcher, and perhaps this resulted in finished pieces that have not sufficiently evolved to look substantially different from their inspirational source.

Time and space to inhabit the material is needed so that the interactions and design develop from within the story context. This “inhabiting” could also help the screen design and dramatic pace avoid becoming frantic with stimulation. There is beauty in slow pacing and simple statements next to the fast and complex.

[Above] Workers build a pedway overpass in Brasília. [Photo] Ian Grivois, 2011

Growth continues at a frenetic pace in Brazil. The city of Brasília was built in the 1950s to house 300,000 people. Brasília and the surrounding cities now hold three million.

NARRATIvE INTERFACE MOCK-UP:

law & Order in BrazilHow do you demonstrate narrative hooks that engage the reader with stories, interaction, and visuals about the struggle for law and order in Brazil? Each story is an episode that contributes to a larger picture of these struggles.

102 / 103

HOMEPAGE The homepage sets the scene and begins to establish familiarity with how the interactions will work. There is an initial moment where the familiar symbols of interaction are relied upon to communicate function, but the interactor must also explore and discover how things will work. Ideally, interest is piqued and the visitor wants to learn more about this topic.

1 Selected state of the main topics navigation.2 Hover state of the main navigation.3 Rotating time progress icon appears while loading media.4 The next (or previous) story title appears when arrow is

hovered over. As well, there could be a sound transition hinting at the next story. The large buttons are to facilitate interaction on a touch screen tablet device.

5 volume and navigation controls for video and sound elements.

6 The hover state for the bottom navigation tab changes colour and displays the title of the piece.

7 Bottom navigation is pulled out from the edge and allows a visual preview of the stories and the ability to navigate in other than a linear next/previous progression.

INTERACTIONS

21

3

4

5

67

104 / 105

STORY PAGES The story pages will have their own momentum. The viewer can listen to spoken word anecdotes or read more deeply into the topic. This screen displays a series of dots that act as a navigation progress indicator for the background slides that will automatically scroll after a period of time. The story lead remains the same until “more” is clicked upon, but the images and captions accompanying them will change.

TRANSITION This shows an in-between state, as the slideshow moves to the next image.

106 / 107

NExT IMAGE This screen demonstrates the next image expanding upon the story of the CSI crime lab in Brasilia. Opposed to a small slideshow window of images, the full-screen high resolution images provide for a greater sense of immersion into their space. The subtle interface elements in white were designed to intrude as little as possible on the reality of the story text and images.

ENRICHING THE STORY Episodes in the greater story of crime, rehabilitation, and prevention are inserted to show the flexibility of the interface and story approach.

108 / 109

PREvENTION & ENFORCEMENT

This story works in tandem with the previous one. Together, they show two different aspects of the effort to prevent crime by outreach to youth at risk and early police intervention in schools.

GETTING PERSONAL The final two stories show examples of how crime touches a community, using characterization to enrich the impression of genuine people. The story is told through the narrator’s own experiences.

110 / 111

CLICKING “MORE” Here is an example of what could happen when the visitor clicks the “more” button, and the full story is revealed. The video can go full-screen or be viewed as part of the interface.

CONTRAST This last story further explores the contrasts between wealth and poverty that can suddenly rise to the surface of Brazilian society. The story is visually reinforced through the contrasts between warm and cool in the main image. In the foreground there is a scene of warmth and community; in the background there is the coolness of the man-made lake water and evening light. It’s a peaceful scene that belies the threat of the man-made problems lurking in the story.

112 / 113

TYING IDEAS TOGETHER Storytelling is explored in this interface about crime in Brazil through dramatic visuals progressing through time, creative writing, and interface interaction with the narrative. Each episode of this interface mock-up contributes to telling the story of what is possible with a narrative interface.

CONCLUSION

These design explorations inhabit and expand on ideas found in this research. We have looked into the nature of experience and into designing to create conditions for audiences to find their own story. Experience sketches probe what is possible, and mock-ups test possibilities and ways of looking at creating narrative interfaces. We plan a route of exploration and, at the end of the expedition, look back and think of what new paths we could explore, next time...

114 / 115

dénOument

cOnclusIOn

this thesis began by looking at how to enrich the experience in the digital medium through narrative

techniques. Research across disciplines sought to uncover unique storytelling forms. Explorations were made into finding poetic resonance in the materials and interactions of the digital medium. The possible use of narrative as a unifying principle of an interface was outlined and tested. In the end, after exploring, musing and probing, the research began to form an ethos for design practice in the digital medium.

The methodology of action research and investiga-tions through design proved useful. A deeper under-standing and trust in the value of art and design practice was achieved through the activities of planning, making, observing, and reflecting. Before embarking on this the-sis, the researcher considered design to be an applied art, different than the practice of fine art. This distinc-tion blurred with a new understanding that work in the digital medium is a culture-building activity that actually invents the medium itself (Murray, 2011), that design can inspire poetic resonance in an audience, and that we can craft situations for their delight and edification.

Research and practice are the woof and weft of the net we must weave to catch our own ideas. Creativity is about going places that will baffle you, where you can divest yourself of your preconceptions, and this might not always be comfortable. Michael Rock (2009) writes about the designer needing to find his or her authorial voice, or creating from a place of personal interest and ability. While his ideas do blend the qualities of art and design, how to achieve this in professional practice is still unclear. Action research arises from sociology and is intended to be practiced within a community, where the process of research is both influenced by, and an influencer of, the

community of practice. Perhaps that is a way forward.We have looked at the advocacy role that aesthetic

interaction and craft can play through examining the Shaker maxim, “don’t make something unless it is both necessary and useful; but if it is both necessary and useful, don’t hesitate to make it beautiful.”

Critical design and literary figures of speech were investigated as ways to find a poetic approach to design, reinforcing the words of Paul Rand (1998), that “to design is to transform prose into poetry” (p. 9). These ideas are later applied in poetic investigations of light as a storytelling medium. As well, looking at the practices of the Situationists, the paradox between virtual and physical reality is bridged to make visual puns and reclaim our imaginative landscape.

Film teaches that information and relationships are revealed through a process of disclosure, and that the quality of that disclosure can attract and focus the attention of an audience. Film also shows the power of visual language to communicate and affirms the technique of “show, don’t tell,” while progressive disclosure reveals the power of mystery and thwarted desire to motivate and enthrall audiences.

Like film, good quality writing is another form of art and craft that can make words and ideas alluring to consumers. Possibilities for drafting storylines and characterizing a narrative interface were outlined in this research, showing how creative writing uses the power of the word to pull an audience along a plot line.

Psychology underpins our understanding of moti-vations and games as well as our pleasure of being in the flow of the moment and immersed in a subject of interest. This indicates that design has a foundation in science, especially in areas that deal with people and perception.

118 / 119

All of these topics of research led the researcher to poetic investigations of storytelling light and storytelling boxes. Projects were created that made the familiar strange, as a tool to reexamine our relationship with technology and the ideas underneath its use.

A design exploration into creating deeper imaginative spaces through story, light, and mobiles revealed unexpected results. Connections were made to a number of similar projects that developed as fine art, the design of collaborative spaces, and the design of ambient experiences. The project led to the discovery of the Phillips’ approach of using a matrix of innovative ideas to allow ideas, which would have never evolved past an exploration, to influence their environment and thereby develop new business practices in which they can thrive.

Further design explorations looked at how to relate still images through story, depth, and a zooming interface. These explorations applied one of the fundamental ideas of this thesis—that design for the digital medium should be considered as a relational progression through time or, in other words, be a story experience.

The final exploration drew from the research to date and applied it to an interface around crime in Brazil. A number of experience sketches were presented that could be expanded into interfaces in the future. A process for analyzing and developing an exhibition to address the needs and interests of a variety of people was discovered in the work of Bart Marable and adapted as a way to order and reorder data to create rich story spaces and interactions.

Indeed, this book itself is a type of narrative interface. The ideas of progressive disclosure and design for an experience happening over time have been applied to the very layouts you are reading. Through this dramatic telling, forms that speak directly and emphatically are revealed, and, in later chapters, it is demonstrated how these lessons can also be applied in the digital medium.

All of these ideas can be seen as a process of envisioning. To envision is to create momentum— a movement that will hopefully lead to more beauty, delight, surprise, mystery, poetry, ease of use, and further acknowledgement of the value of design practice in the digital medium.

120 / 121

references

(2007). K. Barber (Ed.), Student’s Oxford Canadian dictionary (2nd ed.). Don Mills, ON: Oxford University Press.

Abrams, J. J. (2008). J. J. Abrams’ mystery box. In TED: Ideas worth spreading. Retrieved from http://www.ted.com/talks/j_j_abrams_mystery_box.html

Bachelard, G. (1994). The poetics of space. Boston, MA: Beacon Press.

Baldick, C. (2008). The concise oxford dictionary of literary terms. (Third ed.). Don Mills, ON: Oxford University Press. Retrieved from http://www.answers.com/topic/intentional-fallacy

Bentz, v. M., & Shapiro, J. J. (1998). Mindful inquiry in social research. Thousand Oaks, CA: Sage Publications, Inc.

Beinnale Internationale Design (2010). Exhibition/between reality and the impossible. Dunne, A. & Raby, F. (Curators). Presented at the Biennale Internationale Design in Saint étienne, France, from November 20th to December 5th. Retrieved from http://goo.gl/E6pKp

Bramston, D. (2010). Basics product design: visual conversations. Lausanne, Switzerland: AvA Publishing.

Brainy Quote. (2012, March 21). Half the lies they tell about...at Brainy Quote. Retrieved from http://www.brainyquote.com/quotes/quotes/y/yogiberra100356.html

Buxton, B. (2007). Sketching user experiences: Getting the design right and the right design. San Francisco, CA: Morgan Kaufmann.

Cardiff, J. & Miller, G. B. (2001). Paradise institute [video]. Retrieved from http://tinyurl.com/7dcc7tq

Chastain-Rapp, C. (2010). Thinking like a storyteller [Web]. Retrieved from http://vimeo.com/9686849

Coe, J. (2011, April 1). Good book, great film. Retrieved from http://goo.gl/ZHsuZ

Columbia Journalism School. (2009). duPont winners’ circle discussion with Ira Glass, Alex Blumberg and Adam Davidson [Web]. Retrieved from http://youtu.be/I-b5IwMwz4s

Cross, N. (2001). Designerly ways of knowing: Design discipline versus design science. Design Issues, vol. 17, No. 3 (Summer), 26-34. Cambridge, MA: MIT Press. Retrieved from http://www.jstor.org/stable/1511801

Csikszentmihályi, C. (2006). Control. In C. A. Jones (Ed.), Sensorium: Embodied experience, technology, and contemporary art (p. 125). Cambridge, MA: The MIT Press.

Csikszentmihalyi, M. (1991). Design and order in everyday life. Design Issues, vol. 8, No. 1(Autumn), 26-34. Retrieved from http://www.jstor.org/stable/1511451

Csikszentmihalyi, M. (2008). Flow: The psychology of optimal experience. New York, NY: Harper Perennial Modern Classics.

Csikszentmihalyi, M. & Robinson, R. E. (1990). The art of seeing: An interpretation of the aesthetic encounter. Los Angeles, CA: Getty Publications.

Csikszentmihalyi, M. (October 2008). Mihaly Csikszentmihalyi on flow. In TED: ideas worth spreading. Retrieved from http://www.ted.com/talks/mihaly_csikszentmihalyi_on_flow.html

Curtis, H, (2005). Hillman Curtis on creating short films for the Web. Berkley, CA: New Riders Press.

Debord, G. E. (1955). Introduction to a critique of urban geography. Retrieved from http://tinyurl.com/7qrq6ex

Debord, G. E., & Wolman, G. J. (1956). A user’s guide to détournement. Retrieved from http://www.bopsecrets.org/SI/detourn.htm

Drago, C., Leung, L. & Ward, M. (2008). What‘s the story? harnessing the power of storytelling in film for experience design. L. Leung (Ed.), Digital experience design: Ideas, industries, Interaction. Bristol: Intellect Books.

Duarte, N. (2010). Resonate: Present visual stories that transform audiences. Hoboken, New Jersey: John Wiley and Sons, Inc.

Dunne, A. (2005). Hertzian tales. Cambridge, MA: MIT Press.Fischer, E. (2011). The necessity of art. Brooklyn, NY: verso.

122 / 123

Manzini, E. (1986). The material of invention. Milano: Arcadia srl.

Marable, B. (2004, March). Experience, learning, and research: Coordinating the multiple roles of on-line exhibitions. Paper presented at Museums and the Web 2004, Washington, DC. Retrieved from http://tinyurl.com/cmsrutm

Meadows, M. K. (2003). Pause & effect: The art of interactive narrative. Indianapolis, Indiana: New Riders.

McCloud, S. (1994). Understanding comics: The invisible art. New York, NY: William Morrow Paperbacks.

Moore, A. (2010). Alan Moore’s writing for comics. Rantoul, IL: Avatar Press.

Munari, B. (1966). Design as art. London: Penguin Group.Murray, J. (2011, December 4). Four affordances. Retrieved

from http://inventingthemedium.com/four-affordances/Murray, J. H. (1997). Hamlet on the holodeck: The future of

narrative in cyberspace. New York, NY: The Free Press.Murray, J. H. (2011, December 4). Three principles. Retrieved

from http://inventingthemedium.com/three-principles/Quote DB. (2012, March 21) Art is a lie that tells the truth. By

Pablo Picasso. Retrieved fron http://www.quotedb.com/quotes/2306

Rand, P. (1970). Thoughts on design. van Nostrand Reinhold.Rand, P. (1998). A designer’s words. Heller, S., Balance, G. &

Garland, N. (Ed.). New York, NY: School of visual Arts. Retrieved from http://imprint.printmag.com/daily-heller/paul-rands-words/

Raney, C. & Jacoby, R. (2010). Decisions by design: Stop deciding, start designing. Rotman Magazine, (Winter), 34-39. Retrieved from http://goo.gl/c2FxO

Rock, M. (2009). Fuck content. Retrieved from http://2x4.org/ideas/2/Fuck+Content/

Rose, F. (2011). The art of immersion: How the digital generation is remaking hollywood, madison avenue, and the way we tell stories. New York, NY: W. W. Norton & Company.

Glass, I. (Artist). (2009). Ira Glass on storytelling. [Web video]. Minneapolis, MN: Public Radio International. Retrieved from http://goo.gl/uD3TU

Glass, I. (2009, Febuary 17). duPont Winners’ Circle Discussion with Ira Glass, Alex Blumberg and Adam Davidson [Web video]. Retrieved from http://youtu.be/I-b5IwMwz4s

Gray, C, & Malins, J. (2004). Visualizing research: A guide to the research process in art and design. Burlington, vT: Ashgate.

Grivois, I. (2011, August). David Clark, intermedia storyteller. Retrieved from http://goo.gl/W64wB

Grynsztejn, M., Birnbaum, D., & Speaks, M. (2007). Olafur Eliasson. London: Phaidon Press Limited.

Hara, K. (2009). White. Zürich: Lars Müller Publishers.Harris, J. (2012, March 28). The work of Jonathon Harris.

Retrieved from http://www.number27.orgHills, D. (2011, Winter). The Stanford Encyclopedia of Philosophy.

E. Zalta (ed.). Retrieved from http://plato.stanford.edu/archives/win2011/entries/metaphor/

Kyffin, S., & Gardien, P. (2009). Navigating the innovation matrix: An approach to design-led innovation. International Journal of Design, 3 (1), 57-69. Retrieved from http://www.ijdesign.org/ojs/index.php/IJDesign/article/view/305/242

Lakoff, G. & Johnson, M. (2003). Metaphors We Live By. Chicago, IL: University of Chicago Press.

Lancaster, K. (2011). DSLR cinema: Crafting the film look with video. Burlington, MA: Focal Press.

Levi, M. D. (2008, July 19). A shaker approach to web site design. Retrieved from http://tinyurl.com/3t4o8s

Leung, L. (2008), Digital experience design: Ideas, industries, Interaction. Bristol: Intellect Books.

Locher, P., Overbeeke, K., & Wensveen, S. (2010). Aesthetic interaction: A framework. Design Issues, 26(2), 70-79. Retrieved from http://tinyurl.com/d9zlecj

Mascelli, J. v. (1965). The five c’s of cinematography. Beverly Hills, CA: Silman-James Press.

Sadler, S. (1998). The situationist city. Cambridge, MA: MIT Press.

Sauter, J. (2011). Art+com: Digital poetry and analog magic. Berlin, Germany: gestalten.tv: exploring visual culture. Retrieved from http://www.gestalten.tv/motion/artcom

Second Story (2012, April 11). Vault of the Secret Formula [Web video]. Retrieved from https://vimeo.com/34592058

Shawning, M. v. (2004). Examining film’s two-goal plotting. Retrieved from http://tinyurl.com/bupfvz8

Shedroff, N. (2001). Experience design 1. Indianapolis, Indiana: New Riders.

Sinek, S. (2009, September). How great leaders inspire action. In TED: ideas worth spreading. Retrieved from http://goo.gl/HkkAZ

Stein, S. (1995). Stein on writing: A master editor of some of the most successful writers of our century shares his craft techniques and strategies. New York, NY: St. Martin’s Press.

Stewart, M. (2002). Launching the imagination: A comprehensive guide to basic design. New York, NY: McGraw Hill Higher Education.

Swann, C. (2002). Action research and the practice of design. Design Issues, volume 18, Number 2 (Winter 2002), 49-61.

Tan, E. S. (2009). Emotion and the structure of narrative film: Film as an emotion machine. New York, NY: Routledge.

Traub, C. H. & Lipkin, J. (2001). Digital Humanism. In S. Heller (Ed.) The education of an e-designer. New York, NY: Allworth Press. Retrieved from http://goo.gl/vwxaW

Turkle, S. (2011, March 25). TEDxUIUC - Sherry Turkle - alone together. Retrieved from http://tinyurl.com/5w7vgsj

U.S. State Department. (2012, March 25). Brazil. Retrieved from http://travel.state.gov/travel/cis_pa_tw/cis/cis_1072.html

Yanagi, S. (1989). The unknown craftsman: A Japanese insight into beauty. New York, NY: Kodansha America Inc.

vincente, D. (11 March 2012). Seeking is the granddaddy of emotional systems—sources of insight. Retrieved from http://tinyurl.com/cvr7jfr

Wurman, R. S. (2000, Dec 28). The business of understanding: Latch: The five ultimate hatracks. Retrieved from http://ow.ly/9RjN3

Zettl, H. (2011). Sight sound motion: Applied media aesthetics, sixth edition. Boston, MA: Wadsworth.

124 / 125

cover/inside (Public domain) Amedeo Modigliani (1918). Portrait of Jeanne Hébuterne [Detail], oil on canvas, 18.1 x 11.4”. Retrieved from http://tinyurl.com/7j4m8r3

2 © Ian Grivois (2012). Hands and constellations. Ink on paper.

4 © Ian Grivois (2012). Research process visualized. Ink on paper.

6 (Public domain) Leonardo da vinci (n.d.). Crank Device for Manipulating Wings, pen and ink. ARTstor 41822001520061.

8 (Public domain) NASA, ESA, T. Meageath, & M. Robberto. The Orion Nebula.

10 © Ian Grivois (2012). Visualization of design process and action research process. Ink on paper.

11 (CC BY 2.0) thinboyfatter (2010). Satin Bowerbird nest - Across the road from O’Reilly’s. [Photographer]. Retrieved from http://www.flickr.com/photos/1234abcd/4717190370/

12 © Ian Grivois (2012). Hands tapping. Ink on paper.

13 (CC BY-SA 2.0) Marcus (2000). Think different – Hitchcock. [Photographer]. Retrieved from http://fr.wikipedia.org/wiki/Fichier:Think_different_-_Hitchcock.jpg

14 [Right & left] (CC BY 2.0) Nathan Williams (2006). Olafur Eliasson - Weather Project. [Photographer]. Retrieved from http://www.flickr.com/photos/simiant/277289520/

Image credIts

Every effort has been made to secure copyright for the images not created by the author. In some cases it was not possible and images are used at a small size for reference with the assumption of fair use as a not-for-profit educational review.

Please contact the author in case of any errors or omissions for correction in future printings.

grivois.ian@gmail.com

15 (CC BY 2.0) Damien du Toit (2004). ‘The Weather Project’ by Olafur Eliasson. [Photographer]. Retrieved from http://www.flickr.com/photos/coda/409320556

18 [Top, left] Edward Deming Andrews (1854). Tree of Life, 18 x 23”, University of Georgia Libraries. ARTstor 4590002.

[Top, right] Side Chair (c. 1880) New York Historical Society. ARTstor 2360007.

[Bottom, left] Trustees’ Desk (19th century). University of Georgia Libraries. ARTstor 2370001.

[Bottom, right] Spool Box and Cover (c. 1825-40) Philadelphia Museum of Art, Pennsylvania, USA. ARTstor 103883042.

20 [Left column] Kimono, (Japanese, early 20th century). Seattle Art Museum. ARTstor 10312601137.

[Center column] Bedding cover (futonji) (Japanese, late 19th - early 20th century). Seattle Art Museum. ARTstor 10312600836

[Right column] Okinawan kimono, Japanese, (19th century). Seattle Art Museum. ARTstor 10312601480.

22 © Ian Grivois (2004). Evening light [Detail], acrylic on canvas, 9 x 12”.

25 (CC BY-SA 2.0) m.prinke. Duft-Tunnel. Retrieved from http://www.flickr.com/photos/mprinke/2852818333/

126 / 127

26 © Ian Grivois (2009). Roman Holiday. Photograph.

28 © Ian Grivois (2012). Taxi Driver. Photograph.

29 © Scott McCloud (1994). Understanding comics: The invisible art. New York, NY: William Morrow Paperbacks. (74) Reproduced with permission.

30 (CC BY 2.0) Jeremy Thompson. Bust of Aristotle. Retrieved from http://www.flickr.com/photos/rollercoasterphilosophy/4482693082/

35 © Martha Arquero (1980). Frog storyteller, sculpture. Courtesy of National Museum of the American Indian. Retrieved from http://tinyurl.com/7fhe23w

36 © Ian Grivois (2006). A Door Knocker in Spain. Photograph.

40 (CC BY 2.0) Elliott Brown (2011). Piccadilly Circus, London - Shaftesbury Memorial Fountain, Eros - at sunset. Retrieved from http://www.flickr.com/photos/ell-r-brown/6438569187/

44-45 © Ian Grivois (2012). Sketches of mobiles.

45 © Bruno Munari. Mobile. Retrieved from http://www.munart.org/index.php?p=10

46-47 © Ian Grivois (2012). Photographs and sketches.

48 [Top] (CC BY-NC-SA 2.0) davepatten. Telematic dreaming – Sarah. Retrieved from http://www.flickr.com/photos/davepatten/294836008/

[Bottom, left] (CC BY-ND 2.0) Nabi Art Center. Telematic Dreaming (1992). Retrieved from http://www.flickr.com/photos/artcenternabi/5101737416/

[Bottom, right] (CC BY-ND 2.0) Nabi Art Center. Telematic Dreaming (1992). Retrieved from http://www.flickr.com/photos/artcenternabi/5101737372/

48-49 © Ian Grivois (2012). Mobile in Office Sketch.

50 [Left] © Philips Design. Blue man, Nebula – design concept creating a virtual/physical experience for bedtime. Reproduced with permission.

[Right] © Philips Design. Nebula – design concept creating a virtual/physical experience for bedtime. Reproduced with permission.

51 [Top] © Philips. Message-high, Ambient Experience in emergency department for children. Reproduced with permission.

[Bottom] © Philips. Ambient Experience in hospital cath lab. Reproduced with permission.p

52-60 © Ian Grivois (2012). Photographs.

61 © Terrence Malik [director]. Still from Tree of Life. Produced by Brace Cove Productions, Cottonwood Pictures, Plan B Entertainment, & River Road Entertainment.

© Wim Wenders [director]. Still from Pina. Co-produced by Neue Road Movies, Eurowide Film Production, Zweites Deutsches Fernsehen (ZDF), ZDFtheaterkanal, & ARTE.

© Ian Grivois (2012). Papercut, Photograph.

62-63 © Ian Grivois (2012). Photograph.

(Public domain) Domenico Remps. Cabinet of Curiosities. Retrieved from http://commons.wikimedia.org/wiki/File:Domenico_Remps_-_Cabinet_of_Curiosities_-_WGA19254.jpg

(Public domain) Engraving from Ferrante Imperato, Dell’Historia Naturale (Naples 1599). Retrieved from http://en.wikipedia.org/wiki/File:RitrattoMuseoFerranteImperato.jpg

64-71 © Ian Grivois (2012). Photographs.

72 [Top, left] Detail of Bathing Woman (Baigneuse de Valpincon), Jean-Auguste-Dominique Ingres, 1806. ARTstor 1039902284.

[Top, right] Detail of Liberty Leading the People (July 28th 1830), Eugène Delacroix, 1830. ARTstor 1039490420.

[Bottom, left] © Guy Debord (1959). Life continues to be free and easy. (As cited in Sadler, 1998, cover.)

[Bottom, right] © Ian Grivois (2010). A model of the city, Brasília. Photograph.

74-79 © Ian Grivois (2012). Photographs.

82 Google Maps (2012, April 6). Halifax. Retrieved from http://tinyurl.com/7lruz8b

84 © Eames Office (2012, April 6). Powers of Ten: Based on the film by Charles and Ray Eames. An Eames Office website [Screen captures]. Retrieved from http://www.powersof10.com Reproduced with permission.

85 (CC by 2.0) Dave Pape (n.d.). Fractals. Retrieved from http://resumbrae.com/ub/dms423_f06/26/

86 [Left] (Public domain) Anon. (2012, April 6). Water-driven armillary plans. Retrieved from http://tinyurl.com/bpaq2vy

[Top] (Public domain) Su Song (c. 1020-1101). Song dynasty star map. Retrieved from http://en.wikipedia.org/wiki/File:Su_Song_Star_Map_2.JPG

[Right] © Ian Grivois (2012). Sketches.

87 (Public domain) (2012, April 6). Zhang Heng [stamp]. Retrieved from http://en.wikipedia.org/wiki/File:Zhang_Heng.jpg

88-89 © Ian Grivois (2012). Zhang Heng interface mock-up & various sketches.

90-92 © Ian Grivois (2012). Sketch.

93 [Left] © Johnathon Harris (2012, April 6). Cowbird [Screen captures]. Retrieved from http://cowbird.com Reproduced with permission.

[Right] © Ian Grivois (2012). Sketch.

94 [Images] © NFB/Interteractive (2012, May 11). NFB/Interactive - This Land. Retrieved from http://thisland.nfb.ca/#/thisland

128 / 129

95 © Janet Cardiff & George Bures-Miller (1990-1997). Dark Pool. Sculpture and Installations. [Photo] Larry Qualls. ARTstor 10310639677.

96 [Top, 4 images] © Ian Grivois (2012). iOS interface details. Photographs.

[Bottom, middle] © Quark Express. Power delete alien. Retrieved from http://tinyurl.com/23wga

[Bottom, right] © Ian Grivois (2012). Sketch.

97 [Right column, 3 images] © Second Story (2012). Triangle Room Peep Show. Retrieved from www.flickr.com/photos/second_story/collections/72157628829704033/

[Left column, top 2 images] © Google (2012). do a barrel roll - Google Search. Retrieved from http://google.ca

[Left column, bottom 2 images] © Adobe (2012, April 6). Illustrator About Screen & Alternate.

98 © Ian Grivois (2006). Chinese terracotta horses at the Kröller-Müller Museum, Netherlands. Photograph.

99-101 © Ian Grivois (2012). Kröller-Müller Museum interface moock-ups.

102 © Ian Grivois (2011). Construction crew, Brasília. Photograph.

99-100 [Background image] © Gordon Hughes (2010). Wall of victims.

103-114 © Ian Grivois (2010). Crime in Brazil mock-up interfaces.

117-120 © Ian Grivois (2010). Divers. Digtial illustration.

back cover [Right] © Ian Grivois (2009). [Photographer]. Shadow play at the The Royal Alberta Museum: Robert Munch exhibition.

[Top] © Ian Grivois (2012). Hands and constellations. Ink on paper.

COLOPHON

The typeface used throughout this thesis is Whitney by Hoefleur & Frere-Jones.

This printing with Blurb utilized their standard landscape format and image wrap cover. The paper is Proline uncoated, 100# (148 GSM).

The rich visuals of this thesis would not have been possible without the wonderful resource of ARTstor, a gift to scholarly research in art and design, and the Creative Commons image search tool.

The author is grateful for the work of Charlotte at Halifax Proofreading and Resume Services, and would recommend her as a thorough professional and a pleasure to work with.

(902) 876-7969 (local Halifax area)

1-888-233-6183 (toll-free Canada & USA)

hprs@rocketmail.com

130 / 131

“To read poetry is essentially to daydream.” (Bachelard, 1994, p. 34)