0116_thesisfinal_jayne
TRANSCRIPT
Playing with the Here/Now/There/Then:
Crafting Authentic Hybrid Experiences for a Networked Culture
1.0 Introduction
We live in a networked world with increasingly blurry lines between our physical and
virtual lives. Our computation-centric society is more orderly, more systematic, and
more governed by algorithms than ever before. Technologists often rush forward, in the
name of progress and productivity, without looking back to reflect upon the effects (or
missed opportunities) in their wake. Focusing on the “how” instead of the “why” these
new pioneers develop faster and better ways of systemizing, quantifying, streamlining,
documenting and tagging our world objectively. As new algorithms get pumped out
faster and faster, ubiquitous networked computing pervades everything from our built
environment to our pockets.
In contrast to this rushed speed of progress, I believe in a more mindful yet exploratory
approach to developing technology, in order to create idiosyncratic relationships
between humans, computers, and the network. Instead of seeing a spacial or
technological constraint as a potential problem to be “cured” away with technology,
I see it as an opportunity to use play as a tool for challenging existing systems. I am
specifically interested in systems which enable uniquely subjective hybrid experiences
which are made possible by leveraging the real world, the virtual world, and the human
imagination. For instance, the physical world provides the benefit of tangibility while
Jayne Vidheecharoen
1
the virtual world provides the collaborative power of our network culture. And the
human imagination provides the cognitive mortar which holds everything together and
allows a user to create meaning from the composited experience. Due to their hybrid
nature, these experiences have their own unique sets of constraints and therefore their
own unique opportunities for disruptive play. Using play, I aim to create a new hybrid
experience while provoking questions about existing interaction norms, both online and
in real life. In the end, I am hoping to create a new space of possibility for others to
build upon and expand.
2.0 Work & Play: Magic Circles
Our future technology will not only enable us to work more productively, but in fact
more playfully as well. Ubiquitous computing has created a fundamental shift in the
way we separate work and play (Hilbrecht). The idea of work was once confined to the
tangible real world, while play was allowed to exist in the intangible imaginary world.
Now for many of us, much of our daily labor exists only in a virtual, intangible, and
imaginary realm. As a result, this liberates our daily tasks from the rules of reality while
subjecting them to the rules of play.
My early experiments, Zombie Mail & Play Slideshow, challenge the standard
environments of banal digital productivity software to be more playful, while
questioning established user interface norms. We often spend much of our work day in
these dull virtual spaces, but being completely virtual they are especially viable
Jayne Vidheecharoen
2
candidates for becoming magic circles. Johan Huizinga, the classic expert of play in
culture, defines a magic circle as a special temporary play space within the ordinary
world where rules enable specific performances. In this case, having been traditionally
designed for the rules of productivity and efficiency, these environments enable a
particularly objective and task oriented performance by the user. The modifications in
Zombie-Mail create custom metaphors for the e-mail interface, on both a visual and
interaction level, as a way to reflect a users’ subjective feelings about their experience
with e-mail. It also favors a more discovery based navigation as opposed to a
traditional list based navigation. Unlike a mere skin applied over an existing interface,
this navigational shift fundamentally changed how the user interacts with the
information and possibly their attitude towards the task at hand. Similarly, Play
Slideshow makes the normally mundane task of creating slideshow presentations more
challenging and playful by adding constraints. In this case, elements can only enter the
slide by dropping in from above while the user is given limited controls (as in Tetris). By
tweaking the interaction it begins to draw attention to the repetitive and homogenous
act of creating slideshow presentations.
Not only does play, having its own rules of space and time, make something fully
absorbing for a player it can also be used to expose tensions within a system. Since
challenging a tension or established system is consciously outside of ordinary life (a
formal characteristic of play), outlaws and revolutionaries have an inherent element of
play in their activities (Huizinga). Thus, being playful is a potentially revolutionary and
disruptive way to change established systems.
Jayne Vidheecharoen
3
3.1 Constraints & Systems: Activating Public Spaces
Creatively and playfully surmounting the constraints of a rational system is an
opportunity for intervention and improvement. At the urban scale, people who reclaim
public spaces to suit their own specific desires are defying the design of these urban
systems, while actively challenging the way these spaces have been scripted by
architects and urban planners. Margaret Crawford notes that mutability, contestation,
and change is not what constitutes a failure of public space, but instead defines it. The
emergent activities in Los Angeles’ public spaces are the very things that enable us to
pose questions about urban citizenship. Therefore, we should not mourn the loss of
our public spaces but instead see it as a space filled with possibilities (Crawford).
Although it exists in a completely virtual space, Google Street View is essentially a
massive simulation of real world public spaces, frozen in time and filled with un-tapped
potential. Through many of my experiments I’m specifically looking at ways to bring it
to life and activate the space so it can be more like a real public space, subject to
mutability and idiosyncratic uses. First Person Creator questions the use of Google
Street View as a purely static and historically accurate tool for viewing an archive of our
real world. While Google Earth allows people to contribute 3D models of buildings and
monuments, Google’s strict guidelines only accept models which represent real world
structures textured with photographs. Therefore, it is impossible to contribute
innovative or imaginative structures to the collective archive. In contrast, First Person
Jayne Vidheecharoen
4
Creator imagines a sandbox-game style interface which allows the user to augment
and interact with Street View as they desire. The first person perspective is also a shift
from the section, plan, and birds-eye-view of traditional 3D modeling programs. This
perspective empowers people to feel situated within the space, enabling a sort of
spacial play testing. Other mash-up-mock-ups, such as Dancing in the Street View and
Rolling in The Street View, continue this line of enquiry and begin to expose potential
creative opportunities for adding life, through animation, to this static virtual public
space.
3.2 Constraints & Systems: Improvisational Alternatives
In the physical world, Parkour is a clear example of both a form of play and form of
resistance to the scripted use of urban systems and spaces. Nathaniel Bavinton
explains that for Parkour enthusiasts, known as traceurs, constraints do not hinder fun
but in fact add value and enhance it. Thus creative improvisation comes not from
removing the constraints but by reinterpreting and finding new possibility within the
constraints. This is particularly important to remember when considering the role of
technology, which has historically been focused mainly on removing obstacles to make
our lives easier. I am experimenting with what this approach of improvisational
interactive play might look like when we substitute the traceur’s buildings and railings
for the technologies we are given as consumers.
Jayne Vidheecharoen
5
Users are not merely pedestrians adhering to a set clickable-path but instead digital
traceurs who can be empowered to gracefully transcend technological obstacles
through a wide range of embodied techniques. One of the key features of Portals is the
ability to bypass the hurdle of the screen by being able to (seemingly) interact with the
interface inside the screen. While touch screens and gesture based interactions already
allow for a more embodied interaction than the traditional keyboard or mouse, in both
cases the screen still acts like a glass wall between the user and the interface. Portals,
on the other hand, is able to serve as a potential alternative to what Bret Victor calls
“Pictures Under Glass” technology. “Pictures Under Glass” is a predominant vision for
interfaces (in both the present and future) which forego the richness of tactile feedback
in favor of high resolution visual representations. By looking for a fun and fluid way to
both bypass and take advantage of this constraint, I propose an alternative approach
to digital interaction, which is capable of leveraging both the tactile and visual senses
in order to create a more authentic simulation of interaction. The interaction in Portals
may not be easier, as the system tends to cause the user some spacial confusion, but I
believe this additional obstacle adds to the experience by making it more fun.
4.1 Truthiness: Believe
The idea of truth is another rational system open to exploitation & intervention as a way
to create future possibilities. Questionably objective truths, created by algorithms,
already shape our real life in many ways. In his TED talk, Kevin Slavin argues that the
repetitive nature of algorithms enables them to gain the sensibility of truth (even if
Jayne Vidheecharoen
6
they’re not true) until eventually they solidify and become seemingly “real”. For
instance, in the 2010 “Flash Crash of 2:45” we saw algorithms taking control of the
stock market, dropping the Dow Jones Industrial Average about 1000 points before
recouping the losses minutes later. Despite the fact that no human had ordered it or
had any control over what was happening, the effect of the crash was very real.
Code even controls the small decisions we make in our everyday lives. Netflix’s
recommendation algorithms are responsible for 60% of the movies we end up
watching (Slavin). Unlike the code crunching quantifiable financial data, the Netflix
code is essentially turning our subjective relationship with art & culture into something
seemingly objective and true, which influences the way we spend our time in real life. If
algorithms have the power to make us believe in a truth that affects our lives, as
individuals we too have the freedom to play with the truth in order to affect our own
lives.
4.2 Truthiness: Fictional Facts
A subjective and “loose” definition of truth, when used in cahoots with the viewer, can
become a valuable tool for creating a more “real” experience of the present. This can
also allow for the viewer to create their own interpretations, beyond the creator’s
original intentions. P.T. Barnum was infamous for playing with the expectation of truth
in the context of museums. He often manipulated the press to stir up controversy
about the authenticity of his own Dime Museum exhibits. This drove up ticket sales
Jayne Vidheecharoen
7
because he knew the audience loved trying to figure out if the exhibits were fact or
fiction. By inviting the viewers to question his exhibits, he was respecting the viewer’s
intelligence. Many of the viewers enjoyed collaborating on the “intellectual exercise”
regardless if it could be proven to be true (Dennett). The development of 19th century
technologies also left people thinking that just about anything mechanically or
organically probable was possible, making people very susceptible to hoaxing (Harris).
This seems especially relevant in our modern times, given our dependence on the
internet as a main source of knowledge in addition to the rapid pace of developing
technologies which were once only fantasy, such as ubiquitous networked computing,
and complex predictive algorithms.
The narrative I create in my Kickstarter campaign describes Portals as gateways to
another place in the world, at a different point in time, even though they’re clearly not
actual portals with teleportation capabilities. The description also states that it allows a
user to go “inside” the screen, despite the fact that the user is technically behind the
screen. I specifically craft a description which is both true and potentially fantastical. I
also include videos which are just rough sketches of how the Portals could potentially
work. Some see these videos as examples of completely functional prototypes while
some question their verity. Given the honor ecosystem of Kickstarter, it is especially
interesting that this clearly fictional idea of going into a screen to access portals to a
parallel world is an acceptably honest and authentic description of a technology
project to backers. Like P.T. Barnum, I feel it’s important to respect the backers’
intelligence by inviting them to knowingly play along with me in crafting the projects
Jayne Vidheecharoen
8
story and advancing it further. As a result, various blogs continue the story with factual-
sounding headlines like “Media Design Student Reaches Into Another Dimension” (Art
Center Dotted Line), “Portal Becomes A Reality--Sort of--With This Kickstarter
Project” (PC World) and “Thinking inside the box: The television you can 'reach inside'
to grab things on screen” (Daily Mail). Clearly, playing with people’s imagination is a
potentially powerful point of reference for people to grasp a new idea.
4.3 Truthiness: You’re Doing It Wrong
It is also through challenging the true “official” intended purpose of a system or object
that we can create a new possibility space for things to enter the world. McKinzie Wark
explains how hacking is essentially abstracting, and it is through this act of abstraction
that a new space for possibility, beyond necessity, is created. By liberating objects
from their designed purpose, it becomes possible to explore new uses. Within my
Portals project I consciously explore the “wrong” ways to use technology (on both the
software and hardware level) as a way to provoke questions and speculate about what
another “right” way might soon be. While my “wrong” uses of technology may seem
strange at the moment, in the future it may become second nature. Some blog
commenters don’t see any practical use for Portals while others are excited about the
range of potential applications. Several Kickstarter members have contacted me
excitedly with additional application ideas such as table top gaming, race training,
presentation tools, and educational tools. Despite the fact that I’m using everything
“wrong”, to these people, my system feels like the “right” solution for their goals. It’s
Jayne Vidheecharoen
9
important to give people enough context to understand the main ideas behind the
project, while still leaving the space of possibility open enough to allow “wrong”
personal trajectories of inspiration from the initial idea. Hopefully this ensures that the
project is the start of a conversation as opposed to a definitive solution to a single
problem.
5.1 Hybrid experiences: Alternative Reality
As more of our experiences exist in a hybrid reality we are increasingly open to
enhancing our “real” experiences with an “imaginary” virtual layer. We are already
comfortable calling upon “the cloud” to augment our experience of the real world.
Whether it’s changing our behavior to earn a mayorship on Foursquare or letting Yelp
determine our night out, we are perfectly comfortable with letting this virtual layer
permeate our real lives. We can therefore freely borrow from techniques of traditionally
virtual spaces as a way to enhance our real world. Harnessing the power of mixed
reality enables us to challenge our expectations of what is possible in the real world.
Jane McGonigal argues that we can “fix” reality by employing tactics used in games,
such as inviting the user to take on unnecessary challenges, in a personally meaningful
way. She defines alternate reality games as “games you play to get more out of your
real life, as opposed to games you play to escape it.” These alternate reality games are
especially powerful because they give us explicit permission to play outside of social
norms within our real life.
Jayne Vidheecharoen
10
In order to test McGonigal’s theory I created Media Design Bingo, an alternate reality
game designed to increase interaction between all the students in the studio. Other
design goals included designing a game that promoted specific values, such as
gratitude and collaboration, to hopefully encourage a wider range of people to play.
Using the studio floor plan as the bingo board, people were asked to keep track of the
people who had helped them throughout the week, in attempt to complete patterns for
badges. I was surprised to find that people did indeed feel the game increased the
level of interaction within the studio for the week, and people felt more comfortable
approaching some peers they hadn’t in the past. Clearly, this alternate reality game had
created a sort of magic circle where the rules of the game enabled new types of
interactions to occur and actually showed how play could be used to address and
change real life tensions.
5.2 Hybrid Experiences: Movie Magic
While alternate reality games can be powerful, the gamification of real life is only one
type of hybrid experience. If we think beyond gaming and include other forms of “fake”
worlds, such as film, we find another rich set of tools for creating engaging immersive
experiences. The imagined virtual space also has a rich tradition of leveraging
technologically mediated illusions to create the suspension of disbelief, or the
suspension of reality-testing all together. The world of animation, compositing, and
special effects have advanced so far in the past decade that often times we can no
longer separate between what was shot in-camera and what was added in post-
Jayne Vidheecharoen
11
production. As a result, the distinction between the real and the fake dissolves as we
allow ourself to become fully immersed in the experience of the film.
Illusions are powerful because they trigger a feeling of authenticity which can be
manipulated and reshaped (Stark). With Portals I am consciously embracing “movie
magic” techniques, such as chroma keying and layering footage, and applying them to
the context of a real world interface. As a result, the user experiences a simulation of
interacting with the interface directly, while taking advantage of the screen’s ability to
augment and animate. Stop motion animation in particular, is an especially powerful
tool because it has a realness in terms of materiality and motion, but it can still be
stylized beyond what is possible in reality. As a medium, stop motion is inherently
about illusions caused by playing with time. Due to it’s very nature, it also highlights the
uncanny tension between the unexpectedly static or animated when in the context of
the real world. By combining illusion and interaction I strive to create a more magical
experience for the user, which takes advantage of both the real and virtual world, to
create a new type of hybrid space.
5.3 Hybrid Experiences: The Real World
Of course, the hybrid experience goes both ways, in addition to drawing from the
virtual world we should enhance our virtual spaces by drawing from the real world.
With Portals, the ability to physically manipulate objects in the interface takes
advantage of our hand’s sensorial feedback capabilities. Portals’ reality based visual
Jayne Vidheecharoen
12
interface also helps grounds the hybrid experience in the real world as opposed to only
having a place in a virtual one. While virtual spaces like Second Life allow users to
interact with each other as CG avatars in the same virtual space, they also only exist in
a completely fantastical environment. Despite early hype around the potential use of
Second Life as a legitimate alternative to in-person meetings, the idea failed to gain
mainstream acceptance. Relying completely on CG, everything about the world
completely disregards reality and becomes stuck as an escapist amusement, as
opposed to one which could potentially enable the user to benefit from the activity in
their real life.
On the other hand, standard video chat allows for a completely reality-based
conversational experience, but it’s limited in that each person is confined to their own
little window of reality. It also completely ignores the potential for a more fantastical
experience. While visually “real,” the experience of video conferencing distinctly lacks
the ability to really co-exist and visually overlap in a shared third space and play
together. Alternative solutions such as sharing a desktop screen or using a
collaborative white boarding application still leaves the human element stuck on each
side of the side of a screen. The only hint of a human’s presence in these virtual
workspaces is reduced to a wiggly mouse cursor, and maybe a chat box on the side.
Tangibility is just one of many benefits of the physical world, another is the ability to
just co-exist in the same physical space with people while interacting with them.
Portals strives to exist somewhere between virtual worlds, shared desktops, video
Jayne Vidheecharoen
13
conferencing, and the real world in order to challenge our standard manifestations of
co-presence.
6.1 Network Culture: Strength in Numbers
While striving to create hybrid experiences that explore new modes of telepresence,
network culture becomes a powerful tool for generative, collaborative, and distributed
creation. Kazys Varnelis sees our current network culture as a distinctly new
phenomenon, defined by the mix of real and virtual space and new forms of
participatory media. While the effects this new culture seem minor to each of us as
individuals, collectively it’s a radical shift. In the network, individuals become less
important than the emergent outcome of the connections between people (and
machines). Naturally, the process of remixing from many sources is the dominant form
of creation in this highly interconnected culture, dissolving ideas of individual
authorship (Varnelis). As such, experimenting with the affordances of network culture
within my project is very timely and highly relevant to our current experience,
particularly in relation to remixing real and virtual spaces and participatory media.
In many ways, Media Design Bingo is a tangible manifestation of our networked culture
in the real world. The individual game boards log the help each student received from
other students, essentially serving as a visualization of generosity within our real world
social network. Additionally, customizing the individual boards is explicitly encouraged
in the rules of the game, giving people freedom to freely express themselves. As a
Jayne Vidheecharoen
14
result, the ways in which people chose to customize their individual boards combined
with the network activity generates a collective image of the studio culture that can’t be
predicted by any individual or an algorithm. The unique award badges created at the
end of the game also serve as a visualization of the connections between students,
demonstrating the emergent outcome of the network. Testing the capabilities of crowd
sourced creativity more directly, I create a HIT (Human Intelligence Task) for Amazon’s
Mechanical Turks and pay 20 people $0.25 each to modify a panorama. The space
evolves as various people from all over the world playfully add their modifications and
alter the modifications of others. While the test is relatively small and individual
modifications are minor, over time it becomes quite lively and demonstrates how a
system of distributed creation could work. I imagine that if Portals were widely available
(and fully featured) they would enable a similar kind of emergent and collaborative
activity. Launching and running a Kickstarter campaign to raise funding for Portals also
serves as another form of public research into our current networked culture and the
power of the internet. The 30 day campaign has ended with the project 190% funded
by 118 backers, with $2,853 pledged. About 35% of the funding comes from my
personal network and their extended networks, with the other 65% of funding from
random strangers on Kickstarter. These results clearly demonstrate our desire to
participate and help each other through the network as a collective.
Jayne Vidheecharoen
15
6.2 Network Culture: Uncanny
While our network culture creates a whole new range of possibilities, it also creates an
experience of tension within our minds as we reconcile the experience of transcending
space and time. When we can get updates from the other side of the country at the
speed of a tweet and travel the world with a drop of the Street View Pegman, our
concept of the here and now blurs with the there and then. Through my experiments I
intentionally aim to further muddle the distinction in order to create an uncanny
experience for the user. Kio Stark explains that playing with the uncanny is a
particularly powerful tool because of its ability to cause people to question a
fundamental understanding, even if only momentarily. These uncanny experiences
often cause us to suddenly reinterpret our experience using “primitive beliefs” such as
magic or coincidence. But we are less likely to feel like something is uncanny when we
are more familiar with our environment. As we become more and more accustomed to
living in a network culture we run the risk of becoming immune to the strangeness of
our experience. Thus, provoking an uncanny feeling can serve as a way to jolt us from
our daily routines as a node in the network and cause us to question the things we
currently just accept as normal.
In order to create an uncanny experience, I explore alternate ways to show different
points in time within a single space in my panorama mashups. In Pan, looking around
horizontally reveals a different point in time each time the viewer returns to the space.
Scrub, on the other hand, stacks time vertically like a film strip so the viewer can see
Jayne Vidheecharoen
16
one slice of a space over time while also being able to look around a space at a
specific time. Both experiments explore the implications of being able to interact with
and remix a time-based media (for instance animation or film) within the context of a
traditionally static environment. As these extra layers of reality co-exist with a static
representation of reality, the user is able to weave their own non-linear understanding
of the space. Similarly, in Portals the layering and compositing of static images from
the past, footage of the present, and animated interactive stop motion creates a
collage of multiple realities. The collage is by multiple authors and is neither real-time
nor pre-recorded. By creating an uncanny experience of space and time I also strive to
draw attention to the way in which our “authentic” representations of real space, in
virtual environments, are actually stitched together from various moments in time,
breaking the ties between space and time. In doing so I am suggesting alternative
illusions of virtual co-presence that are equally open to a non-linear time structure.
7.0 Conclusion: Starting Principles
From the theory and experiments conducted thus far I have developed several
principles to guide my work moving forward. First, by leveraging spacial and technical
constraints, use play as a tool to disrupt and challenge existing magic circles in the real
world. Through this act, instigate more improvisational interactions between both
human to human and human to computer. Second, embrace alternative “truths” as a
tool for sharing ideas and inviting participation from others. By provoking questions
about future technological developments, hopefully this technique ensures the project’s
Jayne Vidheecharoen
17
core ideas can continue exist beyond its original “actual” form. Third, combine the
affordances of the virtual world, the physical world, and the human imagination in order
to create an experience formerly impossible in the virtual or physical world separately,
and therefore more authentic to the hybrid nature of the experience. Finally, using both
the strength and strangeness of network culture, create an uncanny experience in
order to disorient users and disrupt our habitual routines as active participants in the
network.
With these core principles in mind, I feel well prepared to make progress on my thesis
project through the Spring term. For the duration of the term, I intend to focus
specifically on developing Portals as a system. This includes the design of the objects
in addition any supporting materials as part of the system. Hopefully the creation of
this system will both address the issues outlined above and be the start of many other
investigations to come.
Jayne Vidheecharoen
18
Works Cited
Bavinton, Nathaniel. “From Obstacle to Opportunity: Parkour, Leisure, and the
Reinterpretation of Constraints.” Annals of Leisure Research. 10.3. (2007): 391.
PDF File.
Crawford, Margaret. “Contesting the Public Realm: Struggles Over Public Space in
LA.” Journal of Architectural Education. 49.1. (1995): 4-9. PDF File.
Dennett, Andrea. Weird and Wonderful: The Dime Museum in America. NYU Press,
1997. Print.
Harris, Neil. Humbug: The Art of P. T. Barnum. Phoenix ed. University Of Chicago
Press, 1981. Print.
Hilbrecht, Margo. “Changing Perspectives on the Work–Leisure Relationship.”
Annals of Leisure Research. 10.3. (2007): 368-384. PDF File.
Huizinga, Johan. Homo Ludens: The Play Element in Culture. Boston, Massachusetts:
Roy Publishers, 1950. Print.
McGonigal, Jane. Reality is Broken: Why Games Make Us Better and How They Can
Change the World. New York, NY: Penguin Press, 2011. Print.
Jayne Vidheecharoen
19
Slavin, Kevin. “Kevin Slavin: How Algorithms Shape Our World.” Ted Talks,
Jul. 2011. Web. 4 Dec. 2011.
Stark, Kio. “Notes on Authenticity and Uncanny discussions.” For Reals: Fall 2010.
Oct. 2010. Web. 16 Jan. 2012.
Varnelis, Kazys. “Conclusion: The Rise of Network Culture.”
Networked Publics. Web. 4 Dec. 2011.
Wark, McKinzie. A Hacker Manifesto. Cambridge, Massachusetts: Harvard University
Press, 2004. Print.
Jayne Vidheecharoen
20