PHYSICAL PIXELS

by

Kelly Bowman Heaton

B.A., Yale University (1994)

Submitted to the Program in Media Arts and Sciences,School of Architecture and Planning, in partial fulfillment of the requirements

for the degree of

Master of Science in Media Arts and Sciences

at the

MASSACHUSETTS INSTITUTE OF TECHNOLOGY

June 2000

© Massachusetts Institute of Technology 2000. All rights reserved.

AuthorProgram in Media Arts and Sciences,

School of Architecture and Urban PlanningMay 5, 2000

Certified byMichael Hawley

Assistant Professor of Media Arts and SciencesAlex W. Dreyfoos (’54), Jr. Career Development Professor

of Media Arts and Sciences

Accepted byStephen A. Benton

Chair, Departmental Committee on Graduate StudentsProgram in Media Arts and Sciences

2

Abstract

The picture element, or pixel, is a conceptual unit of representation fordigital information. Like all data structures of the computer, pixels areinvisible and therefore require an output device to be seen. Thephysical unit of display, or physical pixel, can be any form that makesthe pixel visible. Pixels are often represented as the electronicallyaddressable phosphors of a video monitor, but the potential for differentvisualizations inspires the development of novel phenotypes. Four newsystems of physical pixels are presented: Nami, Peano, the DigitalPalette and 20/20 Refurbished. In each case, the combination ofmaterial, hardware and software design results in a unique visualizationof computation. The chief contribution of this research is thearticulation of a mode of artistic practice in which custom units ofrepresentation integrate physical and digital media to engender a newart.

3

PHYSICAL PIXELS

Thesis Committee

Thesis ReaderMichael Hawley

Assistant Professor of Media Arts and SciencesAlex W. Dreyfoos (’54), Jr. Career Development Professor

of Media Arts and Sciences

Thesis ReaderJohn Maeda

Assistant Professor of Design and Computation at the MIT Media LaboratorySony Career Development Professor of Media Arts and Sciences

Thesis ReaderWilliam L. Verplank, PhD

CCRMA, Stanford University

4

Acknowledgments

Thank you to everyone who helped to reveal the physical pixel.

Charlotte Burgess

Carlos Cantu

Andrew Gallant, Peter Morleyand the MIT Central Machine Shop

Steve Gass, Deborah Helmanand the Barker Engineering Library

Steve Gray

Saul Griffith

Michael Hawleyand the Personal Information Architecture Group

Kristin Hall

Camilla Holten

Hiroshi Ishiiand the Tangible Media Group

Golan Levin

Carl Machover

Jacob K. Javits Fellowship Program

John Maedaand the Aesthetics and Computation Group

Tim McNerny

NECSYS

Nicholas Negroponte

Maggie Orth

MIT Council for the Arts

Paul Pham

Robert Poor

Matt Reynolds

Joel Rosenberg

Scott Snibbe

Brygg Ullmer

Bill Verplank

My family and friends

5

Table of Contents

Introduction 7

The Slow Birth of New Media 8

The Evolution of Pixel 12

Beyond the Monitor 17

Four Physical Pixels 20

Nami 21Digital Palette 26Peano 2920/20 Refurbished 37

Discussion 39

Conclusion 42

Appendix: The Design of a Modular Display System 44

Bibliography 48

6

7

Introduction

The effort to make computation visible will always be affected by thematerial interpretation of what is essentially immaterial. Yet, like allforms of art, it is through the revelatory process that we come tounderstand that which cannot be revealed. This thesis examines thepicture element, or pixel, as one method for visualizing the numeric andelectronic abstraction of computation.

The deconstruction of images into a quantifiable unit of representationis at least as old as the tile mosaics of ancient Greece. In contrast to amosaic tile, which has an appearance intrinsic to its physicality, apicture element begins as a non-visible quantum of information. Itserves to represent, but has no appearance per se.

To be seen, the pixel needs a physical form hereafter termed thephysical pixel. To date, video has been the predominant medium formaking pixels visible, along with tapestry, punch cards and print. Allof these forms are different visualizations of the same fundamental datastructure. Due to variation in physical media, different output deviceswill provide a unique experience of the same digital information. Thephysical appearance of a pixel is a matter of interpretation, a specificmaterial manifestation of an invisible medium.

I present four research projects with four physical pixels, each of whichoffers a different perspective on the computer as a medium for artisticexpression. The basic organization of visible data is consistentthroughout, but each project defines the pixel as a differentcomputational and material unit. In the Nami example, our pixel is awireless network node with full-spectrum colored light. The DigitalPalette treats the pixel a source of electronic “paint” for coloring otherphysical pixels. In the Peano project, I demonstrate how the pixel canbe used as a building block for computer animated sculpture. Finally,20/20 Refurbished presents the observer with a reactive surface madeof Furbies™ that are modified to behave as pixels with personality.This body of work is defined by a mode of artistic practice in whichcustom units of representation unite physical and digital media. Likethe painter’s brushstroke or sculptor’s touch, the physical pixel is themark of a new art.

This research is presented as part of a tradition of opticaldeconstruction in the visual arts, including the history of computergraphics. I include several prior examples in which physical pixelshave revealed a novel insight into the nature of computation. Finally, Iendeavor to understand what is consistent about the pixel throughoutany material manifestation, and therefore any art made with a pixel-based organization of digital information.

The principle contributions of this thesis are:

1. The articulation of a new media termed the physical pixel2. Four examples of physical pixels, each of which encapsulates a

unique combination of physical and digital into a synthetic form3. A conceptual framework for the design of physical pixel media

What is a pixel? A pixel is a unit ofrepresentation with a data structureand a physical form.

Each of these circles encapsulates aunit of representation with differentproperties, as described in this thesis.Italics indicate the display system.

Red, green, blue LEDWireless network node

Touch responsiveProgrammable

Nami

Red, green, blue LEDTouch responsive

Programmable by example

Digital Palette

Red, green, blue LEDTouch responsive

Modular network nodeProgrammable

Peano

Animatronic toyPop culture personality

Artificially sentientProgrammable

20/20 Refurbished

8

The Slow Birth of New Media

Among Marshall McLuhan’s most important contributions totwentieth century thought is “The medium is the message.”His statement is not meant to proclaim the disappearance ofcontent, but “merely to say that the personal and socialconsequences of any medium -that is, of any extension ofourselves- result from the new scale that is introduced into ouraffairs by each extension of ourselves, or by any newtechnology.”1 In other words, stories, mythologies andcharacters do not change significantly with time, but mediachange radically; and the way in which we perceive content isequally if not more influenced by the message of a mediumthan the narrative itself. The skillful use of a medium iscritical to the success of an artist. Instinctively, we knowwhen there is a lack of true ownership over the material fromwhich an artwork was made. A message inappropriatelydelivered in paint will look as wrong as a eulogy written inicing on cake.2 This is an extreme example, but the moresubtle situations are those which regularly inflict ourcommunication with unwitting friction and misunderstanding.In the best case, a medium will enhance a message bycontributing implicit meaning to an artist’s explicit intention;but weak content in a strong vehicle, or strong content in thewrong vehicle, will not be delivered effectively. Contentaside, the misguided media-vehicle will drive our attentionelsewhere.

Unpacking the message of a medium is essential to creativework, enabling the artist to see the material for what it is andnot what it seems to be. In his essay “The QuestionConcerning Technology”, Heidegger describes this essentialpursuit in terms of technç, the Greek root for the wordtechnology, originally associated with “knowing in the widestsense … to be entirely at home in something, to understandand to be an expert in it.”3 Technç embodies the wisdom thatprecedes poiçsis, or the “poetical bringing into appearance andconcrete imagery”.4 In the absence of technç, an artist will behindered from this poetic process of revealing truth, for “Onlythe true brings us into a free relationship with that whichconcerns us from out of its essence.” 5

At this early stage in development, art made with thecomputer6 is still undergoing differentiation from previousmodes of expression. Work in new media typically involves a

1 McLuhan, Marshall. Understanding Media: the extensions of man. The MIT Press, Cambridge, MA, 1994. Page 7.2 Note that a failure of this type is due to both societal perception and the impracticality of cake icing for permanent record. Theinappropriateness of the medium is partly cultural, and partly essential to the material.3 Heidegger, Martin. The Question Concerning Technology and Other Essays. Harper and Row Publishers, Inc., New York, NY,1977 (Hereafter referred to as TQCT). Page 14.4 TQCT, page 10.5 TQCT, page 6.6 As Charlotte Burgess astutely observes , “art made with a computer” has no strict definition at this time. Use of the phrase“computer art” is avoided in this document because it merits careful definition, an exercise that extends beyond the scope of thisthesis.

Bush Model TV56 405 Line TV

The computer monitor. From the ConracCorporation’s Raster Graphics Handbook (SecondEdition). Van Nostrand Reinhold Company, Inc.,New York, NY. 1985. Note the similarities in thetelevision and desktop computer interfaces.

9

transposition of previous experience, regardless of whether ornot the content is appropriate or interesting in the newmedium. In his 1932 article, A New Instrument of Vision,Mohology-Nagy summarizes the influence of “the aesthetic-philosophic concepts that circumscribed painting” on the earlyyears of photography:

Up to now, photography has remained in rather rigiddependence on the traditional forms of painting; andlike painting it has passed through the successivestages of all the various 'isms'; though ill no sense toits advantage. Fundamentally new discoveries cannotfor long be confined to the mentality and practice ofbygone periods with impunity. When that happens allproductive activity is arrested.7

Art made with a computer is no exception to this tendency. In1975, Anne Murray stated “The first decade ofexperimentation has shown that the computer can beprogrammed to simulate the styles of previously existing art,”8

as though emulating pre-existing art is somehow a prerequisiteto work in a new medium. Murray continues, “graphicdesigners, printmakers, sculptors, film makers, architects andchoreographers… prefer to work in conjunction with aprogrammer who can translate their ideas into themathematical language of the machine.”9 Her description ofthe programmer as translator for the artist is reminiscent ofMuybridge’s “specific intention” in photography “to create anatlas for the use of artists, a visual dictionary of human andanimal forms in action.”10 In both cases, the technologist –programmer or photographer- was not considered the artist,but servant to an artist and his/her preconceived notions aboutart in an unfamiliar medium.

The computer is often treated as a means for representingtraditional forms of art. This perception can be dangerousinsofar as it can mistake art represented on a computer withthe original form, not unlike the confusion of printedreproduction with an authentic painting.11 The visual narrativemay be the same, but the media of one art is not necessarilyinterchangeable with another. In Being Digital, NicholasNegroponte writes,

Computers and art can bring out the worst in eachother when they first meet. One reason is that thesignature of the machine can be too strong. It canoverpower the intended expression … The flavor of

7 Moholy-Nagy, Laszlo . A New Instrument of Vision , 1932.8 Anne H. Murray quoted from: Leavitt, Ruth. Artist and Computer. Crown Publishers, Inc., New York, NY, 1976 (Hereafterabbreviated A&C). Page 3.9 Anne H. Murray quoted from A&C, page 1.10 Newhall, Beaumont, The history of photography : from 1839 to the present. Museum of Modern Art, New York, NY, 1982.11 Walter Benjamin discusses the profound implications of printed media for painting in his famous essay, “The Work of Art inthe Age of Mechanical Reproduction.”

Eadweard Muybridge’s The Horse in Motion(1878). Developed as part of an “atlas for the useof artists.”

This is not a painting. Adapted from RenéMagritte, The Betrayal of Images (1929).

10

the computer can drown the subtler signals of theart.12

That art made in other media does not jibe with the computeris no fault of the machine. To view a painting on a monitor isto alter the identity of the work entirely, although certainly notto transform the work into “computer art.” In his 1965-66Brushstrokes series, Roy Lichenstein made brilliantcommentary on the transposition of painting into the printmedium by returning the brushstroke to the canvas as asubject, rendered in paint as it appears in print. Lichensteinreminds us that the substance of one medium is simulacrum inanother. Although his brushstroke subjects arecompositionally predominant, the paintings are about arelationship to print. The “flavor of the computer” thatdrowns “the subtler signals of the art” is the essence of mediathat Lichenstein harnesses to make his art.

Numerous artists have emerged with no distinction betweencomputer scientist and creator, producing artworks that haveno comparable identity in any other medium. Nevertheless,computer graphics is nearly synonymous with screen-based,and the pixel is commonly perceived as a phosphor of adesktop monitor.13 To be sure, the popularity of the videomonitor has much to do with its availability and high-qualityimage, but a normative concept of image representation is alsoinvolved. The rectangular monitor, the flat screen, theuniform surface texture, high realism and the Cartesian imageplane, characterize the majority of computer graphics. It is nocoincidence that these are descriptors that could be equallyapplied to a high Renaissance painting, also heavily influencedby the philosophy of René Descartes. Perhaps it is the verysame fear of infinite reality that inspired Descartes to seekproof of the existence of everything, including God, whichmotivates the contemporary thinker to seek Cartesian order inthe approximate infinity of digital information.14

Although innovative art has been made with the computer,metaphor impedes a deeper understanding. In part, metaphoris our only access to an invisible medium; most concreterepresentations of computation look like something else. Themonitor is like the frame, the screen is like the canvas, thesoftware tools simulate brushes, and many peripheral devicesseek comparison to instruments of the draftsman. This is achallenge for every art form, made all the more treacherous assimulation by one medium of another becomes increasinglyfacile. In 1961, Clement Greenberg commented, “What had to 12 Negroponte, Nicholas. Being Digital. Random House, Inc., New York, NY, 1996. Page 223.13 Many people mistake the RGB phosphor for the pixel as opposed to a material instance of a pixel.14 Sean Cubitt traces the history of the grid through Descartes to the foundations of Christianity itself: “The grid derives itsonscreen presentations from modernist design practice, which itself can be traced back to Descartes’s invention of neutral spacedefined only by coordinates rather than contents, and to Mercator’s redefinition of the map as a blank field of longitude andlatitude into which the marks of coordinate space can be drawn. But behind this insipid, unmarked space can be glimpsed anolder grid deployed in the mediaeval scriptoria, a grid made up not of modular, transferable boxes, but of points and cruxes, ineach of which the divine and the sublunary met. From: Cubitt, Sean. Digital Aesthetics. Sage Publications Ltd, London, 1998(Hereafter referred to as DA). Page 89.

Roy Lichenstein’s Big Painting VI (1965). Oiland Magna on canvas, 92.5 x 129 inches.Kunstsammlung Nordrhein-Westfalen,Dusseldorf.

The Cartesian grid of computer graphicsillustrated in full glory. Image from thevideotape “Study of a Numerically ModeledSevere Storm,” National Center forSupercomputing Applications, University ofIllinois at Urbana-Champaign. Caption andillustration adapted from Edward Tufte’s VisualExplanations: Images and Quantities, Evidenceand Narrative. Graphics Press, Inc., Cheshire,CT, 1997.

11

be exhibited and made explicit was that which was unique andirreducible not only in art in general but also in eachparticular art…”15 (Italics mine). To the extent that newmedia inspires a reinvention of traditional expression, aconnection to the past can be deeply meaningful; but thereproduction of old ideas is tiresome. Art made with thecomputer must consciously parry comparison to reveal a freshperspective on historic art forms instead of mimickingtradition. The danger of metaphor derives from its purchase ofa mind frame from pre-existing media, as observed by CharlesCsuri, “Most people are inclined to think of computer art asstatic graphics or animated film.” The connection to painting,film and video is further affirmed in the terminology ofprogramming itself.16 Metaphor may be a useful device foruser interface, but adherence to the familiar can seduce anartist to “shut [their] eyes to the constellation of truth afterwhich we are questioning.”17 As stated by Steven Holtzman,“If virtual worlds are mere reflections of the natural, built onborrowed metaphors from the real world, then indeed we maycreate little more than the antithesis of the natural.”18 Whenquestioning the computer as a creative medium, care should betaken that preconceptions do not preempt a revelation.

15 O’Brian, John (Editor). Clement Greenburg: The Collected Essays and Criticism Vol. 4 - Modernism With a Vengeance, 1957-1969 .16 Metaphor permeates the semantics of programming languages. The examples are too numerous to recount, but suffice a few:Java contains methods for creating a canvas on which to draw , paint and repaint; Lingo conducts animation of cast members onthe stage .17 TQCT , page 35.18 Holtzman, Steven. Digital Mosaics: the aesthetics of cyberspace. Simon and Schuster, Inc., New York, NY, 1998. Page 61.

Three (rather limited) options for visualizingcomputation, adapted from the ConracCorporation’s Raster Graphics Handbook(Second Edition). Van Nostrand ReinholdCompany, Inc., New York, NY. 1985.

12

The Evolution of Pixel

The origins of the picture element derive from the cathode-raytube (CRT), a display technology popularized by televisionand later co-opted by computer graphics. Pixel, short forpicture element, emerged in the late 1960s as a way todescribe video.1 Despite the comparatively large storagerequirements of the raster display, computer scientistsrecognized that specifically addressing the entire grid of aCRT would lead to a dynamic and flicker-free, color image.2

With the development of low-cost memory in the 1970s,bitmap raster graphics replaced vector systems as thepredominant paradigm of computer graphics and the conceptof a digital pixel was born. As described by NicholasNegroponte, “The real power of the pixel comes from itsmolecular nature, in that a pixel can be part of anything, fromtext to lines to photographs. Pixels are pixels is as true as bitsare bits. (…) The same bits can be looked at by the viewerfrom many perspectives.”3 Like any data, the digital pixel hasno absolute appearance; it is a mathematical description thatneeds to be interpreted by a physical output device in order tobe visible. Nevertheless, our contemporary concept of thepixel is concretely rooted in material origins, for reasonsexamined in this chapter.

The history of video has played a dominant role in thedefinition of picture element and thus bitmapped rastergraphics on the whole. To understand this influence requires alook at the development of video in the context of filmanimation, print, painting and other forms of image creation,which have influenced our traditional perception of the image.In 1948, at the dawn of television, Theodore Soller describesthe cathode-ray tube like a form of electric painting,

The cathode-ray tube is a form of visual indicator thatpermits an interpretation of electrical phenomena interms of a picture painted on a phosphorescent screenwhich is controlled in position and intensity byelectrical signals. Under the proper conditions, itpaints this picture in an extremely facile way, beingcapable of utilizing many millions of separate dataper second.4 (Italics mine)

This notion of a painting made by “electrical signals” isaltogether different from film animation, which at the time

1 Oxford English Dictionary. The specific reference is to a 1969 article in Science , vol. 15 (August). Further research into theorigins of pixel reveal a reference to the concept, if not the name, as early as 1940, in which M. W. Baldwin attributes imagesharpness to “the number of elemental areas in the image.” He quantifies resolution of these elements in terms of “liminal units.”(From: M. W. Baldwin’s “The Subjective Sharpness of Simulated Television Images.” Proceedings of the I.R.E., 1940. Page460.) Thanks to Steve Gass for his generous assistance with the search for pixel etymology.2 A. Michael Noll of Bell Telephone Laboratories published the seminal paper on this topic in 1971, “Scanned-Display ComputerGraphics.” He uses the term “dot” or “point” instead of “pixel,” presumably an indication that the language of computer graphicshad yet to merge with video terminology.3 Negroponte, Nicholas. Being Digital. Random House, Inc., New York, NY, 1996. Page 107.4 Soller, Theodore (Editor). Cathode Ray Tube Displays. McGraw-Hill Book Company, Inc., New York, NY, 1948. Page 1.

The pixel as we know it.

Cathode-ray tube images, circa 1948. FromCathode Ray Tube Displays (Theodore Soller,Editor). McGraw-Hill Book Company, Inc.,New York, NY, 1948.

13

Soller made his observation, was the best-known medium fordynamic imagery (and is still popularly considered to be thenearest cousin of the video image). Despite the superficialsimilarities of film and video, Soller alludes to a critical dividebetween the two media. Whereas cinema creates the illusionof motion using the mechanical sequencing of high-resolutionframes, video is the result of “many millions of separate dataper second” changing in unison to present a coordinated imageon an immobile output device. In this respect, videoparticipates in a trend of pictorial deconstruction that beganwith the daguerreotype and Impressionism, was “echoed laterin Seurat’s pointillisme , and is still continued in the newspapermesh of dots that is called “wirephoto.”5 Through a reductionof the image “into a fabric of tiny, uniform, color-bearing dots… repeated over the surface of the canvas with machine-likeregularity,6” video unites the temporal image with the spatialplasticity of painting.

While the cathode-ray tube was still undergoing developmentin the hands of engineers, the notion of a video-like mediumoccupied the minds of many artists. Abstract animators suchas Hans Richter and Oskar Fischinger handmade preciousframe after frame of film to create the moving painting.Notably, Alexander Alexeieff and Claire Parker painstakinglycrafted a physical screen of 1,000,000 pins that they sculptedto achieve a movable surface for their film animation. Kineticartists, including Alexander Calder and Yakov Agaam,incorporated motion into art through the use of naturalphenomena and viewer interaction with sculpture. Inparticular, Agaam experimented with a wide variety ofmoving parts, modular compositional elements and sculpturalsurfaces in which an observer could participate to experience adynamic artwork. Beginning in the 1950s, Mary Ellen Buteand later, Nam Jun Paik, were among the first to directlymanipulate the analog signal to a cathode-ray tube, usingelectronics for abstract effects within the video medium, asopposed to recorded by the film medium. Through the use ofmagnets and circuitry, Bute, Paik and other early video artistswere able to paint with time-varying electrical phenomenadirectly onto the video medium itself.

Whereas early work with the cathode-ray tube depended on ananalog source for the visible signal, computer science elevatedthe electronic painting to a new level of mathematicalabstraction and discrete control. It became unnecessary toknow the nuts and bolts of video electronics; programmingafforded linguistic access to the “black box” terminal. Digitalprocessing was an attractive alternative to the time-consumingprocess of film animation and the analog-dependency oftelevision. In the late 1950s, artists began a gradual shifttowards the computer as a medium, as explained by JohnWhitney regarding his animation machine of 1958,

5 McLuhan, Marshall. Understanding Media: the extensions of man. The MIT Press, Cambridge, MA, 1994. Page 190.6 Anne H. Murray from: Leavitt, Ruth. Artist and Computer. Crown Publishers, Inc., New York, NY, 1976. Page 1.

Detail from George Seurat’s The Circus (1891).Oil on canvas, 73 x 59 1/8 inches. Museed'Orsay, Paris.

Mary Ellen Bute (1954). Photograph by TedNemeth, from Robert Russett and Cecile Starr’sExperimental Animation (Revised Edition). DaCapo Press, Inc., New York, NY, 1976.

Yakov Agaam’s Lumiere de minuit (MidnightLight) showing the visible changes as the samepiece is viewed from four different perspectives.From Frank Popper’s Agaam (Third RevisedEdition). Harry N. Abrams, Inc., New York, NY,1990.

14

[My animation machine] is actually is doingmechanically what an oscilloscope would do; andthat’s when I began to realize that what I was doingmechanically could be done on the cathode ray tubecomputer terminal.7

Through the computer, mathematics provided a new means foranimating an electronic image, achieving effects that werepreviously restricted to direct manipulation of the displaydevice. Mathematics reduced the image to information fromwhich an illusion could be modeled numerically, as opposedto media that captured or physically built a picture. Asdescribed by Richard Norman in his text Electronic Color,

Unlike the image on a television screen, the computerpicture is the invented image of its designer, a plasticimage that can be shaped and changed, an image thatis alive and capable of responding to furtherinformation.8 (Italics mine)

A critical shift occurred when the means by which informationwas used to manipulate an image transitioned from analog todigital. Although film, photography, print and television areall technological, the image-making capabilities of thesemedia are accessible through manual control, even if suchphysical craft necessitates electronic components. In contrast,the programmer’s art is abstract logic organized into computerinstructions, like a form of machine age poetry. The computerprogram is an intangible set of commands by which a machineexecutes logic; and according to this logic, a system ofphysical pixels (or other display device) illustrates the state ofthe machine. With bitmap raster graphics, the video pixelevolved from an analog locale on a cathode-ray tube into apoint with mathematically specific syntax.

Logical access to every pixel of the video monitor, combinedwith sufficient data storage, rapidly broadened the pictorialapplications of the medium. Video games are an obviousexample, but nearly every other form of graphical userinterface of the past thirty years has followed in the videotradition as well. Bitmapped raster graphics on the videomonitor rapidly became the popular perception ofcomputation. Although “jaggies” were a nuisance to earlyraster graphics, Frank Crow and other graphics pioneers of theearly 1970s made it a priority to “perfect” the computer-controlled video image through the elimination of pixelevidence9. Thanks to their emphasis on higher level contentand virtual realism, pixels have become practically invisible inscreen-based computer graphics.

7 Russett, Robert and Starr, Cecile. Experimental Animation (Revised Edition). Da Capo Press, Inc., New York, NY, 1976.Page 184.8 Norman, Richard B. Electronic Color: the art of color applied to computer graphic computing. Van Nostrand Reinhold, NewYork, NY, 1990. Page 72.9 Franklin C. Crow’s seminal paper on antialiasing was presented to the SIGGRAPH community in 1977, “The Aliasing Problemin Computer-Generated Shaded Images.”

Nam Jun Paik’s direct manipulation of the videosignal in Magnet TV (1965). Photograph takenfrom the Guggenhiem Museum web site.

Jaggies: the inevitable consequence of continuousgeometry on a finite grid. Conrac Corporation.Raster Graphics Handbook (Second Edition).Van Nostrand Reinhold Company, Inc., NewYork, NY. 1985.

Realism in computer graphics. Illustration fromNewman and Sproull’s Principles of InteractiveComputer Graphics (Second Edition). McGraw-Hill Book Company, Inc., New York, NY, 1979.

15

Nevertheless, the pixel remains integral to our perception ofthe computer, just like the brushstroke, the tooth of paper, thebrand of film or the grit of clay. Certain artworks specificallybring our attention to the tiny physical pixels of the screen, asexemplified by John Maeda’s Single Pixel (1995). Otherscreen-based work creates graphic representations of the pixel,such as the down-sampled portraits made by Ed Manning’sBLOCPIX™. Similarly, anyone who has “zoomed down tothe level of the pixel” in Adobe Photoshop™ is familiar withtheir illustration of the image pixel as an evenly coloredsquare. Leon Harmon and Kenneth Knowlton (Studies inPerception, 1966-67), later followed by Robert Silvers(Photomosaics™, 1997), substituted the uniformly coloredsquare with digital pictures, treating the graphical pixel asthough it were a collage element.

As the center of attention, or the unit that constitutes an image,the pixel will always represent digital fragmentation. Part ofthe broader trend in scientific deconstruction, the pixel isconsidered by many to be the ultimate form of visualrepresentation, as stated by Carl Sagan,

Any work of art is ultimately pointillist. There are alarge but finite number of atoms or molecules in apainting or sculpture. Color and brightness have nomeaning on any finer scale. If you knew the position,reflectivity, and “color” of every small molecule in awork of art, you could, in principle, reconstruct asmany identical copies as you wished, each of themequally original.10

This mentality goes hand in hand with the Cartesianphilosophy that reality is reducible to a mathematical equation,and it is only a matter of time before we analyze the world intotiny bits and reconstitute it according to our own controlalgorithms. Of course, there are ways to visualizecomputation that do not involve the reduction of everythinginto dots. Digital information can be used to generate acontinuous representation such as the vector display, or thecomputer-generated drawings of Harold Cohen and CharlesCsuri. The use of pixels to visualize computation is a choicethat affects the meaning of the art.

The physical qualities of the pixel also affect our perception.The flatness and luminescence of a monitor contribute to aunique experience of pixel graphics, no matter howimperceptible the individual units of representation may be.The structure and physical properties of the video displaydevice are fundamental to any screen-based pixel, be it anactual physical pixel or a representation of the pixel. In somesituations, the qualities of the screen are problematic. Forexample, visualizations that are three-dimensional, modular ortangible are hard to represent with a single monitor. The need

10 Prueitt , Melvin L. Art and the Computer. McGraw-Hill Book Company, Inc. New York , NY 1984., Page viii.

Ed Manning’s portrait of Ralph Nader usingBLOCPIX™ (1973). From Ruth Leavitt’sArtist and Computer. Crown Publishers, Inc.,New York, NY, 1976.

The digital image as pixel. Robert Silvers,“Portrait of George Washington from the onedollar bill -- for Mastercard "The Future ofMoney" promotion.” (c. 1997). From:http://www.photomosaic.com/gallery.htm

Silicon atoms: the ultimate pixel. Photo fromP.W. Atkins’ Molecules . W. H. Freeman andCompany, New York, NY, 1987.

16

for alternate visualizations has motivated a variety of newoutput devices for computation. In the next section, I discussthe history of physical pixels beyond the monitor.

An image that was not made from pixels.Charles Csuri’s Sine-Curve Man (1966).Silkscreen on plastic, 36 x 40”.

17

Beyond the Monitor

The digital pixel, often made physical as a phosphor element,can just as well be visualized through light bulbs or actuatedtiles or swarming ants or any other discrete unit that can bemade to change visibly in response to digital information. Forexample, Eric Staller’s Lightmobile (1985) uses 1,700computer-controlled light bulbs to animate a car. The choiceof form is a matter of aesthetic preference. There are anendless variety of visualizations made possible by variousmaterials and organizations of pixels, just as there arevisualizations made possible by data structures other thanpixels.

Pixels beyond the monitor are often static. For example, theprinted image is intended to create a permanent artifact ofcomputation that does not respond to further digitalinformation. By contrast, the research presented in this thesisis specifically concerned with physical pixels that changevisibly with the time-varying state of a computer.

John Frazer and his colleagues at the London ArchitecturalAssociation have been innovating alternate representations ofcomputation since the late 1960s. For thirty years, Frazer’sresearch has been motivated by an “evolutionary architecture”,or one in which the computer contributes generative rules,responsive behavior and other life-like qualities to buildingstructures. Not surprisingly, he finds the two-dimensionalityof monitors and plotters restrictive for his exploration of athree-dimensional material space, and his background inarchitecture naturally leads him to physical modeling systems.The 1978 Generator Project was among the first of many“intelligent physical modelling” systems that Frazer wouldconstruct in his quest to “get away from conventional, drawingboard dependent design approaches.” He writes,

Important new ideas emerged from the Generatorproject. These included embedded intelligence andlearning from experience during use, theisomorphism of processor configuration and modelstructure, and the question of consciousness.1

In 1989, Anagnostou (et al.) developed a similar system forengineering problems wherein the use of a two-dimensionalmodel “remains both tedious and complex.”2 In their paperentitled “Geometry-defining processors for engineering designand analysis,” Anagnostou reinforces Frazer’s frustrationswith the flat interface, stating “Unfortunately, the intrinsicallytwo-dimensional graphics tablet is no longer an ideal interfacein the context of three-dimensional geometries.”3

1 Frazer, J. An Evolutionary Architecture. Architectural Association: London, 1994 (Hereafter abbreviated AEA). Page 41.2 Anagnostou, G., Dewey, D., and Patera, A, “Geometry-defining processors for engineering design and analysis,” in: The VisualComputer, 5:304-315.3 Ibid.

Eric Staller’s Lightmobile (1985). A computercontrols the 1,700 bulbs to flicker in twenty-threedifferent patterns of light. From Goodman, Cynthia.Digital Visions: Computers and Art. Harry N.Abrams, Inc., New York, NY. 1987.

Frazer’s Intelligent Beermats (1980) and Three-dimensional Modelling System (1980). Photo fromFrazer, J. An Evolutionary Architecture . ArchitecturalAssociation: London, 1994.

18

Nevertheless, the early building systems of both Frazer andAnagnostou persist with a video display to visualize thecomputation.

In 1990, in his “most ambitious of these models,” Frazerincorporated colored light into the building blocks of theUniversal Constructor making the virtual relationships andbehaviors of a three-dimensional structure visible without theaid of a monitor. Frazer describes the revolutionaryadvantages of display intrinsic to the objects themselves,

Each cube could have any one of 256 states which weredisplayed by means of eight light-emitting diodes (LEDs).Thus the eight-bit code could be used to map the state ofthe cell to any form or structure; to environmentalconditions such as wind; to sound, or even to dance. (…)The range of different applications was diverse andincluded three-dimensional cellular automata respondingto a site problem, a curve-fitting program controlled by aFibonacci series, and an encoding of the Laban dancenotation.4

Frazer’s Universal Constructor uses visual feedback toencourage the interactor to modify the quantity, spatiallocation and orientation of the physical pixels that he calls“cells”. Frazer explains,

In a typical experimental application the system willrequest an interactor to configure an environmentconsisting of cells in different states. Using lights, themodel then indicates its proposed response by asking theinteractor’s assistance in adding or removing units. Theparticipator can in turn modify the environment.5

The computer and the display become an integral unit. Thisnotion of the graphics embedded in a system of networkedcomputers has also been implemented in ProgrammableBeads [Borovoy, Kramer 1997] and Stackables [Kramer,Minar 1997]. In all of the cases described, patterns of coloredLED light are used to reflect the state of a network thatoperates rule-based programs of behavior, such as cellularautomata. As the building units are rearranged, the behaviorof the system responds to the new network geometry in theway that flocking birds assume flight patterns, traffic jamstake form or ant colonies self-organize. Whereas thesemodular systems are intended for physical interaction by theuser, other artists have made the pixel a three dimensional unitin stand-alone sculpture. Artist Nam Jun Paik has used theentire television as a unit of construction and graphicalrepresentation in his sculpture. In Nicholas Negroponte’s Seek(1969-70), a machine arm rearranges paper blocks to build adynamic architecture for gerbils, also creating an interestingdisplay system wherein the blocks are movable pixels.

4 AEA, pages 45.5 AEA, page 49.

Frazer’s Universal Constructor (1990), showing thevertically stacking units of construction, eachapproximately eight inches on a side. Photo fromPhoto from Frazer, J. An Evolutionary Architecture.Architectural Association: London, 1994.

Rick Borovoy and Kwindla Kramer’s Stackables(1997). Photo from:www.media.mit.edu/projects/beads/index.html.

Nicholas Negroponte's Seek (1969-70). Photo fromCynthia Goodman’s Digital Visions: Computers andArt. Harry N. Abrams, Inc., New York, NY. 1987.

19

Computation made visible by a modular building system isequally significant to sculpture as the computer has been forvideo, albeit less explored at this time.

Artists and engineers seek new forms of physical pixels tosolve a problem or create a personal aesthetic. JosephJacobson and the Micromedia Group at the MIT MediaLaboratory have tailored the shape of electrophoretic inkpixels to suit the geometry of a particular display application.Jacobson’s physical pixels are also flexible, suggesting afuture for electronic display that compares with the look andfeel of paper. Danny Rozen’s Wooden Mirror (1999) uses acomputer to convert video input into a real-time video imagemade of 840 wooden chips and the noise of their respectivemotors. The result is a unique reflection of the viewer inwood and sound. In her wall installation Chorus (1999), Sodamember Lucy Kimbell created a grid of “sonic greeting cardsopening and closing in choreographed teams for a raucous andunruly … choir.”6 In each of these cases, the inventor desiresa visualization of computation that cannot be achieved withpre-existing display technologies. The physical pixel presentsa valuable opportunity for specializing, personalizing orreinventing our perception of computation.

6 http://www.soda.co.uk/soda/chorus.htm

Danny Rozen’s Wooden Mirror (1999). Photo byMarianne K. Yeung.

Detail of an early E Ink™ demonstration, revealingthe variable pixel geometry specialized for ananimated display of text, below.

20

In the following sections, four systems of physical pixels aredescribed: Nami, the Digital Palette, Peano and 20/20Refurbished. The projects are presented in the order in whichthey were built to reveal the processes that led from one to thenext, although the flow of research was not strictly linear. Theidea for Peano was conceived shortly after Nami, but longbefore a mechanical design could make it a reality. In theinterim, the Digital Palette was developed to enhance theinterface to Nami. 20/20 Refurbished was motivated by adesire to incorporate the observer into a “living” compositionmade by pixels with personality.

21

Nami

Nami is a display system comprised of untethered pixels thatwirelessly link to form an amorphous network1. The nameNami derives from the Japanese word for wave, describing theradial propagation of a message within the nodes of thenetwork, like ripples made by raindrops in a pond. Theoriginal impetus for Nami was Robert Poor’s Hyphos researchconcerning the self-organization of wireless nodes into anetwork of amorphous structure.2 Nami was the first, physicalplatform for demonstrating the Hyphos concept to a broaderaudience.

The design challenge for Nami centered on the most direct andintuitive way to illustrate decentralized networking. Anadvanced topic normally communicated through technicaldiagrams and graphical simulation, decentralized networkingcan be difficult to comprehend. With Nami, we endeavored tocommunicate the usefulness of a decentralized network towirelessly linking toys, appliances and other physical objectsin the home. This demonstration is best served in situ, wherethe viewer can directly experience the wireless objectscommunicating and watch how the system changes as thenodes are rearranged. Further, decentralized networking isspecifically liberated from central control, and simulation by asingle computer makes this point difficult to appreciate.Instead of building the physical network and illustrating itthrough screen-based simulation, it was decided that thesensible approach would be a decentralized network that wasits own display device. In this case, the physical pixels are thenetwork nodes themselves.

System Overview

Nami is comprised of an unspecified number of nodes that aredesigned to self-organize in a decentralized network. TheNami network node, or orb , is approximately the size, shapeand heft of a large bagel. The physical form of an orb waspurposefully chosen to make the object non-threatening andcomfortable to touch. The most prominent feature is a frostedhemispherical dome, lit from within by a glowing coloredlight that changes hue based on the state of the node. Whenthe dome is touched, an orb picks a new color and emits aninfrared pulse to signal nearby orbs to change their color andrelay the message. In this way, colored light spreads like awave throughout the entire network, creating a visible patternof the communication pathways in a decentralized system.

1 Nami was developed by Robert Poor, Kelly Heaton, Andrew Wheeler and Anita Stout (1999). The concept of an amorphousnetwork used in this discussion was described by: Coore, Daniel and Nagpal, Radhika, “Implementing Reaction Diffusion on anAmorphous Computer,” in: 1998 MIT Student Workshop on High Performance Computing in Science and Engineering,MIT/LCS/TR-737.2 Poor, R. Hyphos: A Self-Organizing, Wireless Network. MS thesis, MIT Media Laboratory, 1997.

The Nami Orb.

The Nami network. Photograph by Robert Poor.

22

Anatomy of the Nami Orb

Each orb contains five major subsystems: light generation, acontrolling processor, touch sensing, wireless communicationsand a power supply, as described in the following sections.

Light Generation

Beneath the plastic dome are eight high-intensity LEDs: threered, two green and three blue. Arranged in a ring, light fromthe LEDs is diffused by two layers of frosted mylar. APIC12C673 microcontroller generates three channels of PulseWidth Modulation (PWM) signals to control the brightness ofthe red, green and blue LEDs. Each channel is refreshed at arate of 112.5 Hz with eight bits of brightness resulting in 24bits of low-flicker color. Color commands are sent from thecontrolling processor to the PWM-dedicated processor over a4800 baud serial line, specifying the brightness for the RGBchannels and a rate to ramp from the current values to the finalvalues. Ramp rates can be as short as one PWM cycle (8.9mSec) or as long as 17 seconds. At each PWM cycle, theRGB brightness values are gamma corrected and converted tocorresponding PWM periods.

Controlling Processor

A PIC16C76 microcontroller is the central processor for anorb. It monitors touch sensing, sends commands to the lightgeneration PIC, receives and transmits infrared data to itsneighbors and controls battery recharging. The behavior ofeach orb is encoded in the controlling processor's program.This program directs touch response, message relay, messageexecution and color expression. Except for a unique ID, everyorb has identical software. Consequently, the emergentbehavior in a Nami network is the visible product of multipleinstances of the same program.

Touch Sensing

An orb has a capacitive touch sensing system that is triggeredwhen direct contact is made with the dome. A disk of mylarcoated with conductive Indium Tin Oxide (ITO) acts as oneplate of a capacitor while the ground plane of the orb's circuitboard serves as the other. By measuring the time to charge thecapacitor, the controlling processor can detect a change incapacitance.

Wireless Communication

A Nami network node is equipped with a bi-directionalinfrared (IR) communication link. A conical mirror at thecenter of each circuit board disperses IR light from threeemitters in a circular plane, parallel to the surface on whichthe orb is placed. Similarly, the conical mirror collects light

Diagram of the Nami Orb that shows the red,green and blue LEDs; touch sensitive dome(capacitive); two microprocessors (one masterand one LED-dedicated); rechargeable batteries;and infrared transmitter and receiver.

Illustration of the spread of a message within theNami network.

23

and directs it to an IR receiver on the circuit board. Using thisinfrastructure, an orb can communicate with its immediateneighbors without regard to the rotation of sender or receiver.Communication is 25 KBaud and intentionally limited toapproximately eight inches.

Power Supply

At full brightness, the LEDs for a single orb consume over awatt of power. An orb uses three AAA NiMH rechargeablebatteries and a DC power jack for recharging. A fully chargedorb will run for about five hours and can be recharged intwelve hours.

Visualizing the Nami Network

The amorphous, community-like structure of Nami isfundamental to the aesthetic of computer graphics in thesystem. Nami can reveal many insights that a Cartesiansystem cannot, and vice versa. Applications that requireabsolute spatial coordinates are not well suited to adecentralized network, making Nami inappropriate for thepixelated image to which we are accustomed on the CRT. Thedetermination of absolute network geometry requires manymore processors than are realistic for a physical pixel networkat the present time.3 It is only possible to know the relativelocation of a picture element in a decentralized network, usingthe following method. The main processor for an orb carries aunique ID that must be announced for the node to be knownby the other elements in the network. This announcement isrelayed through the standard channels of communicationenabling all network nodes to record the number of othernodes traversed by the message prior to reaching them. Thus,the coordinate system for a decentralized network is arelational map; a node will not know spatial coordinates, but itwill know the messaging distance between itself and othermembers of the network. Given the known radius ofcommunication for the Nami infrared system, it is possible togeneralize messaging distances into physical distances.Nevertheless, only directional systems of communication willprovide directional information and the Nami communicationinfrastructure is not designed for this purpose.

There are two general categories of visualization madepossible by the Nami network. On the one hand, Nami is adecentralized network made visible. Quite literally, Namiembodies the spirit of computer graphics: computation madevisible in a direct and effective manner. This is particularlycritical to the educational value of Nami because the actualstructure of the network is radically different from asimulation run by one central processor such as the desktopPC. Although we were not able to conduct a user survey, itseems intuitive that a concrete learning experience would be

3 Coore, Daniel, “Establishing a Coordinate System on an Amorphous Computer,” in: 1998 MIT Student Workshop on HighPerformance Computing in Science and Engineering, MIT/LCS/TR-737

A house illustrated with a Cartesian system.

The same house illustrated with an amorphouscoordinate system. Note that the recognizableimage of both illustrations can be created usingNami, but only with an interface device such asthe Digital Palette (described in the next section).Such images cannot be achieved using theinfrared communications network due to theabsence of absolute spatial coordinates for theorbs.

24

more immediately meaningful to a majority of people than asymbolic one. Once you have experienced Nami, it is mucheasier to make sense of the complex details of decentralizednetworking. At the very least, Nami provides an invaluableaccess to computation for young children or other people whocannot comprehend the computer through “input devices” orsymbolic abstraction. Nami is, in concrete terms, adecentralized system; that it happens to be visible is an addedbonus which requires no conceptual “leap of faith” toassociate with the network. All the more, missed messagesand ignored touch signals are a humbling reality of hardwarethat contributes a uniquely “real-world” flavor to the viewer’sexperience of an electronic system.

From a more abstract approach, Nami provides a platform onwhich content may be visualized. The content suitable forNami may be any behavior that occurs within a decentralizedsystem, such as the spread of gossip through a crowd, the flowof a chemical gradient, a gas lattice structure, or theoscillations of particles in a springy mesh. These areexamples in which the entire network is necessary to witnessthe emergent behavior. Individual nodes carry identicalsoftware, but behave differently according to user input andtheir location within the larger system. The result ofnumerous nodes executing the same instructions in anamorphous organization is beautiful: patterns of behavior thatare unique to the system and impossible to predict withcertainty. Through minor modification of the software, anentirely new experience will emerge out of the new rule set.For example, if the orbs are programmed to change colorimmediately upon receipt of an incoming message, thenetwork will behave much differently than if the orbs changecolor very slowly. In the case of a gradual color change, theviewer can rearrange the orbs or initiate a new message as theold one propagates, thereby creating a flood of color inmultiple directions. Robert Poor’s Tri-state touch programgives every orb three possible states: random color, oppositecolor, or blue-to-black. Each time an orb is touched, it willchange state based on the cycle index and relay a message thatcarries this index, a value between 0 - 2. When another orbreceives the message containing the index, it will update itsstate and relay the message, but it does not increment itsinternal cycle index. An orb must be touched sequentially tocycle through all of the Tri-touch states. As a result, all of theorbs in the network maintain some state in the cycle, but notnecessarily the same state at the same time. It is up to the userto determine the current state of an orb and to use thisknowledge to initiate the desired effect in the network.

Tri-state touch is one example of a program is adaptable to apuzzle or game, such as Othello™. Another application of asmart network node is character modeling with graphic outputas an indication of emotional state. A Nami orb can beprogrammed with preferences, such that it “wants” tocommunicate with some orbs, but not others. This effect isachieved by maintaining an internal counter that is

Children playing with Nami at the Tokyo ToyFair (1999). Photograph by Robert Poor.

One orb will relay a color message until theentire network shares the same hue.

The Nami version of Othello.

25

incremented by messages that have “good” IDs and decrementby “bad” ones. If an orb receives more good messages thanbad ones, it will remain lit with color; but if it does not receivemessages from the desired source, or receives messages frombad sources, it will eventually cease to glow with coloredlight. A simple set of rules such as these take advantage ofNami as a living community, wherein members of thecommunity react differently to each other. Suddenly, themeaning of Nami changes from a visualization ofdecentralized networking to an illustration of society withsocial individuals. By designing a physical form to reflect theidentity of unique characters, such as the form of a cow toenclose a bovine personality, a Nami orb could be transformedinto a doll with colorful emotional state, as suggested by theconceptual illustration for The Color Farm.

The Color Farm. Concept for a network inwhich farm animals use colored light torepresent variable character and emotionalstate. Each character contains the samebasic hardware, but their software isdesigned with unique preferences. Thedifferent software personalities arereflected in the different designs of thephysical casings (a pig, chicken and coware example characters). A children’s storywould come with the play environment tohelp children relate to color mapping andpreferences of the networked characters,including friends and enemies, hunger,sleep and other desires.

26

Digital Palette

The Digital Palette (1999) is a device for mixing colors oflight and recording temporal sequences of colored light.Following the completion of Nami, Kelly Heaton and StevenGray collaborated to develop an instrument for easily andintuitively controlling the color of an LED physical pixelwithout necessarily writing code or mapping color from amonitor. As described by William Finzer and Laura Gould,this method of “programming by demonstration” is motivatedby a belief that,

[I]t should be possible to place the control of interactivecomputer graphics in the hands of … designers, those with anunderstanding of the power of such systems but notnecessarily with the ability or willingness to write the complexprograms that are necessary to control the systems.1

1 Finzer, William F. and Gould, Laura, “Rehersal World: Programming by Rehersal.” From: Cypher, Allen (Editor);co-edited by Daniel C. Halbert … [et al.]. Watch What I Do: programming by demonstration. The MIT Press,Cambridge, MA, 1994.

Knob for shiftingthe color ofpalette pixels

Indicates thecurrentlyselectedpalette pixel

Switch togglesbetween “paint”and “mix”modes

Hidden from view:The “learning”button is locatedbeneath the palette

Palette pixel (LED “pot ofpaint” for mixing colors andcreating chromanimation)

Anatomy of the Digital Palette

27

Our answer to this problem is a hand-held, infrared remotecontrol device that contains five physical pixels, made ofmixing red, green and blue light-emitting diodes, which can beused to program the color or color animation for otherphysical pixels. The color of the palette pixels can be adjustedusing a knob that maps to a value on a wheel of 256 discretehues. An artist selects between the five palette pixels usingdirect touch, just like a Nami orb. The quantity five waschosen to give the artist the ability to select and store multiplecolors, analogous to a paint palette.

To create color animation (hereafter referred to aschromanimation), the Digital Palette requires two modes ofoperation, “mix” and “paint”, which may be accessed using aflip switch. In the paint mode, the color or chromanimationseen in the touch-selected palette pixel is the data that will betransmitted to by the Digital Palette via infrared. The colorcommands encoded in this transmission can be received andexecuted by another infrared enabled physical pixel, such as aNami orb. It is possible to change the hue of a palette pixel inthe paint mode by using the knob, and the hue of temporalcolor sequences may also be shifted. A palette pixel may onlybe programmed with new chromanimation when the DigitalPalette is in the mix mode.

In the mix mode, all of the palette pixels at first appear assolid colors. These colors may be changed with the knob, justlike paint mode, but no data is transmitted to the outside worldwhile in mix mode. The purpose of the palette pixels in mixmode is to provide a source of color for synthesizing a newchromanimation. To create a chromanimation, the artist firstsets the colors of all the palette pixels to the colors they wishto use in their sequence. Next, the artist touches the palettepixel they wish to program. Once ready to create thechromanimation, the artist holds down the “learning button,”located on the handle of the palette, and proceeds to touch upto five colors in a temporal sequence that the palette records.When the learning button is released, the recording process iscomplete and the palette pixel will begin to animate with thelooping chromanimation, the duration of which is determinedby the total length of time the learning button was activated.For example, if the learning button is held while red, greenand yellow are touched for one, three and one secondsrespectively, the palette pixel will display: one second red,three seconds green and one second yellow with a loopduration of five seconds. Similarly, the delay betweenchanging colors in the chromanimation is determined by therate at which the colors were touch selected during therecording process. The Digital Palette is programmed to fadebetween selected colors according to the length of the delay,such that long delays give a gentle transition and short delaysproduce a blinking effect.

If the chromanimation is successful, the artist can switch backto the paint mode and the recently programmed palette pixelwill animate with the new sequence. All of the other palette

How to use the Digital Palette. Note thatthe introduction of color into anotherphysical pixel requires direct selection ofthe object.

2: Touch thetarget physicalpixel to introducethe new color orchromanimation.

1: Pick a color or createa chromanimation. Thepalette will emit thisdata via infrared.

28

pixels will return to their previous paint-mode color orchromanimation. If the artist is unsatisfied with achromanimation, they can remain in mix mode and reprograma new sequence, or else chose a static color and return it to theselected palette pixel in paint mode. Whatever information iscontained in the touch selected palette pixel in mix mode willbe returned with the pixel when the artist switches back topaint mode.

The Digital Palette aspires to be a simple device for assigningcolor or creating color animation for a physical pixel. In itscurrent form, the Palette cannot be used to design animationthat changes across multiple pixels or responds to rules, suchas an interactive graphics algorithm. Instead, it acts like asource of electronic paint that is applicable one physical pixelat a time. For example, if an artist chooses the color red withthe Palette, they must then touch a physical pixel to apply thatcolor to the object. The artist may continue to touch otherphysical pixels and color them red as well, or they may selecta different color. The important link between the DigitalPalette and a physical pixel is touch; although the data istransmitted from the Palette continuously, a physical pixel willonly respond to the information if it is touched. Thispreserves the direct physical relationship that Hiroshi Ishii (etal.) calls a “tangible user interface.”2 There are no commandlines or text menus associated with the Digital Palette, only thedirect response to color that is familiar to an artist from priormedia.

Our use of the painterly metaphor was intended to ease theartist into use of an unfamiliar device, although the additionalfunction of animation may or may not be intelligible from ourcurrent design, particularly in the context of a painter’spalette.3 The Digital Palette’s association with painting alsothreatens to impede a novel interpretation of the physical pixelas a tool or display, given that a computer and painting arecompletely different media. On one hand, the mixing anddirect application of colors to a physical pixel seemsanalogous to painting with real pigment; but unlike physicalpaint, colored light is not permanent and not necessarily static.The five physical pixels of the Digital Palette can become anycolor or chromanimation. Anecdotal experience indicated thatthese properties could be confusing in the context of apainter’s palette, especially without text labels or otherinstructions on the tool itself. Nevertheless, there is somethingfundamental about the artistic experience of mixing andapplying color to objects. Continued work with physicalpixels will hopefully reveal what aspects of the painterlyexperience are relevant and which are mere metaphors.

2 Ishii, H. and Ullmer, B, “Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms,” in: Proc. ofCHI97 , pp. 234-241.3 This observation is based on conversation with numerous visitors to the MIT Media Laboratory, notably RachaelStrickland and Bill Verplank.

The Digital Palette with Nami

29

Peano

Peano is a building set comprised of one-inch cubes thatconnect to form a modular, full-color display in three-dimensions. The original inspiration for Peano was a smartmolecule kit: a set of elements common to organic chemistrythat could be used to model and explore molecular structure.With colored light as a visual indicator, the modeling systemwas intended to provide feedback regarding formal charge orother properties of interest to the student of chemistry. Achemistry set designed as a system of physical pixels couldrepresent the electron distribution across the molecular modelas a continuous, time-varying effect that responds to theaddition or subtraction of atoms in real-time. Simulationsexist for display on a computer monitor, but not as ananimated physical model.

Although technically feasible, the search for appropriateconnectors proved discouraging. A fruitless quest for small,reasonably priced, yet durable connectors with four or moreelectrical conductors and radial symmetry motivated an effortto build the simplest possible geometry with a modular, three-dimensional structure.1 Further, the use of many connectorsper physical pixel threatened to obscure the desired visualeffect. Thus, the decision to restrict the number ofconnections eliminated the molecule concept, whereinnumerous bonds per element would be required for accuratemodeling.

Several others [Frazer 1978, 1990], [Anagnostou et al 1989],[Borovoy, Kramer 1997], [Kramer, Minar 1997], [Gorbet et al1998], [Anderson 1999], [McNerney, 1999] have developedcomputational building kits, some of which reveal simple andclever solutions to the problem of network connectivity.Frazer’s Universal Constructor (1990) and Kramer andMinar’s Stackables (1997) are the closest precedents, eachwith only two connectors per element and full knowledge ofnetwork status and structure. However, neither provides full-spectrum color animation and both systems are limited tostacking objects in one direction only. Frustrated by the“mathy” feel of most computational building systems, not tomention the flat interface of graphics on a monitor, Iendeavored to create an organic, sculptural material forinteractive computer animation.2

I soon discovered that organic forms are not easily translatedinto the inflexible structure of computer electronics. Basicrequirements, such as power and data communications,continually precipitated engineering solutions that were too

1 For the molecule application, I considered the use of potentiometers at each connection. A variable change in resistance couldbe used to calculate basic information about molecular structure, such as “cis” and “trans” conformations. For more informationon connectors, please refer to the “The Design of a Modular Display System” in the Appendix.2 At first, I struggled to design an actuated sculptural system. After investigating pneumatics, hydraulics, heat-actuated gels andshape memory alloys, I decided that such a system is best accomplished with a biological model – an avenue for future research.

“Electrostatic Surface of Aspirin,” by Roger Sayle withRasMol (Glaxo Welcome, 1995).From: http://www.umass.edu/microbio/rasmol/sayle3.htm.

Early stages in the Peano design process.

30

big, too rigid, too impractical or otherwise impossible.3 Thephysical design challenge persisted through several iterationsuntil a satisfactory solution was finally achieved. Thesuccessful geometry was inspired by a string of beads, coiledin a stack to create a three-dimensional form with one-dimensional network structure. This realization led to thenotion of a close-packing curve, or Peano Curve, named afterthe Italian Mathematician Giuseppe Peano (1858-1932). Inhonor of Giuseppe, “Peano” was chosen as the name for oursystem. A Peano Curve, sometimes referred to as a HilbertCurve, is a linear structure that turns at 90-degree angles todefine a three-dimensional space with Cartesian coordinates.The usefulness of the Peano geometry derives from thesimplicity it affords a modular network. Because the Peanotopology is linear, a one, two or three-dimensional geometrymay be built from a set of elements that have only twoconnectors each. Due to the strict rules of the Peano shapegrammar, a central processor can easily infer the structure ofthe display. Like Frazer’s Universal Constructor, Peano’snetwork is one dimensional, but the building elements canrotate in space to define a complex geometry.

Despite the numerous advantages of the Peano design, it isundeniably “mathy.” To compensate for this character, Ifocused on development of smooth color animation for therigid structure. Where the physical form of Peano is cubic andstrictly rule-based, the computer animation of colored lightcan be programmed to give an organic illusion. The gentleanimation of color in Nami, as well as Golan Levin’s drawingprogram Aurora (1999), provided valuable inspiration for theinteractive animation I sought to achieve in Peano. Inparticular, Levin’s screen-based Aurora invites the interactorto draw with a haze of multiple colors that blend with a softand mutable quality. Peano is similarly designed to permitdirect and programmatic control over the animation of coloredlight in sculpture, albeit formally rigid in the present design.

System Overview

In the fall of 1999, Kelly Heaton, Steven Grey, Paul Pham andAlex Jacobs began work on Peano as described above. Peanois a three-dimensional, modular display system comprised of128 or fewer one-inch cubes made from a colorless, diffusiveplastic material. In the center of every Peano cube(abbreviated hereafter as “cube”) there is a single,multicolored LED that illuminates the entire volume toproduce a full-spectrum of colored light. Each cube containssix major subsystems: connectors, light generation, controlprocessor, touch sensing, data communications and powerregulation. These systems are described in the followingsections.

3 These design constraints are discussed in greater detail in “The Design of a Modular Display System” in the Appendix.

The successful Peano design (1999).

The Peano Curve. Image source:http://www.math.ohio-state.edu/~fiedorow/math655/Peano.html

Golan Levin’s Aurora (1999), an organic complimentto the rigid structure of the Peano Curve.

31

Connectivity

Every Peano cube has two connectors, one male and onefemale, that occupy two of the six faces. The male and femaleconnectors are positioned on the cube in one of two ways,forming two different shape grammars: linear and orthogonal.A linear configuration has connectors that are positioned onopposite sides of the cube; and an orthogonal configurationhas connectors that are positioned on adjacent sides of thecube. The connectors are designed such that the cubes can jointogether in one of four orientations, enabling the constructionof a linear geometry with three-dimensional topology. Themale connector has a total of eight pins: six of the eight pinsform the electrical continuity for the network, including twoeach for power, ground and data; and the remaining two pinsare used for peer-to-peer communications, rotation and bus-ready. The rotation pin is keyed to pull one female pad of thedownstream cube to ground, insuring that every cube knowsthe rotation of its upstream neighbor. The bus-ready pin is aflag used by a cube to signal an upstream neighbor to placedata on the bus. This way, data returns to the CPU in linearsequence and the network topology can be inferred.

Light Generation

In the center of a Peano cube is a single multicolored LED thatilluminates the entire volume. The inside cavity of the clearacrylic cube is frosted to evenly diffuse the light across thesurface.4 Since light is not visible through the two sides withconnectors, these interior surfaces are coated with a flat whitepaint to reflect light out of the cube. The LED of theorthogonal cube is purposefully bent at a right angle to faceinto the painted circuit boards, such that the light will bereflected evenly.5 Like Nami, the red, green and blue channelsare controlled by Pulse Width Modulation (PWM). Thecontrolling processor uses the same look-up table used by aNami orb for gamma correction.

Control Processor

Each Peano cube contains a PIC16F876 flash memorymicrocontroller that handles all of the internal processing. Thefunction of the microprocessor is to perform all of the localoperations that could not be accomplished by the master CPUfor a modular network of physical pixels. It monitors touchsensing, receives and transmits data on the bus, handles lightgeneration, detects rotation and initiates upstream datatransfer. The microprocessor is a slave to the CPU,controlling the bus only when queried. If the bus is idle formore than 100 ms, control will revert to the CPU to preventdeadlock in the event that a user disconnects a cube that wascontrolling the bus. Since the controlling processor is

4 Light diffusion is a tricky art. For more on this topic, please refer to “The Design of a Modular Display” in the Appendix.5 This happens naturally in the linear cubes; but were the LED of the orthogonal cubes not bent, the majority of light would travelout of one of the translucent faces.

The two member shape grammar of Peano .

The male and female connectors of a Peano cube.The smaller pins / pads in the middle are electricconductors. The larger circles in the outercorners represent the mechanical connectors (thefemale connector has magnets and the male hasferrous slats). Note that the male pins willconnect to the female pads in four orientations,and power and ground are given two pins each tosupport a higher current rating for the system.Illustration by Alex Jacobs.

Peano builds a three-dimensional structure withlinear network geometry.

32

electronically erasable, a Peano cube may be easilyreprogrammed (in-circuit) with different firmware toprototype new behaviors for the cube.

Touch Sensing

Touch events are detected using the same RC circuit as Nami.A thin wire surrounding the edge of the male and femaleconnectors of each block forms the capacitor for the circuit.The wire is slightly recessed to avoid erroneous touch-selection, but easily accessible from every side of the cube.6

Data Communications

Communication between the host CPU and up to 128 Peanocubes occurs over an RS485 bus operating at 57,600bps.There are 3 commands recognized by a cube: get, set and sync.The get command is used to retrieve a cube's unique ID, type(linear or orthogonal), touch status and neighbor rotation. Theset command designates the new color value for a cube, but isnot executed until the sync command is received. The synccommand avoids the ripple effect of asynchronous colorchange in the display network. The networked cubes can beaddressed in three different ways: individually, in a specifiedrange, or all cubes following a specified node in the network.As mentioned previously, a cube will put data onto the busonly when commanded by the CPU and flagged by itsdownstream neighbor. This protocol insures that thetopological information for the network is correct.

Power Regulation

Power and ground lines for the Peano network are rated for 12Volts and 6 Amps, or the maximum power required by a 128physical pixel network at full brightness (white). For safetyreasons, the system is designed so that only an exposed femaleconnector can be “live.” Internal to each block is a regulatorthat supplies 5 Volts to the circuit. The circuit is alsoprotected against grounding due to wrong connections.

Control of the Peano Network

The central processor for our Peano network is a Pentium III600 MHz processor that is connected to the first cube of thedisplay by an RS232 serial cable. A custom Java applicationwas developed by Paul Pham to manage two major tasks:operate the Peano display and provide a software interface tothe system.

6 I intended to coat the exterior of the Peano cubes with Indium Tin Oxide (ITO), but the expense proved prohibitive for aprototype. For the record, I am told by manufacturers that it is possible to use ion-assisted deposition to apply a thin (severalmicron) film of this conductive material to a volume of plastic. Such material exists in flat sheets, as used for Nami and sometouch sensitive screens. I am not aware of a commercial product with ITO on the surface of a volume, despite the usefulness oftouch detection in many consumer applications.

Nine Peano cubes arranged in two of manypossible combinations of geometry and color.

33

Display Operation

While the display is operational, the CPU must update thecolor information for every cube at a rate of 15 Hz and querythe network status at a rate of 3 Hz. A network query returnsthe following information for each cube: unique ID, linearorder, type, rotation and touch status. The Cartesiancoordinates of the display are calculated from the linear order,type and rotational information for all of the cubes in anetwork. If necessary, the CPU will calculate and transmitnew color values to the display followed by a sync command.

Software Interface7

Information about the physical system is accessible to the userthrough a custom software application that is viewable on acomputer monitor. The program has two options forcontrolling the display network: a graphical user interface andan interpreted language environment. Both of these programstake input from the Peano cubes, such as touch andreconfiguration, as well as traditional desktop input devices,such as the mouse and keyboard. The graphical user interfaceconsists of a 3D simulation of the Peano geometry, and severalcontrols: render and view options for the simulation, a color

7 For more information on the Peano Software Interface, including the Peano API, please refer to:http://www.media.mit.edu/~kelly/physPix/peano.htm

Annotated illustration of the Peanosoftware interface. Design by Paul Phamand Kelly Heaton (1999 – 2000).

Concept for a Peano sculpture. Illustration Alex Jacobs.

34

manipulation program, and information access to the cubes.The simulation was created to illustrate the display geometryand state, thereby aiding in the development of controlalgorithms for software applications, as well as providing ascreen interface to coloring the cubes. ColorAnime is agesture-based drawing tool that was developed as part of thesoftware interface to facilitate color animation of the cubes.8

ColorAnime consists of a color palette, which can be anydigital image represented on the screen, and an internal recordof gesture sequences made within the digital image palette.The duration of a gesture, made by mouse movement or otherdrawing instrument for the PC, defines a temporal sequence ofcolor that is looped to form a chromatic animation, orchromanimation . The chromanimation is applicable to asingle cube, which will then loop through the color sequenceindefinitely, or it can be used like a brush across multiplecubes. When “painting” more than one cube, the userdesignates a pathway across blocks in the display and thenapplies the color sequence to this pathway. The pathway forthe brush may be indicated using the screen interface, or bytouching a sequence of cubes in the physical system. In bothcases, the width of the brush must be selected using the screeninterface. The color animation is then antialiased along thepathway to provide a smooth aesthetic, analogous toantialiased graphics for the video monitor9.

The Peano interpreted language environment, a specializedsubset of Java developed by Paul Pham, was inspired by JohnMaeda’s Design By Numbers. The language is intended toprovide a familiar access to a unique display of computation,such that software applications could be developed withrelative ease as compared to Nami, which requiresreprogramming every node. Our objectives were twofold:visualize Cartesian mathematics in a three-dimensional,modular display; and prototype software behaviors that couldlater become embedded in the cubes themselves. The eventual

8 ColorAnime was developed by Kelly Heaton, Scott Snibbe and Paul Pham (1999-2000).9 The antialiasing algorithm was adapted by Paul Pham from the Gupta-Sproull method, described in: Foley, James D. [et al.]Computer Graphics: principles and practice (Second Edition in C). Addision-Wesley Publishing Company, Inc., Reading, MA,1997.

The ColorAnime method of sampling gestureacross an image to form a temporal sequence ofcolor, or chromanimation .

Using ColorAnime, a gesture drawnthrough any digital image (exampleHSB space shown in center) translatesto color across the Peano cubes. Thecolors will animate sequentially at arate determined by the speed of thegesture. Illustration by Paul Pham.

35

goal for Peano is a free-standing system without the use of ascreen interface, which although familiar, distracts from theinsights of a new display format.

Visualizing the Peano Network

Like the computer monitor, Peano is a Cartesian systemcontrolled by one processor to make a colored-light display ofcomputation. The similarities stop here. Whereas the videodisplay is a fixed image plane, Peano is a flexible quantity ofmodular pixels that connect to form a network with anexponential variety of spatial configurations. The individualelements participate in a common network controlled by acentral processor, but each physical pixel is programmable tofunction semi-autonomously. We take advantage of themicroprocessor in each cube to generate full-spectrum light,detect touch, respond to a unique ID, and return informationregarding the display geometry. Although the overallbehavior for the network is centralized, our control algorithmsare subject to feedback from the physical structure, includingquantity of pixels, linear sequence and orientation.

Many of these parameters are normally associated withpuzzles, such as a Rubik’s Cube™. Peano shares certaincharacteristics with spatial toys, but it can be augmented withsoftware behavior to unite physical and virtual play into asingle device. Imagine, for example, if Tetris™ weremodified to function in a modular, three-dimensional space, orSimon™ could be rearranged into shapes other than a colorfulpie. Computer animation in Peano can be designed to react tomodifications in the physical structure. Peano may be used toteach spatial geometry, color relationships, quantity,sequencing and other properties learned through visualmathematics. Whereas painted blocks have been used to teachconstruction for centuries, Peano introduces concreterelationships combined with the logic and ephemeralabstraction of computation.

Whether perceived as a toy or an art form, Peano is asculptural medium designed for creative exploration. Physicalstructures can be built by hand and then “painted” withcolored light using a tool such as ColorAnime or the DigitalPalette. Colored light may remain static and specific to eachcube, or the artist can write software for dynamic graphics.For example, an antialiased sphere of light can beprogrammed to move slowly about the structure in response totouch. Skillful animation suggests an organic form internal to,or passing through, the rigid physical geometry of Peano.Visual effects, made by possible by software control of theLEDs, are analogous to the illusion of computer animation ona monitor (the graphics are rich and deep, whereas the screenis rigid and flat). Peano adds several degrees of freedom to thephysical interface, including modularity, direct touch responseand three spatial dimensions. Like Nami, the tactileengagement and direct feedback from a physical object drawthe artist into an intimate relationship with the display system.

Peano can be enjoyed from any point of view.

The monitor limits the observer to asingle viewing angle.

36

This relationship is further enhanced by the absence of a strictpoint of view onto the visual experience. A sculptural systemfor computer animation, Peano can be rotated in real space,placed in context of other objects and generally appreciated aspart of our physical environment.

37

20/20 Refurbished

20/20 Refurbished is a proposed art installation, designed toincorporate the observer into a narcissistic relationship with areactive composition. The piece is not intended to reflect theviewer according to straightforward expectations, like a literalmirror or real-time video image. More than a simple digitalmirror, the pixels of 20/20 Refurbished will look and talk backto create a circular dialogue between the observer and theobserved. The resolution of pixels is sufficient that a viewerwho approaches the piece will witness the impact of theirpresence and become drawn into a mutable composition ofsight and sound. If the artwork is left undisturbed, the pixelswill close their eyes and say nothing, save for the occasionaltrigger by doppelgänger.

The physical pixels of 20/20 Refurbished are made from the1999 Limited Edition Christmas Furby™, modified to revealonly the eyes and mouth in a softened triangular shape.Through the phenomenon R. L. Rutksy describes as “the birthor coming to life of the machine,” a pixel made of Furby™ is“infused with a living spirit, with a soul; it is a “dead”technological object reanimated, given the status of anautonomous subject.”1 Usually perceived as a “fuzzy,computerized, mechanized doll that talks, blinks, sleeps andasks to be fed,” Furby™ is part of a new generation ofconsumer products that blur the distinction between real andartificial life.2 Unlike the dolls of yesteryear, enlivened onlyby a child’s imagination, Furby™ is animated by aprogrammatic personality, inviting a curious dialogue betweena real and simulated being. These toys, designed to substitutebiological Fido with a battery-operated pet, are a fascinatingbut disturbing step towards the widespread acceptance ofartificial life.3 As part of 20/20 Refurbished, the culturalimplications of a disembodied Furby™ contribute a complexpsychology to the pixels that spans the superficially playful tothe profoundly perverse.

Each pixel will be actuated by a motor to achieve two visiblestates, open or closed. The pixels will be augmented with aPIC16F876 microcontroller that prompts the Furby™ to speakin response to viewer proximity, as determined by an infrareddetection system. Although the Furby™ pixels retain theiroriginal logic, the added microcontroller will be programmedto control their behavior in an unorthodox manner. WhereasFurby™ normally has sensors that respond to “real”interaction by a human or another Furby™, such as tummytickling, back rubbing and infrared communication, the addedmicroprocessor will cause the Furby™ pixel “think” these

1 Rutsky , R. L. High Techn�: art and technology from the machine aesthetic to the posthuman. University of Minnesota Press,Minneapolis, MN, 1999. Page 24.2 Boston Globe (February 24, 1999)3 Sherry Turkle’s research with children and Furby™ substantiates this claim. From: Turkle, Sherry. The Second Self:computers and the human spirit. Simon and Schuster, New York NY, 1984.

The two visible states of the Furby™ pixel

When no observer is detected, a Furby™ pixel willclose its eyes and remain quietly watching with aninfrared detector (shown here as black dots).

When an observer is detected, a Furby™ pixel willopen its eyes and begin speaking.

38

inputs are occurring even though the inputs to the pixel arefully simulated, or “virtual.” All of the normal sensors of theFurby™ will be disconnected and replaced by our customsoftware.

When the infrared emitted by each pixel is bounced off thepresence of a viewer, an invisible relationship will be mademanifest through the response of the artwork. Four hundred ofthese identical pixels will be arranged in a loose interpretationof the Cartesian grid, suggesting an organic departure from thepixelated rationalism that is characteristic of most computergraphics. An observer located within sensing range of 20/20Refurbished will experience hundreds of disembodiedcreatures looking and talking back at them in the form of theirsilhouette. The observer is thereby reproduced in Furby™,itself a unit of reproducible identity.

Like a mirror, 20/20 Refurbished requires an observer to makethe surface come to life. Like a work of art, the ownership ofthe reflected identity will be ambiguous. The disembodiedcreatures need the presence of a body to come to life. Theobserver needs the disembodied creatures to visualize theirdigital semblance. The observer’s act of observing the art isnarcissistic by default of their reflection in the composition,yet the art needs the viewer to be complete. Does the audiencemake the art; is the art making the audience; or is the art theaudience?

The observer will be invited to participate in these multiplelayers of meaning made by pixels that are artificially sentientand full of personality.

Diagram of the Furby™ pixel showing theinfrared emitter, receiver and the speaker (hiddenfrom view behind the front panel of the display).

Each pixel uses an infrared detection circuit toprobe for an observer in front on the display.

The 1999 Limited Edition Christmas Furby™ prior torefurbishing. Already, the toy is a unit of mechanical

reproduction and pop culture iconography, qualities that weencapsulate in the form of a pixel for viewer reflection.

39

Discussion

The physical pixel is a synthesis of old and new forms of art.Once limited to an artifact of material manipulation, the markof an artist is now defined by a relationship between atomsand bits. Like the signature stroke of a painter, the physicalityof a pixel can still express a unique style; only now, that styleis augmented by information design. This is a property of thephysical pixel that I came to understand gradually, learningalong the route to a body of work for which the designdecisions were not always well defined.

One of our objectives for Nami was to create a playfulinterface to a decentralized network. Since no additionalmeaning was assigned to the orbs at the onset of the project,the physical form was made to be general and abstract. Thestyle was guided more according to the efficacy of userinterface than an artistic vision for a new sculptural media.1

The Peano cube is similarly abstract because we wanted amedium from which other art could be made, like a sculpturalversion of the pixels of a video monitor. Had we developedthe molecule kit, the meaning of the physical pixels wouldhave been specific to chemistry; and consequently, muchdifferent than the general Peano cube.

The decision to create abstract physical forms for these pixelsenabled us to think of them in many different ways, and alsoto focus on a variety of graphical behaviors. But I discoveredthat general platforms are not especially useful for art unlessthe platform is useable as a creative medium. To be useable, amedium needs to be malleable. If everything made from themedium looks like a rearranged version of the medium, then itis not particularly useable. This may be the greatest failure ofNami and Peano: neither system can be made to look likeanything other than itself, and yet they are both designed forgeneral use.2 There is no clear meaning behind these systemsof physical pixels, yet software content cannot “reinvent”them with the level of sophistication which video pixels canbe graphically reinvented.

In contrast, a Furby™ pixel has a character with unalienablecontent. Although people will perceive the meaning of aFurby™ pixel in different ways, it is definitely a unit ofcontent as much or more than it is a unit of pictorialrepresentation. The meaning of a Furby™, both as a toy and

1 Only in hindsight did I come to perceive Nami as a physical pixel; and now, I would approach the system with a muchclearer view of the relationship between physical form and software behaviors.2 I am speaking specifically about their usefulness as an artistic medium. I do think both systems have clear value foreducation and toys, independent of their artistic value. Nami has been an excellent tool for demonstrating decentralizednetworking, but we have not developed any particularly successful games or sculptural applications to date. This ispartly a consequence of time, partly the difficulty of reprogramming the nodes, and partly my own uncertainty aboutthe expressive potential of Nami as a sculptural and graphical medium. Peano was completed so close to the deadlineof this thesis document that I have not had sufficient time to develop applications; and cannot, therefore, assess withcertainty the usefulness of the display.

The physical design of Nami (above, orbsshown on a prototype recharging station)and Peano (below). Both forms werepurposefully designed to be abstract.

Contrast the abstract form of Nami andPeano with the highly content-driven

design of the Furby™ pixel.

40

as a pixel, has been specifically designed and is in no wayintended as a general platform.

I believe that both approaches to the physical pixel are usefuland necessary in the quest for new art. 20/20 Refurbished islikely to have a more immediate impact; but it will not offerinsight into computational media that would be useable by ageneral audience. 20/20 Refurbished is intended ascommentary on the contemporary reflection, made by acombination of software, sculpture and pop culture. As such,the physical and software design of the Furby™ pixels isspecific to the installation, much in the same way thatcompositional elements of a painting are essential to thatcontext, but nowhere else. A kit for the construction of smartmolecules would be similarly limited to a specific application,albeit scientific visualization instead of artistic commentary.

Unlike 20/20 Refurbished, Nami and Peano are systems ofphysical pixels that strive to be more generally useful forcreating sculptural computer graphics. Due to the significantchallenges to engineering a new display technology, a trulyuniversal platform for sculptural computation will requiremore time and effort. Keeping the big picture in mind, anyincremental steps are valuable as a learning experience.

A Conceptual Framework for theDesign of Physical Pixel Media

There are two primary classes ofphysical pixel: specific (Furby™ pixel,Digital Palette) and general (Peano,Nami). Specific pixels have a clear andsingular purpose, in this case describedas the content of a work of art. Thegeneral pixel is a medium with whichart can be made. The general pixel hasno intrinsic content because it serves torepresent a diversity of content. Thevideo pixel is currently the most widelyappreciated form of general physicalpixel. The specific pixel is acontemporary form of the artistic mark,encapsulating a unique style of physicaland digital expression.

The two sides of the diagram reveal theprinciple differences in the designprocess (hierarchy reads from top tobottom, ending with the unit ofrepresentation).

41

The lessons of Nami and Peano are several. Most importantly,a general-purpose creative medium needs to be either highresolution or highly malleable. For example, LEGO™ bricksare not malleable whatsoever, but if you have enough of themyou can build something interesting. By contrast, a singlepiece of rope is only one object, but it can be twisted andlooped to form many interesting knots. A pixel is much morelike a LEGO™ brick than a rope, given that it participates in asystem of discrete units (whereas the rope might describe avector display). This comparison seems to indicate that manyphysical pixel elements are the key to the success of asculptural system for computer graphics. It is not to suggestthat a physical pixel couldn’t deform shape; but suffice it tosay that a high resolution of elements appears to be the magicingredient of pixelated media.

In our prototyping mode of operation, we were never able tobuild more than fifty objects for either network. The effort toconstruct many pixels is a deterrent to high-resolutionsystems, absent a clever technique for mechanizing themanufacturing process. And not only mechanization of thephysical construction, but also the electrical infrastructure andpower distribution for all of the discrete elements.

The Digital Palette differs from the other three projects in thatit is an instrument for working with other types of physicalpixels. In a sense, the Palette is a tool that needs a medium. Itcan be used at the present time to change the color orchromanimation of a Peano cube or Nami orb; yet it seemsthat the Palette can and will be more interesting as futuresystems of physical pixels are developed. Likewise, the extentto which it remains a “palette” may become less emphasized,as the relevance of our painterly metaphor is replaced by abetter understanding of a new medium.

There are two major reasons why future work with physicalpixels is important. First, alternate visualizations ofcomputation, particularly those that explore other spatialdimensions, are critical to learning about the computer as amedium. Whether it is science or art that drives theinnovation, the end result benefits the community at large.The second reason concerns the importance of individuality inartistic expression. There is nothing inherently wrong aboutworking with a prefabricated display device to make art; but tothe extent that an artist avoids the development of new mediain favor of pre-existing ones, opportunities for novel visions ofcomputation are compromised. Despite low-resolution results,difficult engineering and the high risk of failure, the effort toachieve new insights into computation is deeply meaningful inits own right. Often, it is through the toughest struggle that atruly unique art will emerge.

42

Conclusion

The physical pixels presented in this thesis are but a fewexamples of computation made visible. Both the materialform and the computer infrastructure determine our perceptionof a pixel, suggesting that there exists at least as manyvariations on the physical pixel as there are materials andcomputers combined. It is not useful to speculate on everypossible appearance, but instead to open our minds to thepossibility of different manifestations insofar as these newforms will change the way we perceive the computer as anexpressive medium. Variations on the physical pixel mayenhance the modularity of the display, make socialcommentary, provide tangibility, or lend a signature aestheticto a work of art. Imagination and ingenuity can conceive ofmany instances of the pixel beyond this body of research.

To be sure, there are plenty of cases wherein the videomonitor is the ideal medium for illustrating pixel graphics.Likewise, there are visualizations that cannot be revealed withthe pixel data structure whatsoever. Just as the decisionsregarding a physical pixel deserve careful consideration, otherchoices should be thoughtfully addressed by any artist whouses the computer as their medium. Constraint is often themother of invention, but perceived limitations must not betaken for granted. Both the pixel (data structure) and thephysical pixel (visible form) are critical factors in ourunderstanding of art made with the computer. Our choicesmake the art.

When choosing to work with any form of physical pixel, themeaning associated with the invisible pixel is inextricablefrom our perception. There are various cultural and historicalfactors that contribute to the meaning of the pixel, includingCartesian rationalism, scientific deconstruction and a historyof optical analysis in the visual arts. Any work with physicalpixels, of any material form, engages in a dialogue with thisconceptual and perceptual foundation. For example, thegentle flow of colored light within Nami or Peano is an effortto counterbalance the physical fragmentation inherent to apixelated display system. Such fragmentation is fundamentalto the pixel and therefore any art made visible by physicalpixels. Discrete representation is not only aestheticallydistinctive, but suggests a philosophy of perception.

The look of computation is open for artistic interpretation.Like poetic thought, pixels and other data structures have noconcrete appearance. The computer provides a uniqueopportunity to reveal invisible layers of meaning in multipledimensions and with a rich variety of physical media.

Poetry made visible by William Blake inFiery Pegasus, 1809. Pen, ink and watercolor,approximately 9 x 6.5 inches.The British Museum, London.

43

44

Appendix: The Design of a Modular Display System

The design of a modular display is an exercise in constraint. As with any system, a change in one designparameter affects all others, such that a resultant system is the product of compromise. Nami, Peano and20/20 Refurbished are all characterized by design decisions that contribute to certain capabilities, buteliminate others. The purpose of this appendix is to generalize, insofar as possible, the recurring challengesto our development process.

Design Concept

A clear design vision is central to the development of a successful display device. Although this statementseems self-evident, there are numerous occasions in which engineering will present an alternate path; andoften these detours will veer away from the original objective. Strong goals demand the cleverestsolutions. The following questions should be continually revisited throughout the development process:

- What will be visualized?- What are the quantitative and qualitative properties required by the display to achieve the

desired visualization?- What is the power budget?- Will the device be safe for an audience?- Is the system innovative?

Power

At the present time, electronic devices require the distribution of power and ground. If the pixels of adisplay are to be wireless, power and ground must somehow reach every picture element in the system.Batteries internal to each object are the easiest solution to this problem, although not ideal for displaydevices that have many pixels, or picture elements that are very small.

Direct contact between display elements and a power source is one alternative. The obvious disadvantageto direct contact is the spatial freedom lost by tethered picture elements. In cases where the physical pixelsconnect to form an electrical bus, care must be taken to insure that each individual element can handle themaximum voltage and current draw of the entire system. A hybrid of rechargeable battery power andexternal supply benefits from both approaches, but significantly increases the complexity (and expense) ofthe system.

Network Architecture

Data communication within a system of physical pixels shares some common problems with powerdistribution; namely, all elements in the system must be able to receive, and in some cases, send, data.Generally speaking, there are three approaches to network architecture of a display: centralized, partiallydistributed and decentralized. A centralized network involves the standard, I/O display device paradigmwherein a CPU performs all computation and controls the pixel actuation. Here, the pixels do not have anyprocessing power. An example physical pixel device of this kind is an LED matrix display. Partiallydistributed processing describes a system in which one central processor coordinates the behavior for anetwork of smart devices. Pixels that act as “slaves” in a larger network structure, but contain anembedded microprocessor for local sensor or display control, fall into this category. Peano and 20/20Refurbished are networks with partially distributed processing. Fully distributed processing involves nocentral coordination for a network of smart devices. In this case, each physical pixel contains a processorthat controls all behavior and communications for that device. Displays with fully distributed processingcannot be treated as a Cartesian system, but a decentralized network in which emergent behaviors arevisible. Nami is an example of a physical pixel network with fully distributed processing.

45

Mechanical

Mechanical connectivity is immaterial for wireless physical pixels and straightforward for displays that arefully tethered. However, reconfigurable display systems that require direct contact for electricalconnectivity require good mechanical design. The “flakiness” of mechanical connectors is often thedownfall of a modular display.1 The connectors for a modular system should discourage wrong electricalconnections, restrict movement to minimize a noisy signal, and protect the user from electrical shock. Careshould also be taken to avoid the use of magnetic connectors where the magnetic field might interfere withan analog circuit (although we did not experience this problem due to an almost exclusively digital circuit).

Display Materials

This section suggests a few ideas for creating color and graphics, but I encourage the reader to considermore creative solutions to these reasonably traditional materials. For example, I have often entertained theidea of using a matrix of addressable scents to control the swarming of insects as though they were a slow-motion digital video display. Be creative; the physical pixel can be anything you make it to be.

Generally speaking, display materials fall into three categories: emissive, reflective and mechanical. Areflective material or mechanical device should be used wherever possible due to a significantly lowerpower budget. Emissive materials need a constant supply of power to maintain their state, whereasreflective or mechanical substances are usually actuated only during a change of state. Unfortunately, theonly commercial materials that display a continuous range of color with a refresh rate of at least 30 Hz areemissive. E Ink™ is a new, thin film reflective material that will update at 30 Hz or higher, but only twodiscrete colors are attainable at the present time. Light Emitting Didodes (LEDs) are currently the bestoption for color graphics due to their relative power efficiency, ease of use and wide frequency range.Commercially available LEDs are brightest when purchased as a discrete color with a narrow viewingangle, although difficult to diffuse completely without a milky or “pastel” effect. A multicolored LEDcombines an R, G and B filament in one diffuse package that delivers uniform color and a wider viewingangle, but light intensity is sacrificed.2 LEDs are not ideal for volumetric displays that are transparent dueto the need for a diffusive material to achieve color blending. Also, diffusive layers must be offset at least0.75” from the light source to be effective, thereby contributing a minimal thickness to the display.Incidentally, this offset was the only factor determining the depth of the “pots of paint” in the DigitalPalette.3 Organic Light Emitting Diodes (OLEDs) are an emerging technology that is much smaller thantraditional LEDs, enabling color diffusion without a diffusive layer; and they may also be printed on aflexible, clear plastic that will appear semi-transparent in layers. Phosphor-based materials, such aselectroluminescent (EL) film, are a voltage-controlled alternative to LEDs that provide a monochromaticand opaque source of light. With any display material, the life span should be taken into consideration.Finally, every display material will have a unique dynamic range. Where color is the desired effect,gamma correction should be implemented, as described in detail by Foley [et al].4

1 We were fortunate to have fairly reliable connectors for Peano, despite the fact that they were custom-made. Basedon prior experience with other modular systems, we worked really hard to maintain precise tolerances in the custommanufacture and we strongly encourage this approach, albeit tedious and expensive. The female pads of a Peano cubeare gold-plated for longevity and the male connectors are spring probes available through Interconnect Devices, Inc.The spring probes of the male connector were chosen for their high current rating, low contact resistance, durability andlong mating cycle. Had there been a suitable connector available off the shelf, we would have used it without question;but the rarity of modular, digital systems meant that we were on our own. Radial symmetry is especially hard toachieve with multiple data lines, and rotating connectors (such as a multi-conductor slip ring) are totally out of the pricerange of a toy (starting at $60 and up).2 A Peano cube contains a single full spectrum, sunlight visible RGB LED available through LEDtronics, Inc. A Namiorb uses a cluster of discrete red, green and blue LEDs available from Nichia Corporation, but I am not aware of adomestic vendor and the components are difficult to purchase direct.3 I considered making a make-up compact with digital eye shadow, etc., but this cannot be done with standard RGBLEDs due to the depth required for adequate diffusion. Perhaps OLEDs will make this project possible.4 Foley, James D. [et al.] Computer Graphics: principles and practice (Second Edition in C). Addision-WesleyPublishing Company, Inc., Reading, MA, 1997. Pages 564-5.

46

Physical Design

Of all the engineering requirements for a display, aesthetic design is often be the hardest to achieve and theeasiest to ignore. Appearance is not critical to the success of engineering, so often a prototype will workbut look terrible due to multiple sacrifices in aesthetics. This is the greatest failure since, after all, it is adisplay. As discussed for Nami and Peano, decisions in the physical design have major implications for theperception and applications of the system. Circuitry, batteries, sensors and connectors are all likelycomponents of a physical pixel and they must be accommodated without obscuring the visual effect.Similarly, a tangible display element should not be impossible to touch due to heat, electricity, fragility orother consequences of poor design.

47

48

Bibliography

Alloway, Lawrence. Roy Lichenstein. Abbeville Press, New York NY, 1983.

Anagnostou, G., Dewey, D., and Patera, A, “Geometry-defining processors for engineering design andanalysis,” in: The Visual Computer, 5:304-315, 1989.

Anderson, D., Frankel, J., Marks, J., et al., “Building Virtual Structures With Physical Blocks (DemoDescription), in: Proc. of UIST’99.

Atkins, P. W. Molecules. W. H. Freeman and Company, New York, NY, 1987.

Baudrillard, Jean. Simulacra and Simulation. The University of Michigan Press, Ann Arbor, MI, 1994.

Bohm, David. Wholeness and the Implicate Order (Second Edition). Routledge, New York, NY, 1995.

Borovoy, Rick and Kramer, Kwindla, Programmable Beadshttp://el.www.media.mit.edu/projects/beads/index.html. 1997

Brotchie, Alastair. Surrealist Games. Redstone Press. London, England. 1991.

Chip, Herschel B. Theories of Modern Art: a source book by artists and critics. University of CaliforniaPress, Inc., Berkeley, CA, 1968.

Comiskey, J. D. Albert, Hidekazu Yosihizawa & Joseph Jacobson, “An electrophoretic ink for all-printedreflective and electronic displays.” Nature, Vol. 394, 16 July 1998.

Conrac Corporation. Raster Graphics Handbook (Second Edition). Van Nostrand Reinhold Company, Inc.,New York, NY. 1985.

Coore, Daniel, “Establishing a Coordinate System on an Amorphous Computer,” in: 1998 MIT StudentWorkshop on High Performance Computing in Science and Engineering, MIT/LCS/TR-737

Coore, Daniel and Nagpal, Radhika, “Implementing Reaction Diffusion on an Amorphous Computer,” in:1998 MIT Student Workshop on High Performance Computing in Science and Engineering, MIT/LCS/TR-737

Coyne, Richard. Technoromanticism: digital narrative, holism, and the romance of the real. MIT Press,Cambridge, MA, 1999.

Cubitt, Sean. Digital Aesthetics. Sage Publications Ltd, London, 1998.

Cypher, Allen (Editor); co-edited by Daniel C. Halbert … [et al.]. Watch What I Do: programming bydemonstration. The MIT Press, Cambridge, MA, 1994.

David S. Ebert, Edward Bedwell, Stephen Maher, Laura Smoliar, Elizabeth Downing: Realizing 3DVisualization Using Grossed-beam Volumetric Displays. CACM 42(8): 100-107 (1999)

Downing, Elizabeth, Hesselink, Lambertus, Ralston, John and Macfarlane, Roger, “A Three-Color, Solid-State, Three-Dimensional Display,” Science 1996 August 30; 273: 1185-1189.

Elkins, James. The Object Stares Back: on the nature of seeing. Harcourt, Inc., Orlando, FL, 1996.

Fielding, Raymond (Editor). A Technological History of Motion Pictures and Television: an anthologyfrom the pages of the Journal of the Society of Motion Pictures and Television Engineers. University ofCalifornia Press, Berkeley, CA, 1967.

49

Fink, Donald G. Computers and the Human Mind. Doubleday and Company, Inc., Garden City, NY, 1966.

Foley, James D. [et al.] Computer Graphics: principles and practice (Second Edition in C). Addision-Wesley Publishing Company, Inc., Reading, MA, 1997.

Frazer, J. An Evolutionary Architecture. Architectural Association: London, 1994.

Goodman, Cynthia. Digital Visions: Computers and Art. Harry N. Abrams, Inc., New York, NY. 1987.

Gorbet, M., Orth, M., and Ishii, H., “Triangles: Tangible Interface for Manipulation and Exploration ofDigital Information Topography,” in: Proc. of CHI98, pp. 49-56.

H. Abelson, D. Allen, D. Coore, C. Hanson, G. Homsy, T. Knight, R. Nagpal, E. Rauch, G. Sussman andR. Weiss, “Amorphous Computing,” AI Memo 1665, August 1999.

Hearst, Marti A., “TileBars: Visualization of Term Distribution Information in Full Text Information Access,” in: Proc.of CHI99 .

Heaton, K., Poor, R., Wheeler, A., “Nami,” in: Conference Abstracts and Applications of SIGGRAPH99,Emerging Technologies, p. 214.

Heidegger, Martin. The Question Concerning Technology and Other Essays. Harper and Row Publishers,Inc., New York, NY, 1977.

Holtzman, Steven. Digital Mosaics: the aesthetics of cyberspace. Simon and Schuster, Inc., New York, NY,1998.

Ishii, H. and Ullmer, B, “Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms,” in:Proc. of CHI97, pp. 234-241.

Kirsch, Russell A. IEEE Annals of the History of Computing, Institute of Electrical and ElectronicsEngineers, Inc., 1998, pp. 7 – 13.

Kramer, K. and Minar, N., Stackables: Manipulated Distributed Displays.http://el.www.media.mit.edu/projects/stackables/. 1997

Leavitt, Ruth. Artist and Computer. Crown Publishers, Inc., New York, NY, 1976.

Leigh, J., Johnson, A., DeFanti, T., “CAVERN: A distributed architecture for Supporting ScalablePersistence and Interoperability in Collaborative Virtual Environments.” Virtual Reality: Research,Development and Applications, Vol 2.2, December 1997, pp. 217-237.

Maeda, John. Design By Numbers. The MIT Press, Cambridge, MA, 1999.

McLuhan, Marshall. Understanding Media: the extensions of man. The MIT Press, Cambridge, MA, 1994.

Mitchell, William J. The Art of Computer Graphics Programming. Van Nostrand Reinhold Company, Inc.,New York, NY, 1987.

Negroponte, Nicholas. Being Digital. Random House, Inc., New York, NY, 1996.

Newhall, Beaumont, The history of photography : from 1839 to the present. Museum of Modern Art, NewYork, NY, 1982.

50

Newman, William and Sproull, Robert. Principles of Interactive Computer Graphics (Second Edition).McGraw-Hill Book Company, Inc., New York, NY, 1979.

Norman, Richard B. Electronic Color: the art of color applied to computer graphic computing. VanNostrand Reinhold, New York, NY, 1990.

O’Brian, John. Clement Greenburg: The Collected Essays and Criticism Vol. 4 - Modernism With aVengeance, 1957-1969. University of Chicago Press, Chicago, IL, 1995.

Poor, R. Hyphos: A Self-Organizing, Wireless Network. MS thesis, MIT Media Laboratory, 1997.

Popper, Frank. Agam (Third Revised Edition). Harry N. Abrams, Inc., New York, NY, 1990.

Popper, Frank. Origins and Development of Kinetic Art. Studio Vista Limited, London, 1968.

Prueitt, Melvin L. Art and the Computer. McGraw-Hill Book Company, Inc. New York , NY 1984.

Rodis -Lewis, Genevieve. Descartes: his life and thought. Cornell University Press, Ithaca, NY, 1998.

Russett, Robert and Starr, Cecile. Experimental Animation (Revised Edition). Da Capo Press, Inc., NewYork, NY, 1976.

Rutsky, R. L. High Techn�: art and technology from the machine aesthetic to the posthuman. Universityof Minnesota Press, Minneapolis, MN, 1999.

Silvers, Robert. Photomosaics. Henry Holt and Company, Inc., New York, NY, 1997.

Soller, Theodore (Editor). Cathode Ray Tube Displays. McGraw-Hill Book Company, Inc., New York,NY, 1948.

Sontag, Susan. On Photography. Bantam Doubleday Dell Publishing Group, Inc., New York, NY, 1977.

St.-Hilaire, P., Benton, S. A., Lucente, M., and Hubel, P.M., “Color images with the MIT holographicvideo display," in: S.A. Benton, ed., SPIE Vol. 1667, Practical Holography VI (Feb., 1992), paper 1667-73,pp. 73-84.

Sutherland, I. E. Sketchpad: A Man-Machine Graphical Communication System. Technical Report No.296, 30 January 1963. Massachusetts Institute of Technology Lincoln Laboratory.

Thompson, D’Arcy Wentworth. On Growth and Form (abridged edition). Cambridge University Press,Cambridge, 1961.

Tufte, Edward R. Visual Explanations: Images and Quantities, Evidence and Narrative. Graphics Press,Inc., Cheshire, CT, 1997.

Turkle, Sherry. The Second Self : computers and the human spirit. Simon and Schuster, New York NY,1984.

Underkoffler, J. The I/O Bulb and the Luminous Room. MIT Media Laboratory Ph.D. thesis, 1999.

Wolfe, Rosalee (Editor). Seminal Graphics: Pioneering Efforts That Shaped the Field. A Publication ofACM SIGGRAPH, 1998.

Wolfreys, Julian (Editor). The Derrida Reader: writing performances. University of Nebraska Press,Lincoln, Nebraska, 1998.

51

Yarin, P. Towards the Distributed Visualization of Usage History. MS thesis, MIT Media Laboratory,1999.