IEEE spect rum OCTOBER 1967

&pr i n ted from I E E E S P E C T R U MVol. 4, No . 10, October 1967

pp. 89-95Copyr igh t 1967, and reprinted b y permission of the copyr ight owner

P R I N T E D I N T H E U . S . A .

The d ig i ta l computeras a creative m e d i u mIn the computer, man has created not just an inanimatetool but an intellectual and active creative partner that,when fully exploited, could be used to produce wholly newart forms and possibly new aesthetic experiences

A . M i c h a e l N o l l B e l l Telephone Laboratories

Digital computers are now being used to produce musical sounds and togenerate artistic visual images. The artist or composer interacts directlywith the computer through a console. This article explores the possibilitiesof the computer as an artistic medium and makes some predictions about theart of the future.

The notion of creating art works through the medium of machines mayseem a little strange. Most people who have heard about the experimentaluse of digital computers in creative endeavors have probably shruggedthem off as being of no consequence. On the one hand, creativity hasuniversally been regarded as the personal and somewhat mysteriousdomain of man; and, on the one hand, as every engineer knows, thecomputer can only do what i t has been programmed to do—whichhardly anyone would be generous enough to call creative,

Nonetheless, artists have usually been responsive toexperimenting with and even adopting certain conceptsand devices resulting from new scientific and technologicaldevelopments. Computers are no exception. Composers,film animators, and graphic artists have become interestedin the application o f computers in their creative en-deavors. Moreover, recent artistic experiments w i thcomputers have produced results that should make us re-examine our preconceptions about creativity and ma-chines. Some of the experiments, described in this article,suggest, in fact, that a tight interaction between artistand computer constitutes a totally new, active, and ex-citing artistic medium.

How does an artist work?There is an anecdote attributed to Henri Matisse about

how to approach the creative act of painting. You take ablank white canvas, the French artist said, and aftergazing at it for a while, you paint on it a bright red disk.Thereafter, you do nothing further until something oc-curs to you that will be just as exciting as the originalred disk. You proceed in this way, always sustaining,through each new gambit with the paint and brush, theinitial high visual excitement of the red disk.

The anecdote i s a somewhat simplified versionof Matisse's idea, but even i f we take it lightly, it can doa number of things for us. For one thing, it dispels someof the sense of mystery that hovers over the proceduresof the creative person. I t tells us something concreteand easily visualized about the creative process whileemphasizing the role o f the unexpected ideas for whichthe artist lies in wait and for which he sets a formal"trap" in his medium.

Even a relatively "passive" medium paint , brushes,canvas---will suggest new ideas to the artist as he becomesengaged. The resistance of the canvas or its elastic give tothe paint-loaded brush, the visual shock of real color andline, the smell of the paint, will all work on the artist'ssensibilities. The running of the paint, or seemingly "ran-dom" strokes of the brush, may be accepted by him ascorporate elements of the finished work. So it is that anartist explores, discovers, and masters the possibilitiesof the medium. His art work is a form of play, but it isserious play.

Most of all, the Matisse anecdote suggests that the ar-tistic process involves some form of "program," one cer-tainly more complex than the anecdote admits, but adefinite program of step-by-step action. Without doing toomuch violence to our sense of what is appropriate, wemight compare it to a computational hill-climbing tech-nique in which the artist is trying to optimize or stabilizeat a high level the parameter "excitement."

Once we have swallowed this metaphor, it becomes lessimprobable to imagine that computers might be used, invarying depths of engagement, as active partners in theartistic process. But computers are a new medium. Theydo not have the characteristics o f paints, brushes, andcanvas. Nor are the •'statements" that grow out o f theartist's engagement with them likely to be similar to thestatements of, for example, oil paintings. An interestingquestion to explore, then, is how computers might beused as a creative medium. What kinds of artistic po-tentials can be evolved through the use o f computers,which themselves are continually being evolved to possessmore sophisticated and intelligent characteristics?

The character of the computer mediumIn the present state o f computer usage, artists are

certainly having their problems in understanding en-gineering descriptions and in learning how to programcomputers in order to explore what might be done withthem. However, they are learning, and they have alreadyused digital computers and associated equipment to pro-duce musical sounds and artistic visual images.'-'

The visual images are generated by an automatic plotterunder the control o f the digital computer. The plotterconsists of a cathode-ray tube and a camera for photo-graphing the images "drawn" on the tube face by de-flections o f the electron beam. The digital computerproduces the instructions for operating the automaticplotter so that the picture-drawing capability is underprogram control. Musical sounds are produced by thecomputer by means of a digital sampled version o f thesounds that must then be converted to analog form by aconventional digital-to-analog converter.

For both of these artistic applications, a challengingproblem i s the composition o f special-purpose pro-gramming languages and subroutines so that the artistcan communicate with the computer by using terminologyreasonably similar to his particular an. For example, aspecial music compiler has been written so that the com-poser can specify complex algorithms for producing asingle sound and then pyramid these basic sounds intoa whole composition. A similar philosophy has beenused in a special language developed for computer anima-tion called Bellix.5 Both applications share the draw-back that the artist must wait a number o f hours be-tween the actual running of the computer program andthe f inal generation o f pictorial output o r musicalsounds when he can see or hear the results.

Since the scientific community currently is the biggestuser of computers, most descriptions and ideas about theartistic possibilities fo r computers trd%e been under-standably written by scientists and engineers. This situa-tion will undoubtedly change as computers become moreaccessible to artists who obviously are more qualified toexplore and evolve the artistic potentials of the computermedium. Unfortunately, scientists and engineers areusually all too familiar with the inner v. orkina of com-puters, and this knowledge has a tendency to producevery conservative ideas about the possibilities for com-puters in the arts. Most certainly the computer is anelectronic device capable of performing only those opera-tions that i t has been explicitly instructed to perform.And this usually leads to the portrayal of the computeras a powerful tool but one incapable of any true creativity.However, if creativity is restricted to mean the productionof the unconventional or the unpredicted, then the com-puter should instead be portrayed as a creative medium—an active and creative collaborator with the artist.

Computers and creativityDigital computers are constructed from a myriad o f

electronic components whose purpose is to switch mi-nute electric currents nearly instantaneously. The inner-most workings of the computer are controlled by a setof instructions called a program. Although computersmust be explicitly instructed to perform each operation,higher-level programming languages enable pyramidingof programming statements that are later expanded intothe basic computer instructions by special compiler pro-

p H spectrum OCT013ER 1967

This f rame i s selectedfrom a movie producedby Stan Vanderbeek us-ing a special animationprogramming l anguagedevised by Ken Knowlton.Each f rame consists o fa fine mosaic of dots thatare combined t o makedesired s h a p e s a n dforms. Intriguing and un-usual "d i sso lves " a n d"stretches" t h a t a r eeasily d o n e u s i n g t h ecomputer wou ld b e te -dious if not impossible toexecute b y conventionalhand a n i m a t i o n t e c h -niques.

Noll—The digital computer as a creative medium

grams. These programming languages are usually de-signed so that the human user can write his computerprogram using words and symbols similar to those o fhis own particular field. All of this leads to the portrayalof the computer as a tool capable of performing tasksexactly as programmed.

However, the computer is such an extremely powerfultool that artistic effects can sometimes be easily ac-complished that would be virtually impossible by con-ventional artistic techniques. For example, by calculatingand drawing on the automatic plotter the perspectiveprojections from two slightly different directions of somethree-dimensional object, the computer can generatethree-dimensional movies o f novel shapes and forms.Such three-dimensional animation, or kinetic sculpture,is far too tedious to perform by any other method. Thecomputer's ability to handle small details has made pos-sible intriguing dissolves and stretches, such as thoseexecuted by Stan Vanderbeek, without the tedium o fconventional hand animation. Mathematical equationswith certain specified variables under the control of theartist have also been used by John Whitney to achievecompletely new animation effects. Much of "op art" usesrepetitive patterns that usually can be expressed verysimply in mathematical terms. The waveforms repro-duced on page 89, which are like Bridget Riley's paint-ing "Currents," were generated as parallel sinusoids withlinearly increasing period. Thus, computer and automa-tic plotter can eliminate the tedious part o f producing"op" effects.

Computers most certainly are only machines, but theyare capable o f performing millions o f operations in afraction of a second and with incredible accuracy. Theycan be programmed to weigh carefully, according tospecified criteria, the results o f different alternativesand act accordingly; thus, in a rudimentary sense, com-puters can appear t o show intelligence.' They mightassess the results of past actions and modify their pro-grammed algorithms to improve previous results; com-puters potentially could be programmed to learn. Andseries of numbers can be calculated by the computer thatare so complicatedly related that they appear to us asrandom.

Of course, everything the machine does must be pro-grammed, but because o f the computer's great speed,freedom from error, and vast abilities for assessment andsubsequent modification of programs, it appears to us toact unpredictably and to produce the unexpected. In thissense, the computer actively takes over some of the artist'screative search. It suggests to him syntheses that he mayor may not accept. I t possesses at least some of the ex-ternal attributes of creativity.

The Mondrian experimentHow reasonable is i t t o attribute even these rudi-

mentary qualities of creativity to an inanimate machine?Is creativity something that should only be associatedwith the products o f humans? Not long ago, in 1950,A. M. Turing expressed the belief that at the end of thecentury "one will be able to speak of machines thinking

92

1 1 I • 1 • • 4 1 L I

1 -1 •IL - I I 11 I IF111 - 1 •I . L r i 4 . 1 l i n I 1 Imo= 1II 1 1 : • • 11 i 1 1 I +1

• O n l ,

-I 1 ,+I t t i - t 1 • 1 1 4 " 1.1. I .14mmi1 41t I I

1 9 T I —I 4- • 1• , I 1_ m m I • o I - I 1

1_11741 11_1_1_11 11t- 1 1 on 1-1.1 1 n 1 4.1 J4"-

without e x p e c t i n g to be contradicted."' Turing proposedthe now well-known experiment consisting of an inter-rogator, a man, and a machine, in which the interrogatorhad to identify the man by asking the man and the ma-chine to answer questions or to perform simple tasks.

A crude approximation to Turing's experiment wasperformed using Piet Mondrian's "Composition WithLines" (1917) and a computer-generated picture com-posed of pseudorandom elements but similar in overallcomposition to the Mondrian painting.8 Although Mon-drian apparently placed the vertical and horizontal barsin his painting in a careful and orderly manner, the barsin the computer-generated picture were placed accordingto a pseudorandom number generator with statisticschosen to approximate the bar density, lengths, and widthsin the Mondrian painting. Xerographic copies o f thetwo pictures were presented, side by side, to 100 subjectswith educations ranging f rom high school t o post-doctoral; the subjects represented a reasonably goodsampling of the population at a large scientific researchlaboratory. They were asked which picture they pre-ferred and also which picture of the pair they thought wasproduced by Mondrian. Fifty-nine percent o f the sub-jects preferred the computer-generated picture; only 28percent were able to identify correctly the picture pro-duced by Mondrian.

In general, these people seemed to associate the ran-

1:;111!4-1 114:1— 11— TJ11 — •1 1 _ A- 1 II- I —1 1 o-14AJ vFil 1I I 1 • • 1 •1•1

1 1•1 I I IL 1-111• I_ 1 1 : _ N o II I 11Mil I I I I luml• I I , 1

I I I — me .1 IHN1 m t . 1 1 1 1 I 11 • n o m1 H

11-p-•Er 1711•1111I 1 •The picture on the left, "Composition With Lines"(C) Rykmuseum-Kroller-MUller), is a reproductionof a work by the Dutch painter PieeMondrian. Thepicture on the right was generated by a digital com-puter using pseudorandom numbers with statisticsapproximating the Mondrian painting ((DA. MichaelNoll 1965). When xerographic reproductions of bothpictures were shown to 100 subjects, the computer-generated picture was preferred by 59 of them. Only28 subjects identified the Mondrian painting. Ap-parently, many of the observers associated ran-domness with human creativity and were thereforeled astray in making the picture identifications.

domness of the computer-generated picture with humancreativity whereas the orderly bar placement o f theMondrian painting seemed to them machinelike. Thisfinding does not, o f course, detract from Mondrian'sartistic abilities. His painting was, after all, the inspira-tion for the algorithms used to produce the computer-generated picture, and since computers were nonexistent50 years ago, Mondrian could not have had a computerat his disposal. Furthermore, we must admit that thereduction in size of the original painting and its xero-graphic reproduction degrades its unique aesthetic quali-ties. Nevertheless, the results of the experiment in lightof Turing's proposed experiment do raise questions onthe meaning of creativity and the role of randomness inartistic creation. In a sense, the computer with its pro-gram could be considered creative, although i t can beargued that human creativity was involved in the original

IEEE s n e c t r u m OCTOBER 1 9 6 7

Noll—The digital computer as a creative medium

program with the computer performing only as an obe-dient tool.

These questions should perhaps be examined moredeeply by more ambitious psychological experimentsusing computer-generated pictures as stimuli.

Toward real-time interactionAlthough the experiments described show that the

computer has creative potentialities beyond those of justa simple tool, the computer medium is still restrictive inthat there is a rather long time delay between the run-ning of the computer program and the production of thefinal graphical o r acoustic output. However, recenttechnological developments have greatly reduced thistime delay through special interactive hardware facilitiesand programming languages. This tightening of the man—machine feedback loop is particularly important for theartist who needs a nearly instantaneous response.

For example, in the field of music an electronic graphicconsole has been used to specify pictorially sequences ofsounds that were then synthesized by the computer."Functions for amplitude, frequency, and duration of asequence of notes were drawn on the face of a cathode-ray tube with a light pen. I f desired, the computer com-bined specified functions according t o transparentlysimple algorithms. Thus, the fine details o f the com-position were calculated by the computer and the over-all structure was precisely specified by the graphicalscore. The feedback loop was completed by the computer-generated sounds heard almost immediately by the com-

poser, who could then make any desired changes in thescore.

A similar man—machine interactive system has beenproposed for choreography.'" In this system, the chore-ographer would be shown a computer-generated three-dimensional display of complicated stick figures movingabout on a stage, as shown on page 94. The choreog-rapher interacts with the computer by indicating thespatial trajectories and movements of the figures. Randomand mathematical algorithms might be introduced bythe computer to fill in certain fine details, or even to givethe choreographer new ideas to evaluate and explore.

A new active mediumThe beginnings of a new creative partnership and col-

laboration between the artist and the computer clearlyemerge from these most recent efforts and proposals.Their common denominator is the close man—machineinteraction using the computer to generate either musicalsounds or visual displays. The computer acquires a crea-tive role by introducing randomness or by using mathe-matical algorithms to control certain aspects of the ar-tistic creation. The overall control and direction of thecreative process is very definitely the artist's task. Thus,the computer is used as a medium by the artist, but thegreat technical powers and creative potentialities of thecomputer result in a totally new kind of creative medium.This is an claire medium with which the artist can in-teract on a new level, freed from many of the physicallimitations o f all other previous media. The artistic po-

93

94

The choreographer of thefuture migh t s i t a t t heconsole o f a computerand see a display of hu-man figures stylized b ysimple s t ick f igures a sshown i n these framesfrom a computer-gen-erated movie. B y inter-acting with the computer,the choreographer mightcreate his dance compo-sition, perhaps leavingcertain movements to besuggested b y pseudo-random and mathemati-cal algorithms within thecomputer.

tentialities of such a creative medium as a collaboratorwith an artist are truly exciting and challenging.

Interactive aesthetic experiencesIn the previous examples the artist sat at the console

of the computer and indicated his desires to the computerby manually using push buttons or by drawing patternson an electronic visual display. These are probably effi-cient ways of communicating certain types of instructionsto the computer; however, the communication of the ac-tual subconscious emotional state of the artist could leadto a new aesthetic experience. Although this might seemsomewhat exotic and conjectural, the artist's emotionalstate might conceivably be determined b y computerprocessing o f physical and electrical signals from theartist (for example, pulse rate and electrical activity ofthe brain). Then, by changing the artist's environmentthrough such external stimuli as sound, color, and visualpatterns, the computer would seek to optimize the aes-thetic effect of all these stimuli upon the artist accordingto some specified criterion.

This interactive feedback situation with controlled en-vironment would be completely dynamic. The emotionalreaction of the artist would continually change, and thecomputer would react accordingly either to stabilize theartist's emotional state or to steer it through some pre-programmed course. Here then is a completely newaesthetic experience utilizing man-machine communica-tion on the highest (or lowest, i f you will) subconsciouslevels and computer processing and optimization of emo-tional responses. Only a digital computer could performall the information processing and generate the sights andsounds of the controlled environment required for sucha scheme. One is strongly tempted to describe these ideasas a consciousness-expanding experience in associationwith a psychedelic computer!

Although such an artistic feedback scheme is still farin the future, current technological and psychological in-vestigations would seem to aim in such a direction. Forexample, three-dimensional computer-generated colordisplays that seem to surround the individual are cer-tainly already within the state o f the art. Electroen-cephalograms are being scrutinized and studied in greatdetail, using advanced signal analysis techniques; i t isnot inconceivable that some day their relation to emo-tional state might be determined.

Artistic consequencesPredictions of the future are risky in that they may be

really nothing more than what the person predictingwould like to see occur. Although the particulars shouldbe viewed skeptically, they actually might be unimpor-tant; i f the art of the future follows the directions out-lined here, then some general conclusions and statementscan be made that should be independent o f the actualparticulars.

The aesthetic experience will be highly individualistic,involving only the individual artist and his interactionswith the computer. This type of participation in the cre-ative and aesthetic experience can be experienced byartist and nonartist alike. Because of the great technicaland creative power of the computer, both the artist andnonartist are freed from the necessity of strong technicalcompetence in the use o f different meila. The artist's"ideas" and not his technical ability in manipulating

IEEE s p e c t r u m OCTOBER 1 9 6 7

media could be the important factor i n determiningartistic merit. Conceivably, a form o f "citizen-artist"could emerge, as envisioned by M o n Schoener." Theinteractive aesthetic experience with computers mightHIE a substantial portion of that great leisure time pre-dicted for the man of the future.

The artist's role as master creator will remain, how-ever, because even though the physical limitations o fthe medium will be different from traditional media, histraining, devotion, and visualization will give him a higherdegree o f control o f the artistic experience. As an ex-ample, the artist's particular interactions with the com-puter might be recorded and played back by the publicon their own computers. Specified amounts o f inter-action and modification might be introduced by the in-dividual, but the overall course o f the interactive ex-perience would still follow the artist's model. I n thisway, and for the first time, the artist would be able tospecify and control with certainty the emotional stateof each individual participant. Only those aspects de-liberately specified by the artist might be left to chance orto the whims of the participant. A l l this would be pos-sible because the computer could monitor the partici-pant's emotional state and change it according to theartist's specifications. The artist's interaction with thecomputer would be of a new order because the physicalrestrictions of We older media would be eliminated.

This is not to say that the traditional artistic mediawill be swept away; but they will undoubtedly be in-fluenced by this new active medium. The introduction ofphotography t h e new medium o f the last century -helped to drive painting away from representation, butit did not drive out painting. What the new creative com-puter medium will do to all of the art forms—painting,writing, dance, music, movies—should be exciting toobserve. We might even be tempted to say that the cur-rent developments and devices i n the field o f man-machine communication, which were primarily intendedto give insight into scientific problems, might in the endprove to be far more fruitful, or at least equally fruitful,in the arts.

REFERENCES1. Mathews, M . V. , "The digital computer as a musical instru-ment,"Sci., vol. 142, no. 3592, pp. 553-557, Nov. 1,1963.2.. Rockman, A. , and Mezei, L., "The electronic computer as anartist," Canadian Art, vol. XXI , pp. 365-367, Nov./Dec. 1964.3. No l l , A. M., "Computers and the visual iiris," Design Quarterly,no. 66/67, pp. 65-71,1967.4. Zajac, E. E., "Computer animation: a new scientific and educa-tional too l , " J. Soc. Motion Picture and Telecision Eng., vol. 74,pp. 1006-- 1008, Nov. 1965.5. Knowlton, K . C., " A computer technique for producing ani-mated movies," Am. Fed. In/or. Proc. Soc. (AFIPS) Conf. Proc.,vol. 25, pp. 67-87,1964.6. Minsky, M . L . , "Ar t i f ic ia l intell igence," Sci. Am. , vo l . 215,pp. 246-260, Sept. 1966.7. Turing, A . M . , "Compu t ing machinery a n d intel l igence,"Mind. A quarterly review o f psychology and philosophy, vol. 59,N.S., pp. 433-460, Oct. 1950.8. Nol l , A . M. , "Human or machine: a subjective comparison o fPiet Mondrian's 'Composition with Lines' (1917) and a computer-generated p ic ture, " The Psychological Rec., vo l . 16, pp . 1-10,Jan. 1966.9. Mathews, M . V. , " A graphical language fo r composing andplaying sounds and music," presented a t the 3Is t Cony, o f theAudio Engineering Society, Oct. 1966 (preprint no. 477).10. No l l , A . M. , "Choreography and computers," Dance Maga-zine, vol. XXXXI , pp. 43-45, Jan. 1967.I I , Schoener, Al lon, "2066 and all that," A r t in America, vol. 54,pp. 40-43, Mar. -Apr. 1966.

Noll---The digital computer as a creative medium

These f rames a r e t h eleft and right images of a3D pair from a computer.generated three-dimen-sional movie. The objectrandomly c h a n g e s i t sshape in what might becalled a form o f kineticsculpture. To v iew t h e3D effect, place a sheetof paper on edge betweena stereo pa i r. Posit ionyour head so tha t eacheye sees only one image.With a bit of adjustment,the images should seemto converge and appearthree-dimensional.

95