elemental counterpoint with digital imagery

Elemental Counterpoint with Digital ImageryAuthor(s): Brian EvansSource: Leonardo Music Journal, Vol. 2, No. 1 (1992), pp. 13-18Published by: The MIT PressStable URL: http://www.jstor.org/stable/1513203 .

Accessed: 16/06/2014 23:10

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

The MIT Press is collaborating with JSTOR to digitize, preserve and extend access to Leonardo Music Journal.

http://www.jstor.org

This content downloaded from 194.29.185.251 on Mon, 16 Jun 2014 23:10:04 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=mitpress

http://www.jstor.org/stable/1513203?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


ARTI ST' S ARTI C LE

Elemental Counterpoint

with Digital Imagery

Brian Evans

vime-based art is built on the unfolding of discrete events. In analysis these events are measured in modular form showing the structural basis of the work from macro to micro levels. This is perhaps most apparent in music.

In music it is possible to measure a piece in terms of move- ments, sections, phrases and periods down to individual notes. Western tonal music is based on theory that can explain the smallest atomic particle with respect to local and global characteristics of a music composition. (Tonal music is not the only musical vocabulary in which this holistic view of a composition can be found, but from classical literature to pop music it is certainly the most prevalent.)

The relationships of notes to their immediate temporal neighbors create intervals and chords, while the larger scale relationships create progressions that define pitch-centers and keys. This tight organization on all levels allows for strong structural integrity in both the horizontal (melodic) and vertical (harmonic or chordal) musical dimensions.

In the tonal system, notes are heard vertically (simultaneously) within historically established conventions. While discrete and easily describable as major or minor, consonant or dissonant, and so forth, a chord's true nature is discov- ered when it is heard in the context of its neighbors. The Smajor triad shown in the left section of Fig. 1, for example, is a familiar sound with little inherent dynamic quality when isolated. In the right section of Fig. 1, moving through conventional voice leading to a C-major triad, the original G chord is ascribed a relatively high degree of directed tension that finds resolution in the C chord. In tonal theory the G chord is given a dominant function resolving to the tonic C.

Wlaile the notes of the G triad establish a sonorous vertical entity, in moving to the C chord each voice moves independently. This independence is the basis of melodic counterpoint. The harmonic motion is strengthened by the linear autonomy of each voice, and the melodic character of each line is enhanced by its agreement with an overall harmonic scheme. The same note serves more than one structural purpose.

The use of computer graphics allows the discrete elements of a visual composition to be brought under detailed control. Abstract animation brings in the dimension of time and the desire to structure time with a formal elegance that was heretofore only achievable in music. Music composers have the advantage of an audience and a medium condi- tioned by at least 10 centuries of tradition. There are, however, some simple established principles for working with visual materials that can be applied to temporal graphic design.

In the past century visual artists have reduced their vocabularies to essentials of geometry and proportion (the

K) 1992 ISAST Pergamon Press Ltd. Printed in Great Britain. 0024-094)(/92 t5.00+0.00

distilled basis of our musical vocabularies as well). These essentials can be useful in building a time-based visual grammar and in furthering the development of contrapuntal techniques. All that are needed are the estab- lishment of horizontal (temporal) independence of select elements and avertical (spatial) systemic agreement of these elements a visual equivalent to melody and functional harmony.

These ideas have been ex- plored and documented in the work of a few digital artists, most notably John Whitney, Sr. [ 1 ], and Edward Zajec [2] . Whitney, with his differential dynamics, builds a time struc-

A B S T R A C T

In musical counterpoint, separate melodic lines move simultaneously while making structural sense with their temporal neighbors and the overall musical form. The levels of control of abstract visual material now available through computer-generated raster graphics make it possible to establish a similar contrapuntal rigor with visual elements and to explore a struc- tured method of temporal visual design. Building on earlier work in time-based color and some fundamental observations of visual balance, the author proposes and illustrates a formal approach to abstract animation that parallels contrapuntal music technique.

ture by moving through points of visual resonance. In his compositions it is possible to watch an array of points move in and out of resonance, as if following a single melodic line moving through cadences (an effective use of the conso- nance/dissonance dialectic) . There are times when this line breaks into distinct elements thatwork themselves apart and back together.

Wlaitney illustrates this with the analysis of his piece Arabesque. Simultaneous processes running with different timings and with differing controls hint at a visual 'counterpoint'. He admits that his "arbitrary working out of three fractional sequences hardly constitutes a fugue", and indicates that a plan of departures and arrivals is necessary in building a reasonable contrapuntal grammar [3].

Edward Zajec's work grapples with the illusive problem of structuring time-based color. In discussing his work ChrF mas, for example, he says "to animate means to orchestrate the flow of color passages in time, rather than to choreo- graph the motions of objects in space" [4]. He bases his imagery on a static visual composition that allows him to concentrate on "a dynamic part in which a composition is performed in a complex of color changes." His method of dimensional upgrades, borrowed from serial music techniques, provides a means of color control [5]. Combining this with thematic dissolves of static imagery, Zajec creates an elegant visual counterpoint in two parts.

Brian Evans (research artist), Vanderbilt University Computer Center, Box 1577, Station B, Nashvilles TN 37235, U.S.A.

Received 12 March 1991.

LEONARDO MUSICJOURNAL, Vol. 2, No. 1, pp. 1>18, 1992 13



Xo 108 1 7 I {} lX 8 H- Fig. 1. The left section shows a simple S major triad, a sonorous but static entity.

the movement from imbalance to stability. Arnheim gives us some good point- ers to follow in trying to build a balanced image. Elements will influence the overall balance based on their weight and direction [7]. Controlling the interaction of these elements over time provides a compositional method. We can then create visual balance and,

consequently, imbalance. Through computer animation it is also possible to interpolate from one state to another, creating a temporal movement from stability to sway and back again. Weight is affected by varying factors: among them are location, size, color, isolation and shape. Representational objects bring in another level of complexity in determining structural weight, with factors of cultural and personal audience biases coming to bear. These biases are less likely to arise in the context of abstract imagery, and for this research I make the assumption that these subjective factors are not influ- encing the viewer's perception of balance. (Representation and connota- tion introduce levels of complexity that go beyond this research, but do suggest areas for future study.)

The idea of elemental weight is sim- ply illustrated in Fig. 3. This figure dia- grams the distribution offour square tiles as seen in the first frame of Fig. 2 (upper-left square). In the figure the upper row of tiles is weighted by the number of tiles, their similarity and their left orientation. This is balanced somewhat by the lightening effect of their height on the image plane. The upper tier of squares is balanced by the lower-right square, which is weighted by its isolation, bottom-heavy location and larger size. (Color is another contributing fac- tor, which I will discuss separately.) This tile distribution is one of many possible distributions that achieve equilibrium. We could define this particular formal solution as the structural ky of the image (again borrowing from Arn- heim) [8]. The structural key can be thought of as the reduction of line and shape to its barest minimum the skeletal basis of the static visual composition. Figure 4, for example, can be seen as the structural key of the first image of Fig. 2, taking into considera- tion both the tiles and the gray-scale background. In music composition particular chord progressions and melodic mate-

The right section shows a Smajor triad resolving to a Smajor triad (a dominant- tonic progression) with appropriate voice leading. Proper voice leading assures independence of the voices.

In my work I am trying to include color and shape evolving in time into a compositional plan, offering artists another grammatical approach to the problem of temporal visual design.

BALANCE AND STRUCTURAL KEYS My first goal in my research is to balance the disparate elements making up a digital construction. These elements are seen as separate voices and are independently controllable. Balance of these elements can be considered con- gruent to vertical agreement in musical counterpoint. The long range goal is to animate these elements, having them move through different points of equilibrium, creating a coherent composition of visual counterpoint. These methods are illustrated in the storyboard study in Fig. 2. Rudolf Arnheim says, "Equilibrium is attained when the forces constituting a system compensate one another" [6]. Balance in an image should give a sense of being out of transition, with all elements having reached a state of re- pose. This balance is accomplished by the interplay of elements comprising the image. Movement through balance to imbalance and back to balance again can provide a means of structuring time with visual elements and allows us an approach to temporal design. Time is intensified and focused for the viewers, as if they were watching a tightrope walker starting from the stability of the platform, moving across the rope (swaying from imbalance to stability), and finally returning to the safety of the platform. Motion through time is marked with movement from tension to resolution. Time comes under formal control when the artist can specify

14 Evans, Elemental Counterpoint with Digital ImageIy

Fig. 2. Stoxyboard for the counterpoint study. The storyboard illustrates the progression of three &tinct compositional elements-the tile distribution on the image, the tile colors and the grey-scale background. The elements move through points of equilibrium in the keyframes, runliing on the diagonal, from upper left to lower right.

105 0 .._ ._.

__ _

2 _

t1>.

.X.: _ _xr _ | X X 'a' _ g



with the other elements to maintain the integrity of the composition as a whole. Figure 5 shows the ground element of the last frame from the study. The ground can be reduced to a basic con- tour and9 with the addition of the formal basis of the tiles, has the structural key shown in Fig. 6.

The third elemental voice in the compositional study is color. My method of measuring is based on previous work in time-based color design [9]. By summing the amounts of red, green and blue primaries on the digital image and expressing this summation as a triplet of proportions it is possible to determine whether the color is balanced or weighted towards a particular hue. Each value represents the simple formula:

amount of hae on the image maximum amount o/ hae possible

The value for each hue will always be between O and 1.

The summation of red, green and blue (rgb) for the final (lower-right) frame of the study is .418, .386, .382. This tells us that there are nearly equal amounts of red, green and blue light on the image, so the color domain is considered balanced. If this agrees with a balanced structural key, then all elements support the balance of that particular moment in the composition- moving towards a visual tonic function.

It is possible to distinctly see these three elements the distribution, grey-

l

S . .

rials work to define the overall form of the work through devices of repetition, variation and contrast. The tonal system establishes a hierarchic relationship in which the harmonic material is always striving to return to the tonic key and chord. Main melodic material also develops by establishing a hierarchy, creating a sense of resolution when the main theme returns.

Structural keys can be advanced in time-based visual design through repe-

. . . . . . . . tltlon ane varlatlon m a manner slml ar to harmonic/thematic development in music. A hierarchic relationship of structural keys can be established, function- ing temporally as the main motivic material of an abstract animation.

Using a hierarchical balance scale, these relationships are achieved in two ways. First, it is possible to perceive levels of image balance. A completely balanced image, a resolved moment, is deemed 'high' on the scale, especially if it follows low-level imbalanced im- agexy. Secondly, there are factors of temporal precedence and repetition. As long as the viewer remembers a structural key, it will hold a potentially higher position on the scale. The high position is realized with a recognized

. . . . . . repetltlon or varlatlon. As ln t ne musl- cal example of Fig. 1, function in this case is established in retrospect. The G chord is understood as dominant only after hearing the C chord a structural key is recognized as formally significant when it is seen repeated.

Using computer-graphic imagery, visual elements can establish structural keys (balanced or not), and yet evolve independently in time. With these elements able to change autonomously while maintaining functional sense in the overall temporal design, the composition of time-based visual counterpoint becomes possible. Through repetition and variation of structural keys there is motivic development unifying the whole, with the elemental units of these keys distinct yet in agreement with the motivically defined macro form.

ELEMENTAL ANALYSIS

Movement through balanced images, related by structural keys, provides a method for time-based visual composition. In order to have enough control to allow counterpoint, the images need to be broken down into discrete, indi- vidually definable elements. With digital imagery this can be done easily.

Returning to Fig. 2, we have already discussed one element of the composition the tile distribution. The distribution of four rectilinear tiles on the image plane acts as one 'voice' in a three-part, visual contrapuntal texture. A second voice is the grey-scale background. This ground is a simple mathe- matic visualization that brings contrast- ing curves and false depth cues to the image. It has its own separately controlled character, but moves consistently

Fig. 3. Tile distribution for dle first frame of dle study. Elemental weight is illustrated with the upper three squares, weighted by their number, similarity and orientation, lightened by their height on the image and balanced by the tile on the lower right. The lower tile is weighted by its bottom-heavy location, isolation and larger size.

Fig. 4. The stuctural key of dle first frame of dle study. A reduction of the image to its structural basis, the key takes into considera- tion both the tile distribution and the grey-scale background. This is a balanced structural key. The vertical lines of the ground are balanced by the horizontal line implied by the tile distribution.

Evans, Elemental Counterpoint with Digital Imagery 15

| r Y | g

| b



* [ - s-

Fig. 5. The ground element of the final frame from the study. The Fig. 6. The structural key of the last frame of the study. Similar in ground is a visualization of a simple mathematical process. Mathe- composition to the key of the first frame, balance is achieved matical processes provide abstract imagery that can be controlled through the sweeping horizontal lines of the ground balanced by separately from the tile colors and tile distribution. the implied vertical line of the tile distribution.

scale background and color as they develop in the study. When all the elements combine to create a total balance of color and shape at a given moment, that image moment is set high on the hierarchic balance scale. Achieving stability from sway gives a sense of comple- tion similar to musical cadence. On this scale, the balanced image is considered higher still if its structural key is recognized as a repeat of a key seen earlier.

Through this progression from imbalance to balance each elemental unit follows its own path. These units maintain their relationship by simultaneous motion through the structural keys that define the large-scale form of the piece.

TIME ANALYSIS

We can now analyze the study and in- vestigate its temporal design. Each element in the study interpolates between three primary key frames. The key frames in Fig. 2 are the upper-left, the center and the lower-right frames.

The upper-left frame starts the piece with a simple balanced image. Its color summations are .408, .402, .391; Its balanced structural key (see Fig. 4) has vertical lines on the left side defined by the background shape and a curving line sweeping across the image from upper left to lower right, described by the tile distribution.

The movement then interpolates to the middle image, second row, with the structural key shown in Fig. 7. While

this frame also balances forces (with the upsweep of the background balanced by the downward point of the tiles) there is no relation between this and the initial structural key. Formally it is new material. This is supported fur- ther by looking at the red, green and blue color summations for this image- .364, .284, .260. The color domain is weighted toward the larger value, red- a heightened moment.

Note the three key frames in the middle row of the storyboard (frames 4, 5 and 6 counting from left to right). Frames 4 and 6 are identical and work to articulate the center frame, which is also the middle of the piece. Like the dominant-tonic relationship of the G to C chords this is temporally understood in retrospect, after the repetition occurs. As the G is heard as dominant only after we hear the C, so the middle section is understood structurally when the repetition of materials is seen. In analytic terms the form of this middle section could be described as A-B-A with a visual focus moving from upper right to right to upper right.

Moving to the last image in the study storyboard, we see that the color has returned to a more balanced summation (.412, .386, .382). Looking again at the structural key to this last image, some familiar material becomes apparent (see Fig. 6). The structural key con- tains a tile distribution that establishes a vertical sweep on the left side of the image. The ground image creates a ris- ing sweep in the upper left with curving

bands and is balanced by an increase in line density moving to the lower right.

FranSois Molnar has discussed how the "composition of a picture depends first on the exploratory movement of our eyes no matter how we come to 'hierarchize' the surface of the picture, whether by form, by value or by color. The essential thing is the movement imposed on our eyes" [ 10] . In this study I have attempted to create the same

. . scan or eye movement ln tne startlng frame and the final frame. A contrast- ing scan punctuates the middle section. My idea is to create a sense of motivic development through contrast and repetition, building a temporal structure.

What occurs in the end frame is a variational return to the initial structural key, with the background chang- ing from a vertical to horizontal empha- sis and the tiles doing the opposite. The color supports the balance of both end- points and expresses its own elemental independence at the middle frame. In formal terms, looking at the diagonal keyframes, a simple A-B-A structure is expressed. In the middle B section the same structure is articulated on a smaller scale. Figure 8 shows my analysis of the modular and recursive forms of the study.

The three elements ground, tiles and color move smoothly through the composition of the storyboard (see Fig. 2). They are understood structurally in terms of their individual linear develop- ments in time and with respect to their

16 ','iJ{1115, Elelllelltal C,otlllterpoilt zith Digital Imagent



Visual interest would be at a minimum if the camera maintained a well- composed, static shot of the subject a balanced structural key. A whole new visual dynamic is brought into play with the swaying camera, and a sort of visual music is accomplished. A skilled camera operator with a musical sense can build and relax tension as if performing a well-constructed jazz improvisation. The result might seem somewhat dis- turbing at first, but increased interest and focus is brought to the imagerywith viewing time passing dynamically. Again, temporal design is being articulated by movement from and to balanced structural keys.

The use of elemental counterpoint in temporal design goes one step fur- ther, providing compositional control rather than depending on intuition to build visual interest composition ver- sus improvisation. The idea of temporal control of movement from and to equilibrium is constant. My research shows that it is possible to take control of the materials in a disciplined way and create a temporally coherent abstract ex- pression. It simultaneously suggests an approach and codifies a technique.

Elegance of form in composition, whether temporal or spatial, is the result of talent and technique. This becomes most apparent in the process of analysis. Whether it is a symphony by Mozart or a painting by Mondrian, the power of the work exists deep beneath the surface of what is initially seen or heard. While not immediately apparent it is often this structural underpinning that causes a work to resonate beyond emotional responses.

Max Bill asks us, "Where does structure end and art begin? " [ 14] In music composition they are inseparable. I sus- pect this is also true in abstract animation. We are at a place in the development of computer art where the tools have outstripped the techniques. It is now possible for anyone to sit down at a computer and create a colorful image or to make color move across a screen. Is there really a need to search for rigor when pushing a button will sidestep most of the hard decisions in artmak- ing? Is abstract animation to be rele- gated to the role of screen saver a randomly colored turtle-walk on every

. . computer monltor m every computer lab? Craftsmanship will never lose value, but it may be possible to achieve through a menu option. Where does that leave us?

Max Bill says, "Art is unthinkable without the effort of the individual"

Fig. 7. The structural key of ffie middle frame of ie study. While this key is balanced, it is new material. There is no similarity between this key and the stuctural keys of the first and last frames. The downward point of the tiles is balanced by the upsweep of the grey-scale background.

A B A

, f ' > Fig. 8. Formal analysis of the storyboard study shows a recursive A-B-A form. The middle section B (the second horizontal row of frames) shows a micro version of the overall structure (ab<)- the first and last frames of the section are the same.

spatial and proportional relationships. A controlled visual counterpoint is achieved. At any given point in the composition each element serves more than one formal purpose expressing distinct temporal lines, moving to define momentary structural keys and articu- lating the overall temporal picture. The elements have multiple functions within the macro/micro temporal design.

FINAL THOUGHTS

Searching for organizing principles in abstract animation is not a new en- deavor. The work of the earliest practi- tioners, such as Ruttman, Eggling and Richter, shows an understanding of the need for rigor [11]. Ijooking to the work of these pioneers as the logical extension of cubist revelations, Hans Richter felt "increasingly faced with the conflict of suppressing spontaneous ex-

. . . . presslon ln order to gam an objective understanding of a fundamental prin- ciple with which [he] could control the 'heap of fragments' inherited from the cubists.' Eggling, interested in the 'llnity of opposites', was working "to develop a vocabulary of abstract forms and then to explore its grammar and syntax by combining these forms into 'contrapuntal pairs of opposites [ 121

Seven decades later the grammar of filmmaking has developed mostly through representational motion pic- tures. Abstract animation has not reached the apex hoped for by early artists, who desired film as "an organ- ized distribution of light, . . . a tool for vision in motion rather than a setting for sentimental naturalism" [13]. With computer-graphics equipment becom- ing more affordable and accessible there is renewed growth and exploration in the medium.

As research continues it is interesting to note how these codified principles can sometimes be found as a part of common practice. This is especially true of the more experimental film methods that manage to sift down to commercial ubiquity. These practices are frequently by-products of the music-video commu- nity where visual dynamics often serve musical dynamics. For example, one currently popular approach to filmmaking involves the use of the 'drunken camera-man' technique. In this technique the camera seems to sway randomly to and from the subject, as if the camera operator were truly drunk and not quite in control of the equipment. In its best use the viewer can see the image moving away from, and returning to, a visual equilibrium.

Evans, Elemental Counterpoint with Digital Imagery 17



plications (NCSA) at the University of Illinois, Ur- bana-Champaign, U.S.A., and the support of the Computer Center at Vanderbilt University, Nash- ville, Tennessee, U.S.A.

References and Notes 1. John Whitney, Sr., "Fifty Years of Composing Computer Music and Graphics: How Time's New Solid-State Tractability Has Changed Audio-Visual Techniques", Leonardo 24, No. 5, 597-599 (1991) .

2. Edward Zajec, "Orphics: Computer Graphics and the Shaping of Time with Color", ElectronicArt, Supplemental Issue Leonardo (1988) pp. 111-116.

3. John Whitney, Sr., Digital Harmony (Peterbor- ough, NH: Byte Books, 1980) pp. 97-113.

4. Zajec, [2] p. 111.

5. Edward Zajec, "Computer Graphics: Color- Based Time", Leonardo 19, No. 1, 39-43 (1986).

6. Rudolf Arnheim, Art and Visual Perception (Berkeley: Univ. of California Press, 1974) p. 26.

7. Arnheim [6] p. 11-41.

8. Arnheim [6] p. 29.

9. Brian Evans, "Temporal Coherence with Digital Color", Digital Image Dig2tal Cinema, Supplemen- tal Issue Leonartlo, (1990) pp. 43 49.

10. Francoois Molnar, "The Unit and the Whole: Fundamental Problem of the Plastic Arts" in Gyorgy Kepes, ed., Module, Proportion, Symmetry, Rhythm (New York: Braziller, 1966) pp. 20s217.

11. R. Russett and C. Starr, Experzmental Animation (New York: Van Nostrand Reinhold, 1976) pp. 40-56.

12. S. Lawder, The Cubist Cinetna (New York: New York Univ. Press, 1975).

13. Laszlo Moholy-Nagy, Vision in Motion (Chicago: Paul Theobald, 1947) p. 273.

14. Max Bill, "Structure as Art? Art as Structure?" in Gyorgy Kepes, ed., Structure in Art and Science (New York: Braziller, 1965) p. 150.

15. Bill [14] p. 150.

16. Arnheim [6] p. 37.

[ 15] . I would change that to say: "Art is unthinkable without the effort of the informed and hungry individual". Or- der is accomplished through rigor and invention and control of materials. I believe that all artists are responsible for knowing where they came from and where they are going knowing their traditions, always inquiring and striving to contribute to the discourse. Art can- not begin or end without structure, cu- riosity and vision. Arnheim says, "The meaning of a work emerges from the interplay of activating and balancing forces" [16]. I believe it is that simple.

Acknowledgments This work is being done with the help of a grant from the National Center for Supercomputing Ap-

1 8 Evans, Elemental Counterpoint with Digital ImageIy

Shape ffie Arts of ffie Future

Join Leonardo

The International Society for 1he Arts, Sciences and Technology

Associate membership is open to professional artists, scientists, engineers, art collectors and others interested in the contemporary arts and sciences.

Associate Member Beneflts

* A subscription to the bimonthly journal Leonardo and Leonardo MusicJournal, the premier international journals of art, science and technology.

* A discount on all Leonardo/ISAST publications.

* A 50% discount on the Fine Art, Science and Technology (FAST), an electronic bulletin board covering all aspects of art, science and technology. Also, for members who have access to Internet or Bitnet, 12 electronic issues of the Leonardo Electronic News. (Send e-mail to Internet: fastXgarnet.berkeley.edu>; Bitnet: FAST@UCBGARNE>.)

* An opportunity to nominate artists for the prestigious FrankJ. Malina-Leonardo Prize. Only artists nominated by Associate Members are eligible for this prize.

* An opportunity to report and promote your own activities in Leonardo and on- line in the Leonardo Electronic News.

To become an Associate Member of Leonardo/ISAST, see the OrderForm at the back of this issue, or send a check or international money orderfor US $65 to Leonardo/ISAST,

MITPressJournals, 55Hayward St., Cambndge, MA 02142, U.S.A. Tel: 617-253-2889 Fax: 61 7-258-6779 E-mail: <hiscoxEmitsma. mit.edu>



elemental counterpoint with digital imagery

Documents