Volume 10, Number 1 Department of Computer Science Spring, 2001 Brown University

Brown University, Box 1910, Providence, RI 02912, USA

conduit!condu t!All of this became clearer to me after arecent workshop at the Institute for Mathe-matics and Its Applications at the Univer-sity of Minnesota. The conference wasintended to bring together three groups ofpeople who have overlapping interests butseldom attend the same standard academicconferences. I am a member of the “compu-tational linguistics” clan. As a group we areinterested in applying computers to prob-lems in language. Most of us would see asour ultimate goal getting a computer to

understand language. It is also relevantthat I am a “statistical language process-ing” person. That is, in my research I usecomputers to gather statistics from largebodies of text (“corpora”) and then writeprograms that use those statistics to pro-cess new text. Of particular importancehere is the work I have been doing on sta-tistical parsing, of which more later.

The second group of researchers were stat-isticians. Researchers in statistics have,among other things, developed a large bodyof tools for learning things accurately fromrepeated “trials”, as they might put it.From their perspective, a “trial” might be asentence and thus their techniques mightlet someone like me improve how I gatherthe statistical information my programs

It is a common saying in aca-demic computer science (and Isuspect in most researchareas) that you go to a con-ference not to hear the talksbut to talk with otherresearchers in your area.We say this even though alot of what we talk about isthe usual academic gossip—who got a job where, howthe industry is doing, etc. Infact, very important scien-tific information is beingpassed around, but this wouldnot be obvious to a listener; Ithink even the scientific partici-pants might not immediatelyrecognize exactly what it is thatthey are getting out of these

conversations, aside from the dirt aboutso-and-so whose spouse left him/herfor...oh, never mind!

I have come to realize that what is beingpassed around is not so much specificinformation as world views. What are theproblems that people see as important?How important is one line of researchcompared to another? Why does a particu-lar researcher spend time working on Xwhen to me, at least, it does not seem tobe going anywhere?

STATISTICAL PARSING FOR SPEECHRECOGNITION, or HOW I CAMEBACK FROM A WORKSHOP WITHSOMETHING NEW TO WORK ON

Eugene Charniak

“I am a member of the‘computational linguistics’

clan. Most of us would see asour ultimate goal getting acomputer to understand

language”

conduit! 2

the differences between “a” and “the” canbe negligible. However, in the context“They appealed to a/the Supreme Court”we would have no trouble. The idea thenis that one can combine the language-model probability, which tells us how wellthe words fit together, with the acousticmodel, which tells us how well the acous-tic signal supports the hypothesizedwords, in order to get what one hopes isthe correct overall interpretation.

For many years now the speech-recognition community has been usingthe so-called “trigram” language model.In this model one estimates the probabil-ity of a word coming next in the stringbased on the two previous words. So inthe above example we would ask what theprobability is of seeing “a” (or “the”) after

“appealed to” and whatthe probability of “Su-preme” after either “to a”or “to the”. Presumablythe probability of seeing“Supreme” after “the”would be higher and thecorrect decision would bemade. Creating such alanguage model is tricky,but the basic idea is thatone collects a largeamount of text (or evenbetter, a large corpus oftranscribed speech) andgathers the statistics.

The trigram modelworks remarkably well.Indeed, there are con-stant rueful comments

from the computational linguistics com-munity about such a stupid techniqueworking so well, while so far “smarter”techniques have pretty much failed toimprove upon it. However, at this work-shop the speech-recognition people wereshowing the limitations they run upagainst with the trigram model. Some ofthe examples they put on the board were:

Too many American citizens failto vote. (vs.)

To many American citizens votingis a chore.

Put the paper in/and the file.

need. As I left for the workshop, I washoping to pick up some tips I could applyto my research.

The third group was the speech-recognition community. Formally, I sup-pose, one might put them in the computa-tional linguistics community, since theytoo are interested in getting computers tounderstand language, in their case spo-ken language. Practically, however, only asmall fraction of the communities over-lap. There are two reasons for this. First,starting about twenty years ago, thespeech-recognition community adoptedstatistical methods for their research,while the influence of these techniqueswithin the rest of the computational lin-guistics community has been more recent.I started using statistics about nine yearsago, and I was a compar-atively early adopter.The second reason isthat many of the con-cerns of the speech-recognition community,particularly the intrica-cies of how the humanvoice makes words, areunique to them.

There is, however, oneparticular point of po-tential overlap. All mod-ern-day speech-recogni-tion systems have a“language model”—aprogram that assigns toany string of words theprobability of that stringin the language. Thus “Iwent to the store” would have a relativelyhigh probability, while that of the string“store the to went I” would be very muchlower. Such language models are neededbecause often it is very difficult to decidewhat was said without knowing the con-text. Or, to put it another way, if yousplice out a single word from a tape andask someone to tell you what the word is,people cannot do very well, whereas withthe complete sentence there would be nodifficulty at all. The individual words helpsupport one another because we implic-itly know which combinations are reason-able and which are not. For example, innormal speech, when people are not pay-ing particular attention to enunciation,

The trigram modelworks remarkably

well. Indeed, thereare constant ruefulcomments from thecomputational lin-guistics communityabout such a stupidtechnique working

so well”

conduit! 3

and so on. The parser (in principle) findsall possible parses for the sentence andadds up their probabilities, and the resultis the probability for the sentence. Inpractice these parsers find only a subsetof the parses, and instead rely on tech-niques that find high-probability parsesearly on, so that they can ignore the verylow-probability interpretations for a sen-tence. Because natural languages are soflexible, there are zillions of parses formoderate-length (20-word) sentences, butalmost all of them make no sense andthus are given very low probabilities, sotheir contribution to the sum isnegligible.

I am not the first person to think of usinga statistical parser in this fashion. Thereare two very good pieces of previous workin this area, one by colleagues at JohnsHopkins (where I spent my sabbaticalthree years ago), and more recently byBrian Roark, a graduate student here atBrown in Cognitive and Linguistic Sci-ences. In both cases they found that theirlanguage model based upon a statisticalparser performed slightly better than thetrigram model.

Naturally I had known about this workprior to this workshop, and I also sus-pected that, since my parser was moreaccurate at parsing than either of theother two, there was a reasonable chancethat it would perform better as a lan-guage model as well. Yet somehow, with-out the immediate contact with people towhom this was a pressing issue, it did notseem an important thing for me to workon, particularly since, for reasons I do notwant to get into here, my parser wouldrequire significant revisions to make itapplicable to this problem.

In the first pair of sentences, the firstword that can help decide between “too”and “to” occurs four words later, and thedifferences in the acoustic signal between“too” and “to” are, to put it mildly, subtle.(The words “too” and “to”, of course, arepronounced exactly the same. However, ifyou try saying the sentences out loud youwill find that the rhythm you use will bedifferent—it is the difference between“TOO many American citizens ...” and “toMANY American citizens ...”. But currentspeech systems are nowhere near sophis-ticated enough to notice this.) In the thirdsentence all the three word combinations(which is all the trigram model ever looksat) are perfectly fine, i.e., “the paper in/and”, “paper in/and the” “in/and the file”.

In these cases, however, we would nor-mally think of the wrong word as makingthe sentence ungrammatical, and thus aprogram that looked at sentences interms of grammar might have a goodchance of picking the correct word. Moreformally, most statistical parsers work byassigning a probability to each sentence/parse combination. By a “parse” I meansome structure that elucidates the syn-tactic structure of sentences. It is likewhat I did in grammar school when I wastold to draw a line between the “subject”and the “predicate” of a sentence (e.g.,between “The dog” and “ate the steak”)and then to draw a half-line between theverb and the direct object (“ate” and “thesteak”). A parser does the same thing butin more detail, and the results areexpressed in a tree structure. So, forexample, the parses for the two sentences“Put the paper in/and the file” are shownbelow.

Here “vp” stands for “verb phrase” (some-thing like “predicate”), “np” stands for“noun phrase”, “prep” for “preposition”,vp

verb np pp

put the paper in the file

prep np

vp

verb

np

np

put the paper and the file

conj np

conduit! 4

sider the noun phrase “the paper” in oursample sentence. The parser would ask,what is the probability that a nounphrase appearing as the direct object ofthe verb “put” would have as its mainnoun “paper”? What is the probabilitythat, if the noun phrase has the mainnoun “paper”, it will have the form “deter-miner noun”? How likely is “the” to be adeterminer modifying “paper”? and so on.Of course, none of these probabilities aregoing to differ between the two parses,since the noun phrase “the paper”appears in both of them.

If you think about the two sentences youwill soon realize that the intuitive reasonthat “in the file” sounds better than “andthe file” is that the verb “put” almostalways requires a prepositional phraseafter the noun phrase to indicate theplace where the thing should be put. Thefirst of the two parses has this preposi-tional phrase because in this sentence thehypothesized word is “in”, which is apreposition, whereas the second parsedoes not because the alternative is “and”,which is a conjunction. Thus in the sec-ond sentence the parser hypothesizesthat there is a conjoined noun phrase “thepaper and the file”. If we put this in

Max Salvas installs a cooling fan in one of the PCs he’sassembling from commodity components. Says Director of

Computer Facilities Jeff Coady,“Max is amazing at building or fix-ing systems. He is fearless when itcomes to replacing boards and hehasn’t fried one yet! There isn’tmuch electronic or mechanicalthat he can’t decipher, take apart,repair and put back together—from a toaster-oven to a main-frame computer.” This project is acost-effective way to supply thedepartment with PCs that can beupgraded quickly and therebymore closely track enhancementsin component technology. To iden-tify Max’s completed machines, aspecial ‘Max Built’ logo (above)has been designed.

Let me now explain how statistical pars-ers work so you have some idea of howthey might assign reasonable probabili-ties in the above cases. All statisticalmodels of complicated real-world phe-nomena must make what we call “inde-pendence assumptions.” So the trigrammodel assumes that the probability of aword depends only on the last two wordsand is independent of all of the remainingprevious words. In current parsing mod-els we first need to assume that the prob-ability of a sentence is independent of thesentences that came before. We also makemore complicated assumptions aboutwhich words can be ignored in computingvarious probabilities. Naturally, for boththe trigram model and our parser, theassumptions are wrong. However, wehave to make them because otherwisethere would be too many different situa-tions for us to gather reasonable statisticson. The goal in statistical language workis to make the required independenceassumptions as natural as possible.

Statistical parsers assign probabilities toparticular parses by multiplying together

individual probabilities foreverything that goes intothe parse. For example, con-

Max Built logo by RI cartoonist Rick Billings

conduit! 5

terms of the probabilities that my parserconsiders, it comes down to this: what isthe probability that a verb-phrase whosemain verb is “put” will end with a noun-phrase, or instead will have a preposi-tional phrase following the noun phrase?My parser gathers statistics like thatfrom a large “training corpus” of about amillion words of human-parsed newspa-per text (from the Wall Street Journal).According to its statistics, the probabilityof ending with a noun phrase is 0.029,whereas the probability of a followingprepositional phrase is 0.8, almost thirtytimes larger. (If you are wondering how itis that even 3% of the verb phrases do nothave a prepositional phrase, it is becauseanother use of “put” does not use the pp:“No matter how you put it, it sounds

fishy.”) Thus, as you might expect, thesentence with “in” is given a much higherprobability than the sentence with “and”.Naturally, the parser never saw this par-ticular sentence in its training examples,or even a sentence where the direct objectof “put” was “the paper”. However, whenbuilding the parser I assumed that theprobability of seeing a prepositionalphrase in a verb phrase, while dependenton the main verb of the verb phrase, wasindependent of all of the other words.

So, you ask, how well does my parser do?The short answer is very well, thank you.To get more precise I need to tell yousomething about how things are mea-sured in this field.

Intuitively, we want a language model toassign high probability to sentences thatare likely to occur and very low probabil-ity to sentences that are bad for some rea-son and thus unlikely to occur. One wayto do this is to take a corpus of Englishthat you assume is reasonably represen-tative and see what probability the model

assigns to the corpus. That is, take eachof the sentence probabilities, multiplythem together; the higher the result is,the better your language model. (Noticethat the we do not need a corpus of “bad”sentences to check that they get low prob-ability. If all the good ones get high proba-bility, then the bad ones must be low,since the total over all sentences mustsum to one.)

This is, in essence, exactly what we do,except that because the resulting num-bers come out so small we modify the pro-cedure first to take the nth root of theoverall probability, where n is the numberof words in the corpus. The idea is thatsince probabilities are typically combinedby multiplying them together, the nth

root gives us the averageprobability for eachword. Finally, we takeone divided by this prob-ability. So if the averageprobability for a word is,say, .005, then we reportthe number 200. Oneway to think of thisnumber is that, on aver-age, guessing the nextword is as difficult asguessing which of 200doors has the prize

behind it. The resulting number is calledthe “perplexity” of the corpus according toyour language model. The smaller theperplexity, the better the language model,since it is making plausible words moreprobable, thus is assigning higher proba-bility to good sentences, and thus willgive better guidance to the speech-recog-nition system when it tries to decidebetween words that the acoustic systemfinds confusing.

In the experiments I took the trainingcorpus that had been used by priorresearchers and trained (collected statis-tics for) both a trigram model and myparser. I then ran my parser over thesame test corpus that had been used inearlier research and computed the per-plexity. The trigram perplexity for bothmy trigram model and those in the earlierpapers I mentioned above was about 167.The grammar perplexities for the twoprevious grammar models were 158 (forthe Johns Hopkins researchers) and 152(for the paper by Brian Roark here at

”The trigram perplexity for both my trigrammodel and those in the earlier papers...wasabout 167. The grammar perplexities for thetwo previous grammar models were 158...and 152... . My statistical parser had a per-plexity of 131, which suggests that it should

make a very good language model”

conduit! 6

Brown). My statistical parser had a per-plexity of 131, which suggests that itshould make a very good language modelindeed.

However, do not expect to see my parserincorporated into your favorite speech-recognition system soon. Several prob-lems deserve mention. The first is thatparsing is quite time-/space-consuming,so machines will have to speed up a goodbit before parsers are practical for every-day speech recognition. Second, remem-ber that current statistical parses need atraining corpus from which to gathertheir required statistics. If the way youuse words is quite different from thetraining corpus, the language modelmight not work very well for you. Or thenagain, maybe this will not be a problem.In fact, we simply don’t know yet.

Finally, there is one problem that isunique to my parser and is tied up withwhy, I believe, it was able to outperformprevious attempts. You may rememberthat in the “Put the paper in/and the file”

example I noted that verb phrases headedby the verb “put” almost always require aprepositional phrase. More generally, myparser (and all the best statistical pars-ers) works on the principle that it shouldalways condition probabilities on thehead word of the phrase that the programis working on. This creates problems forspeech recognition, however, in thatspeech systems like to work from left toright, following the time sequence of thewords coming in. Unfortunately, some-times the head word of a phrase comes inthe middle or end of the phrase—nounphrases are the most common example,where usually the head noun is the right-most (e.g., “key” in “the small room key”).Thus the earlier language parsing modelsdid not use this principle, and I believesuffered because of it. There are severalways this slight incompatibility with thedictates of speech recognition can be over-come, and I am hoping that my parser’sgood performance will encourage researchin this area.

On November 2, 2000, we held our26th Industrial Partners ProgramSymposium on “Web Technologies,”featuring speakers from six leadingcompanies in the Web industry. Thetalks covered a variety of Web-relatedtechnology issues ranging fromsearch engines and user interfaces tosecurity and content delivery.

The first speaker was PrabhakarRaghavan, Chief Scientist andVP for Emerging Technologies atVerity, Inc. He is better known tomany of our students as the coauthorof a theory textbook on randomized

algorithms used in CS155. Prabhakarwas a researcher and manager in IBM’sResearch Division and only recentlymoved to an executive job at Verity, asmall company specializing in softwarefor intranet applications (intranets areInternet networks connecting users andservers inside a company or an organiza-

tion). His talk on “Networks and Sub-Net-works in the World Wide Web” discussedseveral structural properties of graphsarising on the Web, including the graph ofhyperlinks and the graph induced by con-nections among distributed search ser-vants. He presented a number of algor-ithmic investigations of the structure ofthese networks, and concluded by propos-ing stochastic models for the evolution ofthese networks.

The second speaker was Tom Leighton,Chief Scientist, Akamai Technolo-gies. Tom was one of the two founders ofAkamai, one of the most successful Web-related startups in recent years. Hisbackground is also in theory: when not in-volved in Web business, Tom is a profes-sor in MIT’s applied mathematicsdepartment and the head of the algo-rithms group at MIT’s Laboratory forComputer Science. Akamai’s business isto provide fast access to the content of itscustomers’ Web sites. Although users maynot notice this, when accessing popularsites such as Yahoo! or CNN.com the con-tent is actually delivered to the user fromAkamai’s servers, not from the originalsites. Tom gave an overview of the

THE 26th IPP SYMPOSIUM:‘WEB TECHNOLOGIES’

Host Eli Upfal

conduit! 7

was Stephen Uhler, Researcher, WebApplications Technologies, Sun Mi-crosystems. Steve described the “BrazilProject” on tools to integrate applicationsand data across multiple platforms on theInternet. Brazil provides components forgenerating content, management of re-quests, content transformation with toolssuch as TCL, Python and a scripting lan-guage, handling special-purpose devicesand services, such as SmartCards and Ac-tiveX/Com connectivity, and utilities,such as regular expression processors andproxy services. Brazil’s components couldbe used to build applications involvingthe collection, filtering, and processing ofdata from multiple sources in potentiallydifferent formats and the reformulationand display of the data on Web pages.

The next speaker was Olin Sibert, VPStrategic Technologies, InterTrustTechnologies, who talked about “Digi-tal Rights Management in the Era ofNapster.” The modern digital world, inwhich computation and communicationare (almost) free and (almost) unlimited,poses critical new challenges for manage-ment of rights and information. For ex-ample, there is much hullabaloo todayabout “the end of intellectual property”(postulating a world where information iscompletely uncontrolled) and “the end of

complex technology involved in fast con-tent delivery and discussed some inter-esting research questions related tofurther improvement of the process. Itwas also fascinating to hear a first-handaccount of the two years that moved anabstract idea in theory to a successfulmultimillion-dollar company.

The last morning speaker was TedTracy, Vice President for Product De-velopment at Latitude Communica-tions, who talked about “Web Collab-oration” and gave an overview of Lati-tude’s main business, “Web-conferencing”technologies, i.e., tools and applicationsfor group collaboration across Web net-works. Web conferencing allows multipleremote participants to exchange voice,data, and video information across IP net-works for general-purpose team collabo-ration as well as specific vertical ap-plications. As the Internet continues toimprove in terms of both total bandwidthand “quality of service” mechanisms,Webconferencing will become even more per-vasive. Ted described Latitude’s flagshipMeetingPlace, which provides a powerfulsolution to the need for improved profes-sional worker productivity and associatedcollaboration.

Following the usual excellent lunch andfancy cakes, the first afternoon speaker

Symposium speakers from l to r: Ted Tracy, Latitude Com-munications; Tom Leighton, Akamai Technologies, Inc.;Andrei Broder, Alta Vista Company; Stephen Uhler, Sun

Microsystems; Eli Upfal, Brown; Prabhakar Raghavan, Verity,Inc.; Olin Sibert, InterTrust Technologies

conduit! 8

fair use” (the opposite world, where infor-mation is tightly locked up). These andsimilar apocalyptic visions are inspired byan absolutist interpretation of varioustechnologies; in reality, the picture is byno means this simple and clear-cut. Thetalk discussed the concepts and mecha-nisms of Digital Rights Management(DRM), InterTrust’s solution to securingcontent and transactions, and how it canact as a moderating factor in such visions.

The last talk of the day was by AndreiBroder, VP Research and Chief Sci-entist, Alta Vista Company, who talkedabout “Trends in Search Technology.” An-drei was a researcher at Digital’s (laterCompaq’s) SRC Lab in Palo Alto, workingon theory of algorithms and in particularrandomized algorithms and analysis. SRCwas also where the search engine AltaVista originated. Andrei was fascinatedby the complex algorithmic questions re-lated to this project and when Alta Vista

spun off from Compaq, Andrei joined asthe leader of research and development.The challenge of the Web search industry,as described by Andrei, is to “meet the us-ers’ needs, given their poorly made que-ries and the heterogeneity of Web pages.”Search technology is ubiquitous on theWeb: from the major search engines thatindex hundreds of millions of pages to thetiniest e-commerce site, there is a searchbox on every site. These boxes are pow-ered by a vast array of methods of varyingsophistication, combining classic informa-tion-retrieval and linguistics techniqueswith Web-specific data and algorithms.On the other hand, users increasingly ex-pect and actually receive a substantiallyuniform interaction style—basically un-structured, full-text search—no matterwhat search box they are using. The talkexplored some of the technology trendsand business developments that makethis search paradigm so prevalent andpowerful.

Peter Wegner, Professor Emeritus, received the ACM’s Distinguished Service Award ata ceremony in San Jose on March 11.

“For many years of generous service to ACM and thecomputing community, including outstanding andinspiring leadership in publications and in chartingresearch directions for computer science.”

During his distinguished research career Peter Wegnerhas written or edited over a dozen books in the areas ofprogramming languages and software engineering. Fordecades, he has been an initiator in ACM’s educationaland publication efforts, performing invaluable serviceto innumerable readers, researchers, practitioners andstudents. As editor-in-chief of ACM Press Books (1987-1992) and the ACM Computing Surveys (1995-1999),Dr. Wegner demonstrated innovative leadership andthe ability to inspire and motivate others. In both ofthese editorial positions he innovated and improvedthe publications substantially, reaching out with origi-nality, energy, and good taste.

Dr. Wegner has an exemplary history of service to thecomputing community and his efforts have inspiredseveral generations of computer scientists. He was aleader in charting research directions for CS who con-

tinually found new ways of focusing the field’s intellectual energy, helping to shaperesearch agendas and funding programs. His commitment to CS research and publish-ing was matched by his commitment to education, and his focus on the state and thedirection of education provided an immense contribution. At a time of splintering andspecialization in CS, Peter provided an integrative force to the computing community.

conduit! 9

these examples also illustrate some of thepotential benefits of further study. Whilethis approach is more open-ended than aperceptual psychology approach, both ap-proaches are worthy of pursuit, and thepotential benefits of using the less struc-tured approach outweigh any risk offailure.

How Humans See andUnderstand

Scientific visualization, a term coinedonly a little over ten years ago, is the pro-cess of using the human visual system toincrease our understanding of phenom-ena studied in various scientific disci-plines. While the term is young, theprocess (modulo the computer) has beenused since the beginning of science. Manyscientists have created drawings or built3D models to understand and communi-cate their science. The history of scienceand art can provide us with lessons wecan apply to using computers effectively.Over time, artists have developed tech-niques to create visual representationsfor particular communication goals. Arthistory provides a language for under-standing and communicating thatknowledge.

Historically, the two disciplines approachthe human visual system from differentperspectives. Art history provides a phe-nomenological view of art—painting Xevokes response Y. Art history, however,doesn’t deconstruct the perceptual andcognitive processes underlying re-sponses. Perceptual psychology, on theother hand, strives to explain how hu-mans understand those visual represen-tations. There’s a gap between art andperceptual psychology—we don’t knowhow humans combine visual inputs to ar-rive at the responses art evokes. Shape,shading, edges, color, texture, motion, andinteraction are all components of an in-teractive visualization. But how do thesecomponents interact and how can theymost effectively be deployed for a particu-lar scientific task? Answers to these ques-tions are likely to fill some of the gapbetween art and perceptual psychology.

Reproduced from IEEE Computer Graphicsand Applications, March/April 2001.

In the November/December 2000 issue ofIEEE Computer Graphics and Applica-tions,1 Vicki Interrante posed a visualiza-tion problem that she and I have been

interested in for several years.The problem is that of visuallyrepresenting a 2D field of datathat has multiple data values ateach point. For example, 2D fluidflow has a vector value at each lo-cation and derived values are of-ten available at each location.Interrante suggests using naturaltextures to attack this problem,because the textures can poten-tially encode lots of information.She provides some intriguing ex-amples and proposes a psychol-ogy-based approach for de-veloping an understanding of how

we perceive natural textures, like thosethat Brodatz2 photographed. Under-standing this can help us build better vi-sualizations.

Based on Interrante’s suggestions, Iwould like to posit and explore what is,perhaps, a less well-defined approach.Through evolution, the human visual sys-tem has developed the ability to processnatural textures. However, in addition tonatural textures, humans also visuallyprocess man-made textures—some of therichest and most compelling of which arein works of art. Art goes beyond what per-ceptual psychologists understand aboutvisual perception and there remain fun-damental lessons that we can learn fromart and art history and apply to our visu-alization problems.

The rest of this article describes and illus-trates some of the visualization lessonswe have learned studying art. I believe

1. V. Interrante, “Harnessing Natural Texturesfor Multivariate Visualization,” IEEE ComputerGraphics and Applications, vol. 20, no. 6, Nov./Dec. 2000, pp. 6-10.2. P. Brodatz, Textures: A Photographic Albumfor Artists and Designers, Dover, New York,1966.

LOOSE, ARTISTIC ‘TEXTURES’ FOR VISUALIZATION

David Laidlaw

conduit! 10

During these field trips, we have studied,in particular, the works of three painters.

•Van Gogh, whose large, expressive, dis-crete strokes carry meaning both individ-ually and collectively.

•Monet, whose smaller strokes are oftenmeaningless in isolation—the relation-ships among the strokes give them mean-ing, far more than for van Gogh.

•Cezanne, who combined strokes intocubist facets, playing with 3D perspectiveand time within his paintings more thaneither van Gogh or Monet. His layeringalso incorporates more atmospheric ef-fects. In a sense, his work shifts from sur-face rendering toward volume rendering.

The three artists’ work in this sequencebuilds in complexity and subtlety. In ourfield trips we studied all three, but mostof our experiments thus far are limited toideas we learned from van Gogh’s work.

Van Gogh introduced us to the concept ofunderpainting, or laying down a roughvalue sketch of the entire painting. Theunderpainting shows through the overly-ing detailed brush strokes to define theanatomy of the painting. Fig. 1b showsunderpainting for Fig. 1a. It divides thecanvas into two parts—a primed lower re-gion of hillside, rocks, and ground cover,and a darker upper region of tree, sky,and distant hills. Underpainting helpedus present some overall parts of our data.We found that an analogous underlying

As an example, the human-computer in-teraction (HCI) community is using andextending knowledge about perception totest and develop better user interfaces.We can find analogous inspiration for im-proved methods for scientific visualiza-tion in the gap between art and per-ceptual psychology. Many of these lessonswill impact the visual representation ofmultivalued data.

Looking Up from our Monitors

A number of times over the last few yearsI’ve shepherded my students to art muse-ums for guided tours by my artist collabo-rator davidkremers, the Caltech Disting-uished Conceptual Artist in Biology. Afterinitially searching for scientific visualiza-tion inspiration in art, these visits let usformulate a plan for finding and applyingthe concepts. Our initial focus was on oilpainting, particularly from the Impres-sionist period, because these paintingsare so visually rich. The multiple layers ofbrush strokes in the paintings provide anatural metaphor for constructing visual-izations from layers of synthetic ‘‘brushstrokes.’’ Some of my colleagues look atme askance when I describe our researchfield trips, as if to say, “This is research?”But stepping out of the lab helps studentsbuild a new picture of what they can ac-complish when they come back to thecomputer. It trains their eyes and mindsto see differently.

1(a) Van Gogh’s The Mulberry Tree (1889, oil on canvas) illustrates the visual shorthandthat van Gogh used with his expressive strokes. Multiple layers of strokes combine todefine regions of different ground cover, aspects of the hillside, and features of the

tree. (b) An underpainting shows the “anatomy” or composition of the scene in broadstrokes. (Image of The Mulberry Tree granted by the Norton Simon Foundation,

Pasadena, California. Gift of Norton Simon, 1976.)

(1a) (1b)

conduit! 11

and produced interesting results. Ournext attempts were more directly appliedto scientific problems. We show two of theimages we generated in Figs. 2 and 3. Theproblem-directed approach led us toiconic-looking strokes.

In Fig. 2, we show one 2D slice of a 3Dsecond-order tensor field, which hasabout six different data values at eachpoint in the image. The image shows theright half of a section through a mousespinal cord. To create the visualization.we used a layer resembling an under-painting with ellipse-shaped strokes ontop of it. On each of the strokes, a texturerepresents more of the data. For more de-tails on the scientific interpretation andthe visualization, see Laidlaw et al.3

In Fig. 3, we show 2D fluid velocity to-gether with a number of derived quanti-ties. About nine values are represented ateach spatial location in this visualization.We again used a layer resembling an un-derpainting with layers of ellipse, wedge,and box strokes on top. The ellipsestrokes have a subtle texture superim-posed. More details on the visualizationappear in Kirby et al.4

SpaceWe learned that paintings (and, in somecases, visualizations) are multiscale.They can be viewed from different dis-tances and seen and understood differ-ently. This raises interesting issues aboutthe definition of texture. Let’s considervan Gogh’s Mulberry Tree (Fig. 1a). Froma few inches away (look closely at Fig. 4b)you can see shapes from the bristles ofthe brush as well as colors mixed within astroke. At this distance, the shape andcolor features might be considered tex-ture, but they could also be interpretedindividually. At a distance of 18 inches(Fig. 4b at normal reading distance of 18

3. D.H. Laidlaw et al., “Visualizing DiffusionTensor Images of the Mouse Spinal Cord,” Proc.Visualization 98, IEEE Computer Soc. Press,Los Alamitos, Calif., 1998, pp. 127-134.4. R.M. Kirby, H. Marmanis, and D.H. Laidlaw,“Visualizing Multivalued Data from 2D Incom-pressible Flows Using Concepts from Painting,”Proc. Visualization 99, IEEE Computer Soc.Press, Los Alamitos, Calif., 1999, pp. 333-340.

form in our visualizations anchors and lit-erally gives shape to disparate data com-ponents. Outlines around regions provideseparation and emphasis and lend defini-tion to our sea of data.

In van Gogh’s The Mulberry Tree (1889,oil on canvas), brush strokes representthe solid trunk of the tree, bendingbranches, leaves blowing in the wind, andtufts of grass. We learned that there aremany shorthand ways of depicting com-plexity using icons, geometric shapes, ortextures that evoke a characteristic of thesubject or the data—and with that comesthe responsibility of choosing brushstrokes that don’t create opposing or un-wanted secondary impressions. Beyondthis direct representation, they also in-vite the viewer to experience the scene,not just view it passively. Similarly,brush-stroke size and proximity depictdensity, weight, and velocity. In our visu-alizations, we wanted to capture thismarriage between direct representationof independent data and overall intuitivefeeling of the data as a whole.

Back in the Lab

Returning to our computer lab, we triedto use some of the ideas we had gleaned,once again drawing mostly from vanGogh’s work. We experimented withbrush stroke shapes and ways of layeringthem. Our initial attempts were free-form

2 Visualization of half a section through amouse spinal cord. The data is a symmet-ric 3D second-order tensor field, with theequivalent of six independent scalar val-ues at each point. The detail on the rightshows the lower right part of the section

conduit! 12

scale than the strokes themselves, andthe dark box strokes of Fig. 3 are at a dif-ferent scale than the other strokes.

To take full advantage of the multiscalenature of paintings and visualizations wehave to have ways of interacting withthem—that is, ways of changing our view-ing distance. We rarely change the dis-tance from which we view our monitors—only a bit more frequently do we do sowith paper publications. That’s why thesame image is shown at different scalesin Fig. 4 and in some of the other detailedfigures. However, we do view imageshung on a wall from different distances.And some images on paper—often artis-tic—inspire that sort of study. Projectionsystems like PowerWalls and CAVEs maybe good options for encouraging this sortof exploration, as may other hangablelarge-format output media.

Time

We also learned that paintings (and, insome cases visualizations) have a tempo-ral component. For instance, we see dif-ferent aspects of an image at differentviewing times. Some parts stand outquickly, like the overall composition orpalette of a painting, and some take moretime to become apparent, like the textureor shape of individual strokes. The scaleand speed of recognition correlate, as docontrast and speed of recognition—butthese are not the only factors.

To use this lesson, we can design our visu-alizations so that important data featuresare mapped to quickly seen visual fea-tures. For example, features that we want

inches), these features appear smallerand resemble a texture on each stroke.The strokes themselves are still individ-ual. At a distance of five feet (Fig. 4a), thestrokes merge together to appear morelike a texture. Finally, at 15 feet (Fig. 1a),the strokes blend together and become al-most invisible individually.

We can use this lesson by encoding differ-ent information at different scales. Iconicinformation at one scale can turn into tex-ture information at another scale. Withcare, we can design features at differentscales into the same images. In the scien-tific visualizations of Figs. 2 and 3 we de-signed visual features at different scales.The texture on strokes is at a much finer

3 Visualizationof 2D fluid

velocitytogether with

several deriveddata values.

Approximatelynine values are

representedvisually at

each point inthe image

(4a) (4b)

4 Variances in viewing van Gogh’s MulberryTree. Viewed in this article from about 18

inches, Fig. 1a shows what you would see 15feet from the painting. Comparatively, Fig. 4shows the following: (a) a detail of what youwould see 5 feet from the painting, and (b) a

detail at actual size (what you would seefrom 18 inches). Look at (b) more closely for

viewing distances less than 18 inches

conduit! 13

to measure directly from an image arepresent for detailed study but don’t in-trude on the visualization’s initial im-pression. The multiscale examples fromthe “Space” section illustrate this tempo-ral impact. Fig. 3 gives another example:we can read the wedges more quicklythan the ellipses because of a differencein contrast.

Studies of preattentive vision and knowl-edge about low-level vision are useful fordesigning the quickly seen visualization.It’s more difficult to test the more slowlyseen parts, which makes it more difficultto design them. Task-oriented experi-mental tests seem logical, but the tasksare often so complex that the perfor-mance variance is relatively high, mak-ing methods difficult to compare.

Our Initial ExperimentsOur initial experiments were muchlooser than the examples shown in Figs.2 and 3. Some examples in Fig. 5 show2D or 3D fluid flow. Since I want to em-phasize the overall texture and visualqualities, I won’t go into detail about themappings for each. To many, the imagesare visually compelling, yet it has beendifficult to extract concrete visualizationlessons from them beyond those I previ-ously described. What people see in theseimages includes not only the mappingsthat were used for the data value, butalso other visual characteristics. Despitebeing 2D, some images give an overallsense of depth. Some of the strokes ap-pear to layer, like feathers or scales. Oneof our challenges with these looser imagesis in understanding what works, whatdoesn’t, and (we hope) why.

Closing ThoughtsI’ve tried to illustrate some examples oflooking toward art for inspiration in cre-

ating visualizations. Here we feature vanGogh and mention Monet and Cezannefor context. In your artistic searches,choose the artists in whom you have apassionate interest. I believe that anyartist has lessons to offer to visualization.

Working on scientific visualization prob-lems, we already interact with scientistsand adopt their problems. As toolsmiths,we do better computer science throughaddressing scientists’ problems on scien-tists’ terms.5 Similarly, we benefit fromcritical feedback from artists, despite the

difficulty of creating and maintainingthese relationships. I try to look at andunderstand art—early and often—andemulate it in scientific visualizations andget critical feedback from artists. I ex-plain what I’m trying to do visually andhave artists critique it. Then I iterate,iterate, and iterate.

Of course, the scientists must be involvedin this iterative process. Artists can helpwith inspiration and feedback on the vi-sual and communicative aspects of visual-ization, but scientists define the tasksperformed and therefore must ultimately

5. F.P. Brooks, “The Computer Scientist asToolsmith II,” Comm. ACM, vol. 39, no. 3, Mar.1996, pp. 61-68.

5 Loose texture examples

“we benefit from criticalfeedback from artists,despite the difficulty of

creating and maintainingthese relationships. I try tolook at and understand

art and emulate it...”

conduit! 14

bine perceptual psychology and art to fillin the gap in our understanding of howhumans see. By having artists cognitivelyanalyze what is shown by more complextextures, we might come to a consensuson what works, what doesn’t work, andwhy it does or doesn’t work in the contextof art and art history.

Acknowledgments

I’d like to thank my collaborators—in par-ticular, artist davidkremers—as well asClifford Elgin, Mike Kirby, and MatthewAvalos. I also want to thank Victoria In-terrante and Barbara Meier for their en-couragement and ideas. The NSF (CCR-9619649, CCR-0086065) and the HumanBrain Project (NIMH and NIDA) sup-ported some of the work described.

evaluate the success of the methods. Forinstance, the fluid flow example in Fig. 3may be aesthetically pleasing, but withoutexplanation—perhaps via a legend orkey—it’s not scientifically useful.

Fig. 3 displays as many as nine values ateach point of the image. With some re-search indicating that texture has roughlythree independent dimensions, the abilityto represent nine values is somewhat sur-prising—perhaps it’s due to combiningcolor with texture or layering textures atdifferent scales.

Texture is hard to define. Understandingblack and white natural textures like thephotographs in Brodatz is a good start,but we also need to look broadly. Task-oriented user testing may help, and per-haps we can use the critiques that arepart of artists’ training. This might com-

Numerous New England academic and industrial groups have interestsin programming languages and systems. This means many talks ofbroad interest take place at diverse locations and times, with no singlevenue where researchers in the area can gather to hear about recent re-sults and work in progress.

The New England Programming Languages and Systems (NEPLS)symposium series was founded to create a periodic regional venue. Itdraws direct inspiration from a similar series held in the greater NewJersey area, and from other such events. Its goal is to provide a some-

what informal venue for talks and demos, where emerging work can receivefeedback and younger researchers can hone their skills.

The first NEPLS event was held at Brown University on December 7, 2000 andwas organized and hosted by Shriram Krishnamurthi. The 60-odd attendees rep-resented universities, research labs and industry. Participants came from as farafield as SUNY Stony Brook, IBM’s T.J. Watson Lab, Sun Labs, Williams, RPIand Cornell. The talks ranged broadly, covering topics from call-graph construc-tion through mobile code to program transformation and novel semantics touch-ing on various programming paradigms along the way. The Lubrano conferenceroom and the atrium buzzed with the discussions of researchers and students.By all accounts, it was a successful inauguration.

Numerous institutions have since offered to host meetings. For more informationon NEPLS, see http://www.nepls.org/ (which is hosted at Brown).

CS HOSTS INAUGURALNEPLS SYMPOSIUM

conduit! 15

worked on, so we often ended up spendinga day or two learning the material beforestarting.”

Luke Ma ’03

“As Albert said, we both worked for Lati-tude Communications Inc. this past sum-mer. They were just starting to formalizetheir internship recruiting program/plansand whatnot so we were prototypes ofsorts. We were treated, work-wise, basi-cally like normal employees. Latitudewas extremely hospitable to us through-out our internship.

They gave us an advance on our salary tohelp us move out there and also resolvedsome basic logistical issues for us likehousing. Transportation we had to find onour own and we eventually had to resortto the much-doubted public transporta-tion system of light rails, buses, andtrains.

After we arrived and marveled at ourwonderfully oversized furnished apart-ments, we found the company, which wasabout a mile and a half up the street. Be-fore we found better transportation, wejust took a leisurely 20-minute stroll ev-ery morning to the company. Of course,being the new interns and just arriving atthe company, we would go in at 8 everymorning. That didn’t last terribly long.Our bosses, coworkers, administration,and the company in general really didn’tmind what sort of schedule we kept. We

To highlight Partners in our IndustrialPartners Program and to encouragestudents to do internships with them,we’re initiating a regular column inwhich we describe students’ work ex-periences with Partners. The WiCSgroup (Women in Computer Science)has compiled its own database of WiCSmembers’ internships, available athttp//www.cs.brown.edu/orgs/wics/itern.html.

LATITUDECOMMUNICATIONS

Albert Huang ’03“Last summer, I was a software engi-neering intern at Latitude Communica-tions along with Brown students MelissaCheng and my roommate Luke Ma. Wewere there for about 13 weeks. Since wewere the first official engineering in-terns, we were treated more or less likenormal employees. Work was a lot of fun,mirroring the media-described laid-backSilicon Valley atmosphere. Luke and Iworked together for the most part onWeb application development (SQL,VBasic [ASP], C++,) while Melissaworked on some Mac stuff. We didn’thave prior knowledge of most of what we

SPOTLIGHT ON INTERNSHIPS WITHCS INDUSTRIAL PARTNERS

Department of Computer Science Industrial Partners

conduit! 16

would arrive at work anywhere from 8 to11 and work until quite late. It was defi-nitely more than eight hours of work perday, but most of the time it was becausewe wanted to work. The freedom inschedule also allowed us to consolidatework and play time. We would workcrazy days (on the order of 16 hours) for awhile and then take a three-day weekendand learn to sail in Santa Cruz. Thatparticular trip started on a Friday, andthe night before I had basically not sleptand pulled an all-nighter to get a demoup and running. Wonderfully tiring butfun nonetheless.

Our coworkers were also a blast to workwith and be around. Albert, Melissa andI were all part of the “Engineering In-terns” group. We ended up doing lots ofWeb development work using ASP andSQL, also some work on win32 applica-tions and NT services. We basically wentin every day and wrote lots of code. Thegreat thing about it all was that theprojects we were working on had definitescheduled shipping dates, so we knew wewere working on a viable product—thatdoes wonders for motivation. Interest-ingly enough, most of what we ended updoing over the summer (so far as re-quired skills are concerned) we did notknow before we got to Latitude. Our bossjust tossed a project at us and told uswhat he thought about it and how he’dlike to approach it. We would then go and

research all the technologies needed orcall upon the tremendous knowledge basepresent in our coworkers. Once we knewwhat we needed to know, we got to work.All in all it was a tremendously educa-tional process made that much better be-cause we had wonderful motivation.While we did our work (say from 10am to11pm), we’d play Starcraft for a breatherin between and go downstairs to the con-ference room and watch a movie on a pro-jector afterwards.

Eventually, we all had to say our good-byes and run away, not before our bosstried to get us to skip college and just staythere though! Apparently, he was pre-pared for the worst when he heard aboutus, and we turned out to do some decentwork. They even had a big party for us to-ward the end of the summer (took us tosee an IMAX film, got some food, andgave us some presents....Albert and I gota scooter!). A good many times we felt al-most guilty about how nice they were be-ing to us, our being lowly interns and all,which of course was another great moti-vation to work. But seriously, the Lati-tude attitude is one of the best I’ve seenfor workplaces. It’s never slow, but man-ages to have a sense of relaxed purpose-fulness at the same time.

This past winter break I went back toLatitude to work for a short time. Forabout three weeks in January, I was backin California with my old boss working on

l to r: LukeMa ’03,Melissa

Cheng ’01,Albert

Huang ’03

conduit! 17

DILIP BARMAN, ScM ’92I just read on the department home pageabout Jak Kirman’s death, and wanted toconvey my condolences and share my sor-row with the Brown CS community. Be-fore I came to Brown, I was consideringseveral graduate programs, and Jak (andKathleen, whom he had just started dat-ing) and his roommate Moi helped tomake me feel at home and hosted me intheir welcoming apartment for severalvisits. Jak was very friendly and a greathelp to me always, whenever I neededhelp with getting acclimated to the de-partment or Providence, or with emacswizard questions. I’m sorry I didn’t stayin touch with Jak in recent years, and thenews saddens me as we’ve lost a kindlygentleman scholar.barman@cs.unc.edu

CASSIDY CURTIS, Math ScB’92To introduce a particularly fascinatingwebsite, math major Cassidy Curtis ’92will henceforth be considered an honoraryCS major—besides, his stint as a UTA forCS123 and a slew of graphics work con-firm that he has paid his dues.

We learned of the telestereoscope (“Eye-stilts”) and Cassidy’s installation atBurning Man via graphics faculty andcouldn’t resist including a squib about it.

The Telestereoscope is an optical devicethat increases the distance between youreyes. This has the effect of enhancing

your perception of depth. For some peo-ple, it makes the world seem miniatur-ized: cars become toys, and landscapeslook like model train sets. For others, theenvironment deepens and splits intomany distinct planes. One viewer de-scribed it as “like looking at the realworld through a Viewmaster.”

The device itself is very simple. It con-sists of two periscopes, one for each eye.Each periscope is made of a pair of mir-rors. The periscopes shift the eyes up-wards and out to either side. Theprinciple behind it is based on human vi-sual perception. In order to make sense ofyour visual surroundings, part of whatyour brain must do is estimate how faraway things are. One of the ways yourbrain does this is by using the relativedisparity between the images projectedonto the retinas of your two eyes. Eachobject in your field of view will project toa slightly different location on each ret-ina. Essentially, the closer the object, thegreater that difference will be. Your brainalready knows how far apart your eyesare (about two or three inches), and usingthat information it can make a good esti-mate of exactly how far away each objectis. This is also the principle behind 3-Dmovies and stereograms. Now, if you arti-ficially exaggerate this relative disparityby placing your eyes several feet apart,your brain, still believing that your eyesare only two inches apart, will come to

a project to add some load-balancing/server-farming features to an existingdata-conference engine. Again, I learnedmore in those three weeks than I wouldhave ever expected (lots of C++ code,T120 conferences, server farmingschemes, funky stuff). And again, every-one was very friendly.”

Melissa Cheng ’01

“I worked on the web interface to Lati-tude MeetingPlace hardware, fixing bugsin the Mac/Solaris versions and adding a

main feature. I had a great time workingat Latitude. The project was fun, I got in-volved right away, and my code went intothe final product, which shipped a fewmonths after I left. Moreover, the com-pany culture is ideal—especially for in-terns who are new to living in the area—with laid-back, smart, and friendly peo-ple. So if you’re willing to ask questionsand either have a car or lots of patiencewith public transportation, you’ll loveworking at Latitude.”

COMMUNICATIONS FROMALUMNI AND FRIENDS

conduit! 18

Our research in this area spans an entirespectrum of conversational systems tech-nologies, including multimodal dialoguemanagement, natural language genera-tion, and statistical natural language un-derstanding. Our goal is a flexible, free-flow conversational system that placesthe user in the driver’s seat. Simply byspeaking naturally in their own words,users will control the application in theirown personal style. We are exploring sev-eral techniques to construct both univer-sal and application-specific dialogueengines. The emphasis is on scalable dia-logue systems capable of simultaneouslyhandling many users, tasks and inputmodalities (voice, keyboard, gesture andmouse). We have developed prototype sys-tems for many applications, ranging fromstock and mutual fund trading systems tophone banking, air travel reservations,and Web-based shopping, in many lan-guages, including English, French, Ger-man, Mandarin and Spanish. Regards!niyuge@us.ibm.com

KEN HERNDON, ScM ’96I’ve got my own consulting company, aone-man shop at present, called Arpali.This has been active for a little more thantwo years since I left Sony in NYC. I’vebeen working on various projects, gener-ally with a strong user-interface compo-nent. Donna (Miele, AB AmCiv ’92)graduated from Law School last May, andpassed the NY and NJ bar exams last fall.She’s going to be in private practice work-ing with me and with the family real es-tate business. We’ve got three children,Armand (6), Paul (4) and Lionel (3), andare expecting a fourth in August.ken@yellahouse.com

BENOÎT HUDSON, ScB ’99I’ve been working at NASA Ames sincegraduating almost two years ago. I’velearned plenty here about AI, symbolicarithmetic, software engineering, space-craft, and the national budget, to name afew. In the fall I’ll be going to CMU tostart a Ph.D. in the algorithms program.To get in touch, send me email: bh@tech-house.brown.edu.

the wrong conclusion about how far awaythings are. This incorrect informationabout depth then interacts with every-thing else you know about what you’reseeing (that is a tree, that is a car, etc.)and you begin to draw strange conclu-sions about the size of things. In short, be-cause it’s hard to believe that your head is

really ten feet wide, you are forced to con-clude that the world around you is reallysmall.

Cassidy Curtis and Chris Whitney tookthe Eyestilts to last summer’s BurningMan Festival in the Black Rock Desert,Nevada. Burning Man is an annual exper-iment in temporary community dedicatedto radical self-expression and self-relianceheld on the 400-square-mile playa, ter-rain that looks like another planet. Nowthat we’ve piqued your interest, you canenjoy the website (and read up on Burn-ing Man too) at http://eyestilts.com. Ifyou’d like to check out some of Cassidy’sgraphics research and other projects, go tohttp://www.cs.washington.edu/homes/cassidy.

NIYU GE, PhD ’00I am now working in the conversationalmachine group of IBM natural languageprocessing research. The following is abrief summary of what we do (this infor-mation is also available on the Watson re-search website):

Participants at theBurning Man Festivalline up to try the Eyestilts. Photo, Cassidy Curtis

conduit! 19

HAGIT SHATKAY, PhD ’99After two years as a post-doc in the Na-tional Center for Biotechnology Informa-tion at the NIH, where I worked on usinginformation retrieval for gene analysis, Ijoined the Informatics Research group atCelera Genomics. For those who have fol-lowed the human genome sequencingrace, Celera is also known as “the privateeffort.” It is a very exciting place rightnow. The genome sequencing itselfamounts to converting a large book fromone media format (human cell) to another(computer disk), without looking closelyat the actual contents. Given this se-quence, the current challenge is to findthe genes and their products and, moreimportantly, to understand what they aredoing. This is analogous to actually un-derstanding the semantic contents of thebook; needless to say, it is a hard prob-lem. We are working on developing com-putational techniques, utilizing a multi-tude of data sources, to find out moreabout the discovered genes and their ef-fects. There’s a lot of room for interestingmachine-learning research, and I amlooking forward to working on theseprojects.

Suzi has specifically asked for an updateabout Eadoh and Ruth, so here it is.Eadoh, our seven-year-old son, enjoys lifeas a second grader. He has started play-ing the piano, and he and I have alreadyperformed a duet in his first recital. Healso swims a lot and, defying my genes,enjoys playing baseball and basketball.He does still miss the New Englandsnow; today was another day in which wewere promised a foot of snow but only adisappointing drizzle materialized.Ruth is now three years old and growingfast. She attends preschool and, likeEadoh, enjoys swimming and playingwith all of HIS friends. We are living in avery pleasant neighborhood and Eadohhas made lots of new friends, but stilllikes visiting his RI classmates. This pro-vides us with a good excuse to take I-95North and come back for ice cream atMaximilian’s. It was a real pleasure tocombine our RI trip this last Octoberwith a visit and a talk at the department!Will be looking forward to futureconduit!s and good departmental news.Best wishes!hagit.shatkay@celera.com

We were delighted to have Ken Herndon ScM ’96 and hisfamily visit the department at the end of February. l to r:

Paul, Ken, Donna ’92, Armand and Lionel

conduit! 20

JACK STANKOVIC, PhD ’79Andy van Dam suggested that I sendsome info about an award I won for inclu-sion in the next conduit! issue.Professor Jack Stankovic, the BP AmericaProfessor and Chair of the CS departmentat the University of Virginia, has receivedthe second-ever IEEE Real-Time Systemsaward for “Outstanding Technical

Contributions and Leadership in Real-Time Computing.” This award was pre-sented at the Real-Time Systems mainconference in Orlando in December 2000and comes with a $1500 cash award.Jack has also been re-elected to a three-year term on the Computing ResearchAssociation Board of Directors.stankovic@cs.virginia.edu

Michael Black. Michael received a giftfrom the Xerox Foundation’s UniversityAffairs Committee to support research onhuman motion estimation in video se-quences. He was also a Program Commit-tee member for the IEEE Workshop onHuman Motion held in Austin in Decem-ber. Also in December he had a paper atNeural Information Processing Systems(NIPS’2000) with colleagues from Stan-ford and Sweden on learning and trackingcyclic human motions. In November hegave an invited talk at the Robotics Insti-tute at Carnegie Mellon University on thestochastic tracking of humans.

John Hughes. Spike has been chosenas the papers chair for SIGGRAPH 2002in San Antonio. He went to the first ofmany planning meetings in mid-January;if you try to reach him in the next 18months and he’s not around...it’s probablybecause of SIGGRAPH. In the meantime,start buying cowboy gear and bookingyour tickets to San Antonio...

Shriram Krishnamurthi. Shriram’stextbook, How to Design Programs, co-authored with Matthias Felleisen, RobertBruce Findler and Matthew Flatt, waspublished by the MIT Press in February.The book continues to be distributed on-line at http://www.htdp.org/. He orga-nized the New England ProgrammingLanguages and Systems symposiumseries by hosting its inaugural event in

the department on December 7, 2000 (p.14.), and he was on the program commit-tee for the Third International Sympo-sium on Practical Aspects of DeclarativeLanguages, which took place in March.

Franco Preparata. As a member ofthe Scientific Advisory Committee forMathematics and Computing at ArgonneNational Laboratory, Franco participatedin the review of their technical program.He is also a continuing member of theGödel Prize Committee. In December hewas invited by the Academia Sinica tovisit their institutes in Taipei. He hasbeen invited to deliver a plenary lectureon his recent work in Computational Biol-ogy by the Institute of Mathematical Sta-tistics at their annual meeting inCharlotte, N.C.

John Savage. John is in his first yearas an officer of the Faculty. He serves asVice Chair this year and will move on toChair and Past Chair in subsequentyears. The officers, who are also membersof the Faculty Executive Committee (acentral steering committee for Facultybusiness), are very busy people. Each hasa dozen or more scheduled meetings onfaculty business per month. John traveledto Japan in late February to give talks atKyoto and Sendai Universities.

▼▼▼

fac.activities@cs.brown.edu

▼▼▼

▼▼▼

▼▼▼

▼▼▼

conduit! 21

Eli Upfal.Eli was appointed associateeditor of the SIAM Journal on Com-puting. He served as the chair of theACM Doctoral Dissertation Award com-mittee and had a paper on modelingthe Web in the 41st Annual Symposiumon Foundations of Computer Science inCalifornia in November 2000.

Andries van Dam. Andy gave the key-note at a workshop sponsored by theLearning Federation, a non-profit founda-tion that is exploring the requirementsfor a grand challenge project in e-learningresearch for post-secondary science,mathematics, engineering and technology.The forty attendees at the workshop in-cluded some of the most prominent lead-ers in the field of educational research,such as Hal Abelson, John Seely Brown,Ed Lazowska, Don Norman, and RajReddy.

You may remember my men-tioning the five-foot-tall ver-sions of Mr. Potato Head thatpopped up around town awhile ago. Each one, it turnsout, was designed and spon-sored by a private organiza-tion. So Fleet Bank did theMr. Potato ATM that weshowed in the last conduit!Well, if you were not able tocome to Providence to seethem, you can now have apotato head come to you. Thepotato heads are comingdown and at least some ofthe sponsoring companiesare selling them on e-Bay. Soif you hurry you can have

one of your very own. The ‘Spud of Steel’(aka Mr. Potato Construction Worker),created for the Gilbane ConstructionCompany, was just sold for $1500. Ms. Po-tato Bishop is on the auction block evenas I write. I saw the Episcopal Bishopplugging her potato incarnation on a TVnewscast a few days ago. She was quitecharming in the role.

Just about the time our last conduit! wasbeing printed our esteemed chair, TomDean, sent around the following e-mail(here slightly abridged):

From: Tom Deantld@cs.brown.edu>Date: Tue, 14Nov 2000 13:24:02--0500(EST)Subject: just for fun:

missing pants--a story for adrab day

As some of you may know, Jo and Iare in midst of renovation pur-gatory and have been since latesummer. We just completed thecarpentry on the new stairs thispast weekend—-mahogany newelposts and balusters, oak treads,lots of trim work—-and we wereusing the evenings to sand,stain and finish the woodwork.

OK, we remembered to bring ourclothes downstairs before apply-ing the varnish to the stairtreads-—our bedroom is on thetop level of the house and thenew stairs are the only access.We all remember the lesson ofthe cartoon character who paintshimself into a corner. I didcarry a hastily chosen pile ofclothes downstairs but somehowmy pants slipped out of the pileand when it was time to leavethere were no pants available onthe accessible levels of thehouse. Fine, I had been wearingsome ragged paint-stained sweatpants and I had some "emergency"pants at the office.

I decided to take a swim beforeheading to the office. I felt alittle funny walking past Char-ley at the Smith Swim Centerwearing a suit coat and sweatpants, but Charley’s used tosuch things. I take my swim, re-turn to the locker room andfind... my sweat pants are gone.

▼▼▼

CHARNIAK UNPLUGGED

EugeneCharniak

conduit! 22

I look everywhere but no luck; Ilook in the lost-and-found forsome pants... still no luck.Maybe someone took them by mis-take or the custodian threw themaway thinking that they wererags. Charley manages to find mea pair of baggy swimming shortsin the custodial rag bag, how-ever, and since they are lessrevealing than my Speedo swimsuit I put them on.

That’s it; end of story. Coinci-dence? I don’t think so. As Iwalked down Thayer St. in therain with my baggy swimming suitflapping, I felt a littlestrange until I noticed a lot of

students in baggy

shorts and sandals. The adultslooked a little crooked at mebut the students took my look instride. It dawned on me thatperhaps if I dressed like thisall the time I wouldn’t be askedto serve on any more committeesor invited to fancy dos on PowerStreet. Perhaps the CS facultywould ask me to step down aschair so as to avoid further em-barrassing the department by mytacky dressing habits.Perhapsthis is a way out of another formof hell...

After reading this, Shriram Krishna-murthi came to ask me if I thought itwould be ‘‘appropriate’’ for the faculty tobuy Tom the most ridiculous pair of pants

A French film crew for Electralis 2001 visited Brown CS recently. Elec-tralis 2001 (www.electralis.com), a major exhibit that just opened inLiege, Belgium, on the history and future of electricity, features mul-timedia presentations on future electrical research topics such asinnovations in medicine, physics, and of course the future of theWeb and the Internet. Looking for a glimpse of cutting-edgeresearch in computer graphics and human-computer interaction,they interviewed graphics groupies and Andy van Dam. Andytalked at length about the philosophy and social implications of vir-tual reality (e.g. what is “truth” in a psycho-physical computer-induced experience) in addition to the more general theme ofhow the technology will evolve and mature. The crew interviewedthe following staff/grad students/post-docs—Takeo Igarashi,Cagatay Demiralp, Jean Laleuf, Tim Miller, Anne Spalter. They alsosaw cave demos from Dan Keefe, Andrew Forsberg, Daniel Ace-

vedo, Cagatay Demiralp, and Song Zhang.They were very impressed by the research andwere particularly awed by Dan Keefe’s CavePainting program, seeing it as a major milestonein the evolution of artistic expression throughtechnology.

Much more film was shot than could possibly beused in Electralis 2001 and the producer anddirector expressed interest in producing a sec-ond documentary for French television. Theremay also be interest in a government-fundedcultural project further exploring the new fron-tiers of technology in artistic expression.

Don’t miss the Cave Painting website:www.cs.brown.edu/research/graphics/research/sciviz/cavepainting/cavepainting.html

Virtual paint can bedripped out of a

real bucket that hasa tracker attachedto it. This paint can

be thrown ordripped onto thewalls and floor of

the Cave.

An artist at work

conduit! 23

we could find. It was clear that Shriramcame to me because he wanted to do itand figured if there was any faculty mem-

ber with no sense ofdecorum, it was me.Living up to my rep-utation, I said it wasa great idea, and im-mediately went toour Editor-in-Chiefto make sure wewould capture theevent on film (Suzistill uses analog me-dia). I was not ableto attend the facultymeeting at whichthey were pre-sented, but a whileafterward I askedShriram how itwent. He said fine,but that Tom hadtold him subse-quently that he wasusing the pants quitea bit. Shriram ex-

pressed his consider-able disappointment

at this development. On the other hand,Tom has not, as he implicitly threatenedat the end of his e-mail, worn them towork. Not, mind you, that it would havedone him any good. As Milo Minderbindercould tell him, people who wear clownsuits to work in order to be declared somentally incapacitated as to be no longerfit to be chair—those folks are clearlysane. That Catch 22 is really some catch!

Recently Steve Feiner (BA Music ’73, PhDCS ’87) was featured in a New Yorker arti-cle on global positioning systems. Steve’sname recently came up at a faculty meet-ing here. We were reviewing PhD stu-dents’ progress and one student wastaking so long we wondered if he was go-ing to break Steve’s record for number ofyears to graduate. (He isn’t even closeyet.) After finishing Steve went to Colum-bia and is now a world expert in “assistedreality,” which is sort of like virtual real-ity except you overlay the “virtual” on topof the “real.” To do this, however, the com-puter needs to know what you are lookingat, and to do this outdoors it needs globalpositioning.

To appreciate the true vileness of these pants, check outthe color version of conduit! on our website: http://

www.cs.brown.edu/publications!

Congratulations to DepartmentManager Trina Avery on her 20+

years at Brown (see over)

Department of Computer ScienceBrown University

Box 1910, Providence, RI 02912, USA

conduit!

conduit! 24

NON-PROFIT

U.S. Postage

PAID

Providence, RI

Permit #202

Printed on recyled paper Address changes welcomed

conduit!A publication of

The Computer Science DepartmentBrown University

❦Inquiries to: conduit!

Department of Computer ScienceBox 1910, Brown University

Providence, RI 02912FAX: 401-863-7657

PHONE: 401-863-7610EMAIL: sjh@cs.brown.edu

WWW: http://www.cs.brown.edu/publications/conduit/

Jeff CoadyTechnical Support

Katrina AveryEditor

Suzi HoweEditor-in-Chief

I was amused by the author’s descriptionof Steve’s office: ‘His office looked as if ithad been constructed as a set for a filmabout an absent-minded professor: lap-tops—whole and disassembled—digitalcameras, special optics, and antique com-puter mice were everywhere. There werereprints of articles from the days whentransistor radios were making news andseveral bottles of Taittinger champagnesat on a table in the center of the room.’My amusement stems from the fact thatSteve was Andy van Dam’s student, andthis could very well serve as a descriptionof Andy’s office (see conduit! vol. 2, no.2). However, the most interesting quotefrom the article is the description of Steveas ‘Walter Mitty with a governmentgrant.’ I sent Steve e-mail asking his reac-tion to this description. Steve’s reply:

“Well, it’s not exactly my fa-vorite quote from the article.However, the writer seemed tothink it was a compliment. Iguess they don’t assign Thurberin high school anymore. :-)”

I received a nice letter from the Univer-sity a few weeks ago inviting me to a re-ception in honor of staff members whohad been at Brown for 20 years or more.In particular, the reception would recog-nize Trina Avery’s “20.33 years of service.”That .33 mightily impressed me—HumanResources was really up on things! Ofcourse, the effect was spoiled slightly bytheir having sent the letter to the wrongperson. It should have gone to Tom Dean,our chair. I have not been the chair of thisdepartment for nigh onto 3.66 years.