Volume 10, Number 2 Department of Computer Science Fall, 2001 Brown University

Brown University, Box 1910, Providence, RI 02912, USA

conduit!condu t!I wondered whether this was an artistic de-cision or whether it contributed to the dy-namics of the walking motion. According toone of the long-time researchers on thisproject, the dynamics of the arm swing ac-tually are important, and swinging thearms in this way saves them about 16% ofthe total energy required for the walk. Thisis good news for those of us who believethat concepts such as energy minimizationare important for understanding human(and humanoid) motion.

Sony has entered this area of humanoid re-search more recently. Their 50cm highSony Dream Robot (SDR) has been posi-tioned as the successor to the Aibo dog.With their focus on entertainment, Sonymay be the first actually to turn a profit onhumanoids. We saw the SDR kick a ball ina videotaped demo, so it is clear that Sonyis thinking about Robocup (www.rob-ocup.org) soccer tournaments with the SDRas a competitor.

There is also a great deal of humanoid re-search at Japanese labs and universities.At a workshop with the theme “New Fron-tiers in Intelligent Robotics,” hosted in To-kyo by the Japan Society for the Promotionof Science and Professor Inoue from theUniversity of Tokyo, we heard about andsaw humanoid robots that could fall downand get up again, humanoid robots whosemotion was controlled by a network of ten-

How can we makehumanoid robotsexpressive, useful,or entertaining? Isthere a market forhumanoid robots?How can we makethem safe to bearound?

I had the opportu-nity to work in Ja-pan for a fewweeks this sum-mer, and human-oid robots were arecurring theme ofthe trip. There is alarge concentra-tion of humanoidresearch in Japan,

including significant corporate efforts byHonda and Sony. Honda has been workingon humanoid robots for 15 years now.Their interest in humanoids is quite un-usual when you think about it, given theirprimary business of cars! You have to ad-mire their ability to think long-term.Honda is now expressing interest in mak-ing its humanoid research program beginto pay for itself, however, and their child-sized Asimo robot will be available forlease.

As an aside, here is an interesting factabout Asimo. If you see it walk, it swingsits arms vigorously—more so than theother Honda robots. When I first saw this,

ADAPTING HUMAN PERFORMANCEFOR HUMANOID ROBOTS

Nancy Pollardand DB

“we heard about andsaw humanoid robots

that could fall down andget up again”

conduit! 2

oid may be the most compact general-pur-pose robot possible.

(2) People may be better able to relate torobots that look something like ourselvesand that can communicate in a similarmanner.

(3) It should be easier to teach robots thatare similar to ourselves. Ideally, we wouldlike to be able to just show the robot howto do something and have it perform thetask correctly.

(4) There is expected to be a growing needfor robots in the home. The Japanese pop-ulation is aging very rapidly compared toother nations, with 25% of the populationexpected to be over age 65 by the year2020. There is a great deal of interest inservice robotics, especially robots for thehome, and many imagine that these ro-bots will be at least partially humanoid inappearance.

(5) The basic desire to reproduce our-selves probably figures in at some levelas an additional motivating factor!

THE HUMANOID ROBOT DBHumanoid robotics research seems to fallinto two main camps at the moment thatcan loosely be described as navigationand upper-body skills. A large proportionof humanoid robotics research focuses onthe problem of transport—getting the ro-bot simply to traverse hard floors, car-peted floors, and sloped floors and moveup and down stairs is difficult, and bipedrobots cannot yet walk on uneven terrain.A completely separate set of problems hasto do with giving the robot the ability tomanipulate objects and communicate inan expressive manner. The two sets ofproblems are often decoupled: if naviga-tion skills are the primary research focus,manipulation skills of the robot may belimited; if manipulation or expressivenessis the primary research focus, the robotmay have wheels or be fixed in place.

The robot DB was constructed to studyhigher-level functions of the brain, suchas learning from demonstration. DB has30 controlled degrees of freedom, but trueto form, DB cannot walk. Although itslegs do move, it is rigidly mounted inplace by a mechanical connection at thepelvis. Skills that have been explored us-ing DB range from drumming to visual

dons instead of the usual motors, and “so-cial” robots that were programmed tointeract with and entertain children. In-formation on these and other humanoidefforts can be found at http://www.an-droidworld.com/prod01.htm.

Along with a few friends and colleagues—Jessica Hodgins and Chris Atkeson fromCMU and Marcia Riley from GeorgiaTech—I spent a couple of weeks in Kyotoworking with DB, yet another humanoidrobot, but more about that shortly.

WHY HUMANOIDS?

One obvious question to ask is, “Why allthis interest in humanoid robots?” Hereare some of the reasons I heard duringmy visit:

(1) Man-made artifacts in the world havelargely been designed with people inmind. A humanoid robot should be able togo wherever we go, reach whatever wecan reach, use whatever tools we can use.Some researchers believe that a human-

The humanoid robot DB in action. The paper towelsunder his feet are to soak up leaking hydraulic fluid!

conduit! 3

is mapped to a skeleton used for anima-tion purposes. This process also intro-duces error, because the joints of theskeleton may not exactly match the jointlocations of the performer, and the skele-ton does not allow the same types of mo-tions as a person. It cannot, for example,capture the full flexibility of the humanspine.

▼ Restricting the motion to the degrees offreedom of the robot. The robot has fewerdegrees of freedom than the animationskeleton. The most substantial differenceis that the robot has a completely rigidtorso.

▼ Restricting the motion to the robot’sjoint limits. The robot has a much smallerrange of motion than the human subjects.One perceptually salient example is thefact that the elbow joint cannot be fullyextended.

▼ Restricting the motion to the robot’s ve-locity limits. The robot also has somewhatrestrictive velocity limits. Velocities arelimited when the robot nears the maxi-mum torque that can be supplied by themotors.

▼ Controlling the robot to track a desiredtrajectory. The robot will not go exactlywhere it is commanded to go, which leadsto another source of error.

tracking to Okinawan folk dance in re-search supported by the Kawato Dy-namic Brain Project at ATR in Kyoto, Ja-pan (http://www.erato.atr.co.jp/DB/ho-me.html).

EXPRESSIVE ROBOTS?Our interest in DB is to try to understandhow human performances can be adaptedto the robot with as little quality loss aspossible. We chose entertainment—story-telling—as a domain because this robot iswell-suited to the task and because story-telling utilizes a wide variety of motionsexercising a large portion of the per-former’s workspace.

It is not straightforward to map a humanstorytelling performance to DB becausethe robot has many fewer degrees of free-dom than our human subjects, differentlimb lengths, a different range of motion,and different velocity limits. In fact, wecan think of the process of transferringmotion from the human actor to the robotas putting the motion through a series offilters, each of which introduces error.

These filters are:

▼ Capturing the motion. We used an opti-cal motion-capture setup in Jessica Hod-gins’ lab at CMU. Eight cameras recordmotion of reflecting balls placed on oursubjects as they perform. The mainsource of error at this step is motion ofthe reflecting balls with respect to theskeleton of the performer due to motion ofthe clothing or skin.

▼ Mapping the motion to an animationskeleton. After the motion is captured, it

Jessica Hodgins’ motion-capture lab at CMUOur animation skeleton has joints

in the locations shown

conduit! 4

mance by each level of filtering? We aredesigning a user study to be run this fallthat will test subjects’ ability to matchmotion samples with videos of the origi-nal actor performances. (Was this motionobtained from this actor?) We expect thatthe task will be trivial when the motionsample is taken from raw motion-capturedata, but that it will become difficultwhen we present motion that has passedthrough a number of filters. In particular,the filter for the joint limits of the robotoften results in a substantial change tothe motion. It is also possible that the me-chanical appearance of the robot will besufficiently distracting to make the taskmore difficult than if the motion wereplayed through an animated humanlikecharacter.

Even before the user study results areavailable, we can see room for improve-ment in the robot’s performance. One flawis that we treat each degree of freedom(each motor) separately, scaling the mo-tion of that degree of freedom into thejoint and velocity limits of the robot.What a person observes, however, is themotion as a whole, and meaningful ges-tures that comprise that motion. Thesegestures are a result of the coordinatedefforts of many joints, and so these jointscannot be treated independently. For ex-ample, the robot has a very restrictedrange of motion side to side (abduction/adduction) at the waist and a more gener-ous range of motion forward and back(flexion/extension). This difference in lim-its, along with our relatively simple pro-cessing technique, results in the followingartifact: when the robot bends to the tuneof “tip me up and pour me out,” it firstleans mostly to the left or right, hits theabduction/adduction joint limit, and thencontinues to lean forward, making thetipping motion look more circular thanlinear. This sort of problem is easilysolved if we have a model of meaningfulgestures. The tipping motion could bemapped in its entirety into a linear mo-tion within the robot’s workspace.

Even though there is room for improve-ment in the robot’s performances, Istrongly believe in the importance of hu-man examples for creating realistic mo-tion of both animated characters androbots. On an orthogonal track, I am ex-ploring how to design a robot hand that is

Our first pass, which we implementedwhile in Japan, was to obtain as similar a“rendering” of the original motion as pos-sible. This meant throwing out degrees offreedom that the robot does not have, lo-cally compressing and smearing the mo-tion to fit within joint and velocity limitsrespectively, and adding a learning phaseso that the robot would perform the re-quired trajectory as accurately as possi-ble. The figure shows a snapshot of someof our results in a single frame of the ro-bot and human performers doing the chil-dren’s song “I’m a Little Teapot.”

How do we evaluate these results? Inother words, how do we assess the extentof the damage done to the original perfor-

DB trying to follow a human performance of“I’m a Little Teapot”

conduit! 5

mechanically practical and has similarforce and stiffness capabilities to the hu-man hand. I hope to show that designsthat are similar to the human hand inthese ways will facilitate teaching grasp-ing and manipulation skills by example.In other words, it should be easier tograsp and manipulate objects successfully

following human examples with such ahand than with existing hand designsthat are not so strongly based on humananatomy, simply because the passive dy-namics of the hand work in your favor in

an anatomically motivated hand design.But perhaps this is the topic of a futurearticle.

FINAL THOUGHTS

This project was in part an excuse to visitJapan and spend some time working atATR, but it did raise some very interest-ing questions that we look forward to ad-dressing in future research, such as whatcharacteristics or features of motionshould be preserved in a mapping fromhuman actor to animated character or ro-bot. I also enjoyed the overview of the cur-rent state of the art in humanoid roboticsresearch. There are of course many issuesto be resolved before we have C3PO wan-dering around, not the least of which issafety of large autonomous devices thatare about as stable as inverted pendu-lums. Humanoid robotics research has agreat deal of support behind it right now,however, and with the added ability eas-ily to capture large datasets of motion ex-amples, we can expect to see humanoidrobots becoming more graceful, expres-sive, and perhaps somewhat useful in thenear future.

Mark Dieterich, senior systems administrator for the graphics group, is also an amateur ra-dio operator. When disaster strikes, the Federal Government activates the Radio Amateur Civil-ian Emergency Services (RACES) and places its operators on standby. After the WTC tragedy,New York City’s operators were immediately called into action to provide non-secure radio net-works for emergency personnel working at ground zero.It soon became necessary to introduce fresh operators from surrounding areas and Rhode Islandoperators were put on standby. When the call came the weekend after the attack, Mark and fourothers took their equipment to the Red Cross’s initial staging area in Westchester County. Fromthere they were bused to the Red Cross in Brooklyn and thence to the site in lower Manhattan.Cell phones in the area were next to useless and a stable communications network was criticalfor smoothly coordinated rescue efforts. The amateurs operated on a single frequency with a con-trolled flow of information and requests going to a net controller and then out to other operatorsassigned to various tasks. Mark was responsible for directing calls to various Red-Cross-runshelters: respite centers for police, firemen and other workers, shelters for dislocated people andsecure government shelters. He also spent a day ‘tailing’ a Red Cross executive around the site.

For the last four days of his week-long stay, Mark became the day-shiftmanager for amateur radio operators. His job was to brief and debriefteams going into and out of the field and to deal with non-routine prob-lems. Most 12-hour shifts were in fact 18 hours long, as they still continueto be.Mark was a dispatcher for his hometown (Pittsford, NY) volunteer ambu-lance company for over four years, so he has experience with emergencysituations; however, he found working at the WTC site overpowering. Asthe week progressed, the tight security became increasingly evident, withMPs on guard outside Red Cross headquarters. Said Mark, “If you knewNew York before, it’s changed—the atmosphere—people are making eyecontact in the subway and are talking to one another instead of reading.”Anyone involved with rescue and cleanup has found restaurant owners re-fusing to accept payment, taxi drivers who turn off their meters when aworker gets in; for workers, the subway is free. Being of use has helpedhim cope with the enormity of the tragedy.

Jessica, DBand Nancy

conduit! 6

The folks from conduit! contacted me awhile back and asked if I would considerwriting a short article for them. Havingbecome a conduit! fan and being happyto repay the CS Department and facultyfor all the time and energy they put intoeducating me so many years ago, I agreed.

At the time, I didn’trealize the challengeof combining a dis-cussion of my workwith some biogra-phy—but the two areintertwined and theeditor asked for both,so here’s my bestshot.

Let me start with alittle biography. Up-on completing mydoctorate at Brownin 1986, I joined thefaculty at the Univer-sity of Maryland and

started a research group in artificial intel-ligence. With a few lucky breaks and a lotof great students, I managed to get pro-moted a couple of times and became a fullprofessor with a large research group.Much of my group’s work centered onscaling up work started at Brown and try-ing to show that AI could be a valuableplayer on the then-emerging World WideWeb. A language we developed, calledSHOE for “Simple HTML Ontology Ex-tensions,” gained a certain cachet in aparticular research community, and weduly published various boring academicpapers, developed a number of small andinconsequential demos, and did all theother things needed for me to get pro-moted and my students to find academicjobs.

Somewhere about four years ago, how-ever, I realized that although the workwas going well, it was also going rela-tively nowhere. The World Wide Web is anunimaginably large cyberspace, and eventhe best academic efforts rarely get anyreal visibility on it. I thought that if Icould get, say, 10,000 people using mystuff, it would be one of the most used aca-

demic AI products ever—but it would stillbe being used by fewer than one in every100,000 web users! To really change theweb, and as you’ll soon see that’s my goal,something bigger was needed.

Unfortunately, to do something on alarger scale there’s not a lot of places toplay. Academics wanting this sort of im-pact have generally had to choose be-tween two alternatives: start your owncompany or join some large software con-cern (generally one located in Redmond,WA). Neither of these appealed to me, ascorporate America has never been anarena I found attractive. But I found athird alternative—if you want to work atweb scale, join the folks who made theweb happen!

The Internet, as we now call it, startedout as a project called the “ARPAnet” un-der the sponsorship of the Defense Ad-vanced Research Projects Agency(DARPA1) of the U.S. government.DARPA also helped to support work inthe creation of the World Wide Web andin tools for using web information. Fur-ther, DARPA is currently the govern-ment’s largest funder of research incomputer science. It was clear that tohave the sort of impact I wanted, DARPAwas a place to consider. So in late 1998, Itook a three-year leave of absence fromUMD and signed up to work at DARPA.

Now I’d like to introduce my partner incrime in our effort to change the world. Inthe late 1980s, a researcher named TimBerners-Lee created a program calledWorld Wide Web that caught on prettywell. With backing from various research

1. Why ARPAnet and not DARPAnet? Gener-ally, under Republican administrations orCongresses the organization has been asked tofocus on the science needs of the DoD and hasbeen called DARPA. When the administrationis Democratic, the D is dropped, and theagency becomes ARPA and focuses on dual-use technology. The name changes oftenenough that the official coffee mugs say“DARPA” on one side and “ARPA” on the other,so that employees won’t have to rememberwhich one to give visitors.

SPINNING THE SEMANTIC WEB

Jim Hendler, PhD ’86

conduit! 7

The vision of the semantic web is not thatof a new web but of new languages andfunctionalities that extend current webfunctionalities. On the Semantic Web, notjust web pages, but databases, programs,sensors and even household applianceswill be able to present data, multimedia,and status information in ways that pow-erful computing agents can use to search,filter and prepare information in newways. New markup languages that makesignificantly more of the information onthe web machine-readable power this vi-sion and will make possible a new genera-tion of technologies and toolkits.

The semantic web is expected to evolvefrom the existing web as these new lan-guages and tools find their way into themarketplace. Figure 1, based on a figurein a recent Nature article4 on the futureof electronic publishing, shows the poten-tial evolution of the semantic web. We arecurrently seeing new web languages likethe eXtensible Markup Language (XML)and the Resource Description Framework(RDF) that let users produce more “meta-data” about web resources. These meta-data can be as simple as saying whoproduced the document and when, orcomplex enough actually to replace theweb page with machine-readable infor-mation.

To understand the difference in using thisemerging net, imagine going to a searchengine and typing the query, “How manytrain lines are there in Japan?” If you useany of the popular search engines, you getmany, many pages back (at this writing,between 122,000 and 99,000,000 answerswere returned)—and few, if any, actuallycontain the answer to the query beingasked. As a foundation for browsing theweb and learning lots about Japanesetransportation, this may be okay, but foranswering the question—worthless!

3. A. Swartz and J. Hendler, The SemanticWeb: A Network of Content for the DigitalCity, Proc. 2d Annual Digital Cities Confer-ence, Kyoto, Japan, October, 2001. (http://blogspace.com/rdf/SwartzHendler)

4. T. Berners-Lee and J. Hendler, Publishingon the Semantic Web, Nature 410, 1023-1024(26 April 2001) (http://www.nature.com/na-ture/debates/e-access/Articles/berner-slee.html)

agencies including DARPA, Tim moved toMIT in the early ’90s and in 1994 startedthe World Wide Web Consortium (W3C)to help define and standardize the lan-guages on the web. The consortium nowhas a membership of over 500 companies,including all the largest players in infor-mation technology and web applications.Berners-Lee is the director of the W3Cand remains one of the leading thinkersabout the future of the web.

One of my first acts at DARPA was to talkto Berners-Lee about an idea of his calledthe “Semantic Web.” The work my grouphad been doing in SHOE was aimed atsome of his semantic web ideas, and Ithought some joint effort between DARPAand the W3C could help move this tech-nology out of the laboratory and into com-mon use. Our interaction led to two newcreations—a DARPA program called the“DARPA Agent Markup Language”(DAML) and a W3C activity called “Se-mantic Web Advanced Development.”

I recently joined with Berners-Lee andanother colleague to produce a ScientificAmerican article2 describing aspects ofthis new work—I summarize below someof that article. I also steal from a forth-coming article, “The Semantic Web: ANetwork of Content for the Digital City,”coauthored with a 14-year-old whiz kidnamed Aaron Swartz3.

2. T. Berners-Lee, J. Hendler and O. Lassila,The Semantic Web, Scientific American, May,2001. (http://www.sciam.com/2001/0501issue/0501berners-lee.html)

Figure 1: The evolving semantic web—newweb languages and toolsets for revolutionary

functionality on the World Wide Web.

conduit! 8

Why, however, can’t this answer be found?Many resources on the web could be used.First, some of the many documents foundprobably have this information, but cur-rent language-processing and search tech-nologies are nowhere near good enough tofind them. Second, web-accessible data-bases and programs could provide the an-swer, but word-based text matching is not

sufficient to pull them out. Third, sinceeach train line in Japan makes its pres-ence known on the web in one way or an-

other, a complicated program could bewritten to find these, identify them, andcount how many there are—but writingsuch a program is a massive undertaking,far more effort than users would want tomake for a single query.

We can imagine a day when a query likethis could elicit a very different sort of re-sponse. For example, a semantic webquery tool could give replies like:

➡ http://www.transit.co.jp/lines saysthe number of train lines is over 5000.

➡ There is a database that can providethat number; please provide an autho-rization number.

➡ There is a web service that can com-pute that number; please provide 500yen for the answer.

➡ I can get you an approximate an-swer by search and filtering, but it willtake about 4.5 hours to compute.

The goal of semantic web research is todevelop the languages and tools to makeanswers like this possible. (Details oncurrent research can be found in a num-ber of places; two of particular use are ageneral information site about the se-mantic web at http://www.semanticweb-.org and the site describing the ongoingDARPA work at http://www.daml.org.)

Does this sound like a crazy science-fic-tion dream or a lot of hype? A decade ago,who would have believed a web of text,conveyed by computer, would change theway we live and work? This new visionunites old players such as Berners-Leeand DARPA with new visionaries such asyoung Aaron Swartz, who recentlysummed it up well. Speaking about thesemantic web, he said, “Sure, it’s a longway from here to there—and there’s noguarantee we’ll make it—but the possibil-ities are endless, and even if we don’t everachieve all of them, the journey will mostcertainly be its own reward.”

As for me, I’m now back at the Universityof Maryland where I’m Director, SemanticWeb and Agent Technology at the newlyformed Maryland Information and Net-work Dynamics Laboratory. This lab is anattempt to develop new interaction modes

“Sure, it’s a long way from here tothere—and there’s no guarantee

we’ll make it—but the possibilities areendless, and even if we don’t everachieve all of them, the journey willmost certainly be its own reward”

Academic office aide Akina Cruz, looking elegantin traditional African dress brought back on arecent trip to Ghana by her sister, Adeola Oredola,a senior at Brown. Despite its exotic look, this out-fit is called simply a skirt and headwrap.

conduit! 9

also played basketball, enjoyed skate-boarding (yes, he watched some of theGravity Games), biking and bodybuilding.His favorite reading subject is popularphysics, followed by history as a close sec-ond. He’s also a classical rock fan (PinkFloyd was one of his early favorites, al-though he couldn’t understand a word atthe time!). Ugur looks forward to gettingtogether with Pascal Van Hentenryck, aone-time provincial-level soccer player inBelgium and senior faculty member, toplay soccer and perhaps start a CS team.

Ugur’s wife, Gamze Tunali, is still work-ing in Maryland at a networks company.She hopes to join him in Providence be-fore the end of the year. She and Ugurwere classmates at the Bilkent Univer-sity; they were married in Maryland andremarried the following year in Turkeywith both families present. Ugur spendsalmost every weekend in Maryland withGamze; consequently, he hasn’t even seena WaterFire yet, but is keen to do so.

Living in Providence means everything iswithin walking distance and easily acces-sible—no more 45-minute commutes. Heenjoys the many activities and good res-taurants, and sees Providence as a smallcity but an eclectic one. Gamze loves itand is a big fan of the “Providence” TV se-ries. She is an expert on Turkish cuisine,enjoys eating out and has a weakness fordessert! Fortunately, she can burn off theextra calories by indulging in her otherfavorite occupations, playing tennis andworking out. They are both big sushi fansand are eager to learn about the bestsushi restaurants in town. Gamze is anamateur Turkish folk dancer; she hopesto keep up with her dancing via her con-nection with Andy van Dam’s son-in-law,who is a professional folk dancer.

Although they still feel they’re in transi-tion mode with all their possessions inMaryland, they’re looking forward to set-tling down together in their new Provi-dence apartment and becoming realRhode Islanders.

between academia and industry; we hopeto create an organization that can notonly perform research into new web andnetwork ideas, but also help transitionthese ideas into use more quickly.

I look forward to hearing from Brown col-leagues, old and new, interested in the se-mantic web vision.

hendler@cs.umd.edu

Ugur spent his first tenyears in Ankara, Turkey, be-fore moving to Izmir on thewest coast. Izmir is Turkey’sthird largest city, very beau-tiful and, according to Ugur,the best city in Turkey tolive in. His father is a consti-tutional judge (i.e. not in-volved with litigation), hismother a bank manager, nowretired, and his older brotherworks in the insurance in-dustry. He returned to An-kara to attend BilkentUniversity as a CS major,when he was 19. Given thestiff university entrance ex-

aminations and intense competition foruniversity places in Turkey, Ugur’s fullscholarship award was a major accom-plishment.

After receiving his MS at Bilkent, Ugurcame to the US for graduate work in CSat the University of Maryland, CollegePark. He started working for Mike Frank-lin, a database professor, who was soonlured to Berkeley. Then in the middle ofhis third year Ugur changed advisors andtopic area and started studying distrib-uted systems under Pete Keleher. Ugur’scurrent research explores data manage-ment issues for advanced distributed sys-tems, such as mobile databases, sensornetworks, and peer-to-peer systems. Hisbroad goal is to design and build scalablealgorithms and infrastructures that willefficiently support next-generation dis-tributed applications and services. Hewill be teaching a topics course on mobileand ubiquitous computing in the springsemester.

Ugur is a huge sports buff and a greatsupporter of the soccer team Fener-bahce—“Go yellow canaries!” Besides be-ing captain of his college soccer team, he

Ugur Cetintemel

NEW CS FACULTY MEMBER

conduit! 10

For more information, visit http://www.-cs.brown.edu/cgc/stms/

EXAMPLE APPLICATIONSCENARIO

Imagine a trusted financial portal service(such as Yahoo! Finance) that providesdynamic and up-to-date financial infor-mation, including corporate and newsfeeds, market quotes, and online trading.Some key characteristics of this distrib-uted application are:

❁ High volume of accesses, queries, andtransactions

❁ Widely distributed clients (e.g., geo-graphically dispersed and with varyingaccess bandwidth)

❁ Information originating from varioussources outside the portal (e.g., stock ex-changes, government agencies, and corpo-rate investor relations offices)

How can such a portal become fullytrusted so that the integrity of all dataand transactions is guaranteed authenticend-to-end? Today, the economics aredaunting and the implementation wouldbe largely impractical. For example,HTML pages may be dynamically con-structed from several frequently chang-ing information sources, based on userrequests and user profiles. The volumeand diversity of such requests make cre-ating individual time-stamped digital fin-gerprints for any given page very difficultand time-consuming. Common techniquesfor load balancing, replication and multi-hosting subdivisions of a web site makeusing previously available trust ap-proaches unrealistic, since access to sen-sitive data must be limited and strictlysecured.

Our distributed authentication approachoffers significant new capabilities andcould dramatically change the economicsof authentication for distributed business

INTRODUCTIONSecurity is an essential re-quirement of distributedplatforms for business ap-plications. We address theproblem of authenticatinghigh volumes of data andtransactions in non-trusted distributed envi-ronments. Current solu-tions are typically:

❁ Centralized—and there-fore subject to network de-lay and denial-of-serviceattacks

❁ Expensive—to build,maintain, and operate be-

cause trusted data must be maintained ina secure environment, guarded 24/7

❁ Non-scalable—with limited throughputdue to operating and economic con-straints

We are developing a high-throughput sys-tem for authenticating data and transac-tions in non-trusted environments, at thenetwork edge and outside the firewall. Weuse a unique, patent-pending approachfor wide distribution of authentication in-formation. As a result, we can dramati-cally lower the cost of authentication insuch applications as wireless authoriza-tion, B2B e-commerce exchanges, distrib-uted storage, end-to-end integrity, tamperdetection, and certificate revocationchecking.

This work is being conducted in collabora-tion with AlgoMagic Technologies, JohnsHopkins University, and the University ofCalifornia, Irvine, and is supported inpart by a grant from the Defense Ad-vanced Research Projects Agency. My col-laborators Steve Baron, Robert Cohen,and Rich Sneider at AlgoMagic Technolo-gies and Mike Goodrich at the Universityof California, Irvine, contributed to writ-ing this article.

EFFICIENT LOW-COSTAUTHENTICATION OF DISTRIBUTED

DATA AND TRANSACTIONS

Roberto Tamassia

conduit! 11

Previous ApproachesThe simplest approach to implementingan authenticated dictionary is to makethe responders trusted parties. Thistrusted-responder approach is used,for example, in the online certificate sta-tus checking protocol (OSCP), wheretrusted OCSP responders answer certifi-cate revocation queries posed by clients.The main disadvantage of the trusted-re-sponder approach is that each respondermust be placed in a secure location, withensuing large operational costs. Thus,this approach is not scalable for economicreasons.

An alternative approach entails havingthe repository periodically sign a finger-print of the current version of the entiredatabase. The database itself, togetherwith the signed, time-stamped finger-print, is then sent to the client as a proofof the answer. This database-forward-ing approach is used, for example, toauthenticate certificate revocation lists.The database-forwarding approach allowsnon-trusted responders and thus haslower operational costs. However, it iscomputationally demanding for the client,

which needs to process the entire data-base in order to validate an answer. Also,it is not scalable for communication rea-sons, since sending the entire databasetogether with the answer uses consider-able network bandwidth.

The STMS ApproachThe main feature of STMS is that itmaintains trust even when respondersare located in insecure, non-trusted loca-tions. That is, when a client makes aquery to an STMS responder, it gets backnot only an answer but also a proof of theanswer. The client can easily validate theanswer and determine that the responderhas not been tampered with, while rely-ing solely on trusted statements signed

applications. Highlights of our approachinclude:

❁ It works with and leverages existingInternet infrastructure, protocols andcomputing platforms

❁ It is highly scalable in terms of bothcomputational and economic cost

❁ It provides succinct and timely authen-ticity proofs for entire collections of dataoriginating from diverse sources

❁ It widely distributes authentication in-formation to non-secure replication loca-tions, while strictly maintaining trust

❁ It responds to authentication requestsby clients with minimal network latencyand computational cost

We have developed a distributed authen-tication system called Secure Transac-tion Management System (STMS). Afully operational prototype of STMS hasbeen implemented.

STMS TECHNOLOGY

Authenticated DictionariesThe computing abstraction under-lying STMS is a data structurecalled an authenticated dictio-nary, which is a system for distrib-uting data and supporting authen-ticated responses to queries aboutthe data.

In an authenticated dictionary, thedata originates at a secure centralsite (the repository) and is distrib-uted to servers scattered around the net-work (responders). The respondersanswer queries about the data made byclients on behalf of the repository.

It is desirable to delegate query answer-ing to the responders for two primary rea-sons:

1. It is undesirable that the repositoryprovide services directly on the networkdue to risks such as denial-of-service at-tacks

2. The large volume and diverse geo-graphic origination of the queries requirea distributed system of servers to provideresponses efficiently (much like DNS)

“when a client makes aquery to an STMS responder,

it gets back not only ananswer but also a proof of

the answer”

conduit! 12

❁ Query formulation by a client

❁ Proof verification by a client

The key benefits to the STMS approachinclude:

❁ Answers given by the responders are astrustworthy as if they came from the re-pository

❁ Data is distributed close to the clients,minimizing network delays

❁ Deploying responders is inexpensive

❁ The repository is protected from riskssuch as denial-of-service attacks

An additional unique advantage of STMSis that it supports historical persist-ence; that is, one can perform queries onpast versions of the database, whichmakes possible a variety of nonrepudia-tion applications.

STMS comes with Java and C++ toolkitsfor building clients, including modules forvalidating responses. For maximum reli-ability and security, cryptographic func-tions are performed with the availablestandard cryptographic libraries of Javaand Windows. We also provide an XML-based protocol for querying a responderover an HTTP connection. We are cur-rently working on a Web services ap-proach in which STMS is accessedthrough the SOAP protocol.

The table on the opposite page comparesthe STMS approach with the trusted-re-sponder and database-forwarding ap-proaches with respect to operational cost,run-time performance, scalability, andsupport for historical persistence.

APPLICATIONSAs a foundation for delivering a widerange of trust services, STMS brings au-thentication to the network edge and canbe viewed as a distributed low-cost “au-thentication cache” for high-volumetransactions. Classes of trust applicationsin which STMS can be exploited include:

❁ User authentication and access controlfor large-scale corporate portals, privateexchanges, and Virtual Private Net-works—User signon can be accomplished

by the repository. The design of STMS al-lows untrusted responders to provide ver-ifiable authentication services on behalfof a trusted repository. This unintuitiveyet mathematically provable fact is thekey to achieving cost effectiveness.

The figure below shows a high-level viewof the STMS parties and protocol. The re-pository sends periodic updates to the re-sponders together with a special signed

time-stamped fingerprint of the databasecalled the basis. A responder replies to aquery with an authenticated response,consisting of the answer to the query, theproof of the answer and the basis. Infor-mally speaking, the proof is a “partial fin-gerprint” of the database that, combinedwith the subject of the query, should yieldthe fingerprint of the entire database. Aproof consists of a very small amount ofdata (less than 300 bytes for most appli-cations) and can be validated quickly. Theclient finally evaluates the risk associatedwith trusting the answer using the fresh-ness of the time-stamp.

Our patent-pending algorithms, whichemploy only standard cryptographic func-tions for signing and hashing, guaranteethat the following tasks can be performedin near real time and with minimal use ofcomputing resources:

❁ Creation of the new signed basis by therepository for each time quantum (the ba-sis is incrementally updated)

❁ Update of the database copy residing ata responder for each time quantum (thecopy is incrementally updated)

❁ Assembly of the proof by a responderfor each query (the proof is obtained bycombining precomputed partial proofs)

conduit! 13

❁ Secure DNS—Host name to IP addressmappings could be upgraded to includedigital signatures. The data-distributionapproach would be essentially similar tothat used today. Widely distributedtrusted DNS servers (as STMS respond-ers) would not have to be located in se-cure settings.

❁ Tamper detection for Web sites—Webpage digital fingerprints can be madeavailable as a reference. The computa-tional load and delay of signing everypage on each web server at all replicatedlocations are avoided.

❁ Tamper detection for operating sys-tems—System file fingerprints can bemade available as a reference on a local-ized basis for large groups of client sys-tems.

❁ Tamper detection and license checkingfor software applications—Software is in-creasingly distributed or accessed overthe network. Metered access can be effec-tively implemented, identity and status ofservers and clients can be mutually con-firmed, and component fingerprints canbe quickly verified.

Cost Performance Scalability Persistence

Trusted-responder High High Low No

Database-forwarding Low Low Low No

STMS Low High High Yes

“Access rights, roles, andderived trust assertions canbe quickly checked againsta localized store of consoli-

dated credentials”

without the need to contact a remote cen-tral server.

❁ Wireless two-way authentication—Mo-bile device identity can be confirmed bygateways, network services, and applica-tions, and devices can also confirm theidentity of selected network services andapplications to establish two-way trust.

❁ Trusted XML Web Services—Two-waytrust can be established between the par-ties involved in a “chain” of Web Services.Keys for signed or encrypted portions ofXML can be obtained from widely distrib-uted servers (STMS responders) ratherthan centralized systems.

❁ Certificate status checking (e.g.,for SSL, secure email, and code cer-tificates)—Identity and status canbe mutually verified by both senderand receiver of email, publisher anduser of code components, client andserver in a secure dialogue.

❁ Validation services for businesspartner networks (e.g., healthcareproviders)—Access rights, roles, and de-rived trust assertions can be quicklychecked against a localized store of con-solidated credentials. Public networkscan be employed instead of incurring thecost of a private value-added network.

❁ End-to-end integrity for content collec-tion and distribution systems (the exam-ple scenario presented at the beginning ofthis article).

❁ Non-repudiation services for electroniccommerce—Third-party verification andaudit capabilities can be implemented tosupport large-scale B2B networks (e.g.,corporate portals, exchanges, trading sys-tems), as well as high-volume B2C sys-tems (e.g., media distribution, digitaltickets, electronic payment systems).

conduit! 14

DILIPD’SOUZA,MS ’84In August, Pen-guin India pub-lished my firstbook: Brandedby Law (seewww.penguin-booksindia.com/

books/aspBookDetail.asp?ID=4567). It’sabout some of India’s most forgotten peo-ple—its denotified tribes, once actuallydefined as criminal. But more than that, itis also a look at prejudice, a phenomenonthat shapes their lives every day and alsogreatly interests me for various reasons.

Now that it’s out, I get asked all the time:CS, but now you write? What’s the story?

I came to writing late, true. It neveroccurred to me as a possible careerin my university years, nor in myyears in software. But when I re-turned to India in 1992, I tried writ-ing for a lark. Got published a fewtimes. Wrote some more, then moreregularly. And pretty soon it wasclear to me that I had found what Itruly wanted to do with my life.Write.

Yet I never regretted my time in CS,because it strikes me as marveloustraining for writing. Not everybodywould take this route, I know. Butthe lessons from CS serve me wellwhen I write. At its best, and I’msure conduit!’s readers will agree,programming is an intensely cre-

ative process. Good software is every bitas satisfying and elegant as a good pieceof writing. And of course the word “ele-gant” is one all programmers aspire to.Software described that way is an exhila-rating mix of functionality, leanness, clearthought and a certain beauty.

I try to write like that: clearly, simply. Cutto the core of issues, find connections, besure of myself. These are the attributes ofwriting that I value, that I aspire to. (Also

to having someone tell me someday that Iwrite “elegantly”.)

All this, in many ways, is where this bookof mine came from. Too much happensaround us that is based on unquestionedstereotypes people hold about each other.One such hounds denotified tribes: theimpression that they are all vicious crimi-nals. In reality, they are not particularlymore (or less) criminal than anyone else,but that means little. So via the experi-ence of these forgotten people, I try to getmy readers to ask questions about preju-dice, to ask questions above all of them-selves. I try to do this with those tools Ifirst learned about in CS: reason, clarity,cut the fluff.

And the interesting thing is that while alot of people see it as startling, this appar-ent “switch” from CS to writing, I see it asa smooth and even logical transition. It’salmost as if I write, and write the way Ido, because of my years in CS. I owe thatto Brown and to several people who fea-ture regularly in these pages. So pleaseread my book! Tell me what you think.dilip@alumni.brown.edu

RICHARD HUGHEY, PhD ’91Hi all! This past year I have bumped intoa few Brown folks. The most surprisingwas at a July computational biology con-ference (ISMB) overhearing talk of rub-ber chickens. On turning around, Imentioned having one in my office (onedecaying original rubber chicken, andthree plastic replacement juggling chick-ens!). The speaker turned out to be SoniaLeach, not only a Brown student but alsooccupying my old office—555. We chattedabout what we were doing, and I was ableto clear up her puzzlement about how aVLSI chip plot wound up in the window of555—it’s from my dissertation. I’ve goneon to supervise the design, construction,and use of another parallel processor atUC Santa Cruz.

Ethan Miller (ScB ’87) joined our CS de-partment this year; I remember seeing

ALUMNI EMAIL

conduit! 15

ing Silicon Valley tech economy, I beganworking for Tufts University Hillel inBoston, and then a year later moved toUC Berkeley Hillel in California, where Iwas earning a non-profit salary living justabove the poverty line. Hillel is the inter-national organization that serves theneeds of Jewish college students (I loggedmany hours as a student at the BrownHillel when I wasn’t in the CIT), and inmy misleadingly titled job of “ProgramDirector” I spent my days meeting Jewishstudents, planning and attending funevents with them, and playing my guitar.In addition, because of my extensive com-puter science background, I was also reg-ularly called upon to teach my co-workershow to perform highly technical com-puter-related tasks such as: sendinggroup emails, copying files to and fromour local area network, replacing printertoner cartridges, and of course, program-ming the VCR. All in all, the job was agreat experience for me.

I enjoy working with people, and find thatI fit in comfortably in the non-profitscene, so with that in mind, I have de-cided to go back to school to brush up onmy management skills before trying myluck as a director of my own non-profit or-ganization. So, I have just recently leftBerkeley and moved to Evanston, Illinois,where I will be starting as a business stu-dent at the Kellogg/Northwestern Schoolof Management. Ironically, it seems thata large number of my classmates have CSundergrad degrees and are former dot-commers who decided to sell their BMWsand apply to business school when theeconomy collapsed and their companieswent under! I guess the old adage is true,“If you don’t know where you’re going,any path will take you there.”

Despite my unusual path, I have alwaysbeen glad I went the CS route at Brown,and in a true liberal arts spirit I believethat my whole CS experience—TAing,group work, and even building computersystems—was a great educational prepa-ration for my work since graduating. I’dlove to hear from folks, and encourageother Brown CSers on nontraditionalroutes to write in and tell their stories!Josh Miller, jmiller@songleaders.org

his name a few times as a grad student,perhaps in 169 (operating systems).

Being an involved graduate student hascaught up with me, and now I’m chair ofour computer engineering department. Iran into Tom Dean and Jeff Vitter at theCRA meeting last March as a pre-chairand, not ducking quickly enough, the po-sition caught up with me two months ago.I won’t go into all the pleasures of the po-sition, as then no graduate student wouldever get involved in anything for fear ofeventually becoming a department chair!!

Parts of the job, like the planning of re-search and academic programs, are quitefun—CE has five positions this year,starting the ramp-up to UC’s “Tidal WaveII” enrollment surge. I’ve also been devel-oping BS (starting 2001-2), MS and PhD(starting 2002-3, we hope) curricula forBioinformatics degrees with my col-leagues Kevin Karplus (protein structureprediction) and David Haussler (puttingtogether the human genome, amongother things). We’re planning to buildthis into a department in a couple ofyears, and are currently recruiting threefaculty in bioinformatics and related ar-eas.

Otherwise, Santa Cruz is a great place—with a campus of redwoods, I quicklyovercame my trepidation at interviewingon the west coast! rph@ce.ucsc.edu

JOSH MILLER, AB ’96Hello out there to the Brown CS Commu-nity—Well, I decided to mail a littlechange-of-address notice to Suzi Howeand she craftily cornered me and askedme to submit a quick update to conduit!on what I’ve been up to, not realizing thatsuch a request would elicit a dual confes-sion of unprecedented proportions. Sohere it goes: 1) I haven’t done anythingrelated to computer science since I gradu-ated and 2) despite that, I still read myconduit! cover to cover (including thetechnical articles) every time it comes inthe mail. Weird, eh? So what exactly haveI been up to? Well, in 1996, when most ofmy graduating class (including the phi-losophy majors) went straight fromBrown to cushy jobs working in the boom-

conduit! 16

JON MOTER, ScB ’99

Spike forwarded the following, which JonMoter posted to brown.cs.stupid:

With Microsoft’s new version of Office,they’re finally killing off the annoyingPaperclip Assistant. As a marketingmove to push MS Office XP, they’ve cre-ated a site for Clippy, bemoaning his ownfate and looking for a job. It’s actuallyfairly amusing: http://www.office-clippy.com. He has a resume (http://www.officeclippy.com/resume.html) with

past work and education experience. Loand behold, it turns out he graduatedfrom Brown in ’94 with an Art/Semioticsdegree. Apparently he “graduated cumlaude, with a performing arts thesis thatinvolved twisting myself into a represen-tation of Michelangelo’s David.” Alwaysgood to see Brown alums in prominentpositions. Jon “You still have an ac-count???” Moter.

Said Spike: “So...even though the paper-clip’s not from Brown CS, perhaps it de-serves mention in conduit! (or to ourlawyers...)”

SCOTT SMOLKA, PhD ’84

Thought this photo might be of interestto conduit!’s readers: three of Paris’sPh.D. students: me, Alex ShvartzmanPh.D. ’92 and Dina Goldin Ph.D. ’97. Iwas in Boston visiting Dina; we’re work-ing on a joint paper with Peter Wegnerabout Turing machines, transition sys-tems and interaction. Alex was visitingNancy Lynch’s group at MIT. The restau-rant is somewhere in Kendall Square. Allthe best, Scott. sas@cs.sunysb.edu

The paradigm of human-com-puter interaction based on win-dows, icons, menus, and pointingdevices has changed little sinceits development in the early1970s. With the exception ofsome limited voice-recognitiontechnology, a computer’s knowl-edge of the external world comesthrough this screen/keyboard in-terface. In contrast, one-on-onehuman interactions involve agreat deal of context in whichthe speakers are aware of theirshared environment and the ob-jects in it as well as each other’s

facial expressions and gestures.

This sort of rich interaction is not possiblewith today’s computers due to their lim-ited intelligence and their inability to per-

ceive the world outside them. The 27thIndustrial Partners Program Symposium,held last May, brought together leadingresearchers from companies that are at-tempting to revolutionize the interactionbetween machines and humans by en-dowing computers with perceptual capa-bilities. Speakers from Microsoft, Intel,Compaq, Mitsubishi, and IBM spokeabout the fundamental science needed toenable computers to “see” and under-stand their users. They also presentedsome novel “perceptual user interfaces”that explore how these abilities mightchange the way computers and humansinteract. We first heard about the “Ea-syLiving” project from Dr. John Krummof Microsoft Research. At their Red-mond research facility, Microsoft has builta “living room” with computer screensembedded in the walls that can provideinformation and entertainment (figure onnext page). The room is also equippedwith cameras and other sensors that de-tect the location of people in the room, de-

27th IPP SYMPOSIUMVISION-BASED INTERFACES

Symposium host,Michael Black

l to r, Scott Smolka, Alex Shvartzman and Dina Goldin

conduit! 17

termine their identity, and track them asthey move about. The project is exploringthe kinds of services that might be sup-

ported by this sort of “intelligent room.”For example, the room can “learn” indi-vidual preferences that are used tochange the lighting, heating, or music se-lections. Additionally, when a known per-son enters the room, their computingenvironment follows them with theiremail, bookmarks, and instant messages.

Supporting these applications are a num-ber of cameras that compute the three-di-mensional structure of the room anddetect the location of people. A trackingsystem combines information from thecameras and pressure sensors in the fur-niture to track multiple people in the

room at one time. These features providea testbed for exploring novel interfacesfor product groups at Microsoft.

Reliable detection and tracking of peoplein complex, changing environments suchas the living room of the EasyLiving pro-ject is still an open problem. Dr. JohnMacCormick of Compaq’s SystemsResearch Center spoke about theirprobabilistic methods for people tracking.Their BraMBLe system builds probabilis-tic models for what the background(room) and foreground (people) look like.These models are used in an elegantprobabilistic formulation that exploitsBayesian probability theory to combineinformation measured from a camerawith past measurements and priorknowledge of the world. The result is a

Microsoft Research, EasyLivingProject. This “intelligent” living room

senses people in it

IPP Symposum speakers from l to r: John Krumm, Microsoft Research; MichaelBlack, Brown; Matthew Brand, MERL; John MacCormick, Compaq; Jim Rehg,

Compaq; Myron Flickner, IBM Almaden; Gary Bradski, Intel Corporation

Compaq System Research Center.Probabilistic tracking of multiple

objects in a scene

conduit! 18

rived. He showed impressive results oftracking a face in a monocular video se-quence while recovering its three-dimen-sional shape and motion (figure below).

The performance appears more robustthan previous systems and the informa-tion derived from the motion estimationcan be used for both expression recogni-tion and the animation of 3D graphicsmodels of faces.

The motion of the eyes is of particular in-terest in human-computer interfaces. Dr.Myron Flickner of the IBM AlmadenResearch Center described their “BlueEyes” project for detecting and trackingeyes using infrared light and a video cam-era (figure below, left). By locating the pu-pil and reflections on the cornea, they areable to track the human gaze accurately.

To explore new perceptual interfaces,IBM’s new robot Pong can detect facesand facial expressions as well as expressthem (figure below, right). Pong hasspeech recognition and synthesis softwarethat lets it interact with a user while acamera hidden in its nose detects facesand analyzes facial expressions. Dr. Flick-

“real-time” tracker (figure above) that candetect multiple people from a single cam-era, determine how many people are inthe scene, and track them even when theypartially occlude each other.

Dr. Jim Rehg talked about related workat Compaq’s Cambridge ResearchLaboratory that uses body tracking in a“smart kiosk.” This kiosk uses video cam-eras to detect people that are near it andtrack their motion (figure at left). A dis-play screen has a realistic-looking humanface that talks to people and encouragesthem to approach the kiosk. An installa-tion of the kiosk in the Cybersmith Cafein Harvard Square allowed Compaq to as-sess the potential market for kiosks withsensing capabilities. This installation alsolet Compaq experiment with different fea-tures and evaluate this type of perceptualuser interface. The researchers observedthat the interactive talking kiosk drewpeople to it and that high-quality contentwas the most important factor in holdingtheir attention. Furthermore, the experi-ment revealed that interactive entertain-ment was more compelling in thisapplication than information services. Toimprove the interaction with the kiosk,Dr. Rehg’s recent research is focusing onprobabilistic methods for detectingwhether or not a person in front of a cam-era is actually speaking by analyzing themotion in the video in conjunction withthe auditory signal.

While the above systems focused prima-rily on detecting and tracking people ascrude “blobs,” many types of human-to-human interaction exploit detailed infor-mation about the face and hands.Recognizing information about fa-cial expression requires an analysisof facial motion, which is challeng-ing due to the complex deformationsthe human face can undergo. Dr.Matthew Brand of the Mitsub-ishi Electric Research Labora-tory (MERL) described a novelmathematical formulation of thismotion estimation problem thatuses matrix algebra to propagateuncertainties in image measure-ments in such a way that informa-tion loss is minimized and optimalestimates of the facial motion can be de-

Mitsubishi Electric ResearchLabs. 3D facial motion tracking

with uncertainty

IBM Almaden Research. l, BlueEyes gazetracking; r. lifelike Pong robot

Compaq CambridgeResearch Laboratory.Smart kiosk installed in

Cybersmith Cafe

conduit! 19

ner gave a live demonstration of Pongand talked about other projects at IBMexploring “attentive” user interfaces thatcombine physiological measurements,eye gaze, and body language that theyhope will make computers more widelyuseful.

The above methods have onesimple thing in common thatdistinguishes them from tra-ditional interfaces—the visualprocessing is computationallyintensive. Intel is in the busi-ness of selling computation, sotheir interest in these newperceptual interfaces is notsurprising. Dr. Gary Brad-ski from Intel’s Micropro-cessor Research Labspresented their efforts to de-velop an open source com-puter vision library to supportthe growth of novel applica-tions. This library of visionsoftware provides a hugenumber of basic and advancedmethods that support every-thing from low-level segmen-

tation to full-body tracking andprobabilistic action recognition(figure at left).

While the form of future interfaces can-not be predicted, it is clear that many ofour partner companies believe that theywill change radically and that they willexploit new perceptual capabilities. Inparticular, this workshop explored com-puter vision technologies that might givecomputers information about the visualworld occupied by their users. With theactive research on this problem in aca-demia and industry, it is a fertile area forpartnerships like those supported by IPP.

For more information about these pro-jects, see:

Microsoft, EasyLiving project: http://re-search.microsoft.com/easyliving/

Compaq, Smart Kiosk: http://crl.re-search.compaq.com/projects/kiosk/de-fault.htm

MERL, facial motion: http://www.merl.-com/people/brand/

IBM, Blue Eyes http://www.almaden.-ibm.com/cs/blueeyes/

IBM, Pong http://www.almaden.ibm.-com/almaden/media/image⁄pong.html

Intel, Open Source Computer Vision Li-brary: http://www.intel.com/research/mrl/research/opencv/

Got you to look, didn’t I? I promise I’ll getto this subject at the end. Maybe youwant to just jump right there—go ahead!

BACKGROUNDThe TeachScheme! Project was conceivedin frustration. Our group at Rice Univer-sity had spent several years overhaulingRice’s introductory curriculum. Rice hadadopted an approach based on Abelsonand Sussman’s seminal work at MIT but,despite its sublime content, it wasn’t do-ing the job for us: it provides immense in-sight into structuring systems but, wefelt, gave little insight into structuringprograms.

In response, Matthias Felleisen, CorkyCartwright and their students (includingme) built a new curriculum for introduc-tory programming. Like Abelson andSussman, we steered clear of the naggingterminology and low-level details thatdog so many introductory programmingtexts. Unlike them, we stressed the sys-tematic construction of programs fromcrisp and accurate definitions of data—the stuff of OOP, in other words, withoutthe verbiage and overhead of OO. This letus take students with no prior program-ming experience through all the interest-ing topics—lists, trees and other usefuldata types, higher-order functions andnon-local control, as well as the routinestuff of scope and mutation—with a veryparticular emphasis on program design.

Over the years, three features havedriven our success.

HOW TO CREATE THE BEST OO PROGRAMMERS

ShriramKrishnamurthi

Intel. Human activity analysisfrom the Intel Open Source

Computer Vision Library

conduit! 20

1. We developed a methodical response tothat most frustrating of questions, “Canyou help me find my bug?” Our work pre-sents a six-step design recipe that beginswith an understanding of data, then pro-ceeds to understanding the black-box be-havior of the function, then to deriving itstemplate from the data. Only in the fifthof the six steps does the student actuallywrite the function. (The last, in caseyou’re wondering, is testing.)

This methodology not only gave our TAsand us a grading rubric, it also gave stu-dents a valuable debugging checklist. Toour surprise, we found that most studenterrors had little to do with their code:their errors were at a much earlier phase,usually in defining the structure of theirdata. If you don’t understand the data youhave, you obviously can’t hope to processit very well. We suddenly developed amuch more sophisticated understandingof student errors (a big help when plan-ning what to revisit in lecture, for in-stance). For their part, students now hada much more useful debugging methodol-ogy than making random syntacticchanges and putzing about in a debugger.

2. We built a beginner-friendly program-ming environment, DrScheme (“DoctorScheme”). DrScheme is an outgrowth ofseveral years of observing how studentsprogram in labs. Its innovations are thefruit of several PhD theses. From a peda-gogic perspective, one feature towers overothers. It exposes the fundamental mis-match between texts, which provide astratified view of languages, and environ-ments, which toss the student into a lin-guistic mass for which they’re oftencompletely unprepared. DrScheme in-stead provides language levels that a stu-dent can increment as her skills grow. Inparticular, each language level is carefulto provide feedback using only the lan-guage and terminology a student is ex-pected to know at that level. In contrast,pretend you’re in your first week as a pro-gramming student, type ‘wage * hours= salary’ into a C++ environment, andmake sense of its response. (We’ve ob-served high-school students nearly re-duced to tears from trying to decipherrepeated feedback of this sort.)

3. We exploited the multiplatform, graphi-cal nature of DrScheme to build numer-ous libraries to support interestingexercises. The libraries help students

When asked to elaborate on some jpegs of him windsurfing, Tom Deansaid, “You can ask Keith Hall and Stu Andrews. I walked down to thebeach a few blocks from our house on a Saturday a couple of weeks backand found Keith and Stu throwing a frisbee, with Stu’s windsurfer lying

nearby. Stu asked me if I wantedto give it a try and I said sure. Iused to do quite a bit of wind-surfing, having planned vaca-tions around wind-surfing atMaui, Columbia River Gorge,the Sea of Cortez in Baja Mexicoand numerous other windsurf-ing meccas. My short stint onStu’s board was the second timeI’ve windsurfed in three years. Iwas lousy but Stu and Keithwere supportive.”

Keith is a fourth-year PhD stu-dent, Stu is in his third year.

conduit! 21

write simple games, build very rudimen-tary animations (no, no, not like Andy’s orSpike’s!), try a little Web programming,create file-system managers, and so on.The libraries plug into our Scheme’sgraphics and systems APIs, but studentswrite all the critical control elements.Most of these are extended exercises thatgrow in complexity, letting students peelaway layer upon layer. In short, we ex-pose them to two important software en-gineering principles—model-view-con-troller designs and iterative refinement—all while they’re having fun!

THE EARLY DAYSWe’ve taught this material since 1994 atRice, and it’s been a great success. Enroll-ments skyrock-eted; perhapsmore impor-tantly, the num-ber of womengrew dispropor-tionately. Manywho’d taken pro-grammingclasses in high school (especially women)said they enjoyed the more structuredand goal-driven curriculum, in contrast tothe seemingly random activity they’d ex-perienced before. When students mas-tered even our rapid pace of material, wefried their brains in bonus lectures.

One small group of high school students,however, always resisted this material.Over time, we came to realize most ofthese were students who’d had AdvancedPlacement computer science (AP CS)courses. Most AP students recognizedthat, by about the sixth week, our coursewas in terrain they found completely un-familiar. A small group, however, resistedthe call to treat the material at its ownlevel of abstraction. Common questionsand complaints included, “Isn’t that just alinked list?” “Where’s the null pointer?”“Why won’t the number overflow?” “Isn’trecursion bad for the stack?” “This doesn’tbehave like a computer.” What amazed uswas not simply how frequently these stu-dents complained, but how misinformedthey were.

When we eventually set to studying theAP CS material and talking with high

school teachers, we realized why thesestudents had these views. The AP CS ma-terial has a lowest-common-denominatorfeel; indeed, almost uniquely among theAP disciplines, many leading universities(including Brown and Rice) don’t recog-nize scores in this subject. This lack ofrecognition stems, in part, from a hugedisparity in depth between the highschool and college levels.

Despite the best intentions of its framersto remain at a principled level, we foundthat teachers in the field were forced todeal with a myriad of low-level details ofsyntax and machine organization, to thepoint where most never got to useful datastructures or other concepts. Many teach-ers reported that material we’d consid-

ered routine—programmingover trees, for in-stance—was pre-sented gingerly,at the end of twosemesters, andonly the best stu-dents were ex-

pected to get it absolutely right; mostnever extracted themselves from the longdark night that is debugging. In contrast,we considered it a failure if virtually ev-ery student, aided by the design recipeswhere necessary, didn’t nail this material.

Matthias Felleisen then asked the ques-tion we hadn’t dare pose: Why can’t weteach this material in high schools?What’s the real difference between a high-school student and a first-year college stu-dent anyway, save for a summer of frolic?(Not our students, who’re too high-mindedto spend their summers that way, butmaybe someone else’s.) Despite what ourviewbooks would like to suggest, studentsaren’t magically transformed when theywalk through a Van Wickle Gate or a Sal-lyport. So why not take this to highschools?

GROWTHYep: we had no idea what we were in for.We were used to college educators, towhom you can mail a book with a friendlynote. But high school teachers—especiallyin computer science—are so overworked,it’s utterly unfair (and impractical) to

“Enrollments skyrocketed;perhaps more importantly,

the number of women grewdisproportionately”

conduit! 22

simply mail them a manuscript. (Com-puter science professors joke about beingasked to fix printers; for teachers, it’s partof their job.) In the furious market of thelate 1990s, many computer science teach-ers left for industry; the ones remaininghad no training in the discipline at all(and they were teaching C++!). They werevaliantly keeping their classes alive, butlearning a whole new approach, unaided,was out of the question.

In the summer of 1996, therefore, webrought in a high-school teacher for a fewdays of intensive training. We had no ideawhat she knew or how to teach her, and Iimagine those few days were pretty roughon her. But she survived it, so the nextyear we brought in two. (Exponentialgrowth! We had visions of Malthus.) Asour first big test, in 1998 I recruited agroup of 18 teachers from Texas comput-ing conferences to attend a week-longsummer workshop at Rice. Looking back,we still knew very little about our audi-ences; it’s a wonder that many of thoseteachers are still with us. (What doesn’tkill us makes us stronger.)

Suddenly, TeachScheme! had become real.We were conducting school visits, observ-

ing how students learned C++ andScheme. Somewhere along the way, wemade a semantic shift: our on-line “lec-ture notes” grew into a “book.” Havingtrained the teachers, we became victimsof our limited success: as teachers beganto use the material in classes, we wereobliged to support them. We had to writeand publish solutions. We began to writea programming environment user’s guide.We added new libraries and exercises. Werevised the language levels. The codebasegrew to over half a million lines.... In2000, our demand grew too large to fitinto one workshop, so we ran two. Wedrew teachers from as far afield as Mex-ico, Saudi Arabia, Switzerland and Japan.(Okay, so maybe Mexico isn’t all that farfrom Houston.) We also began to benefitfrom the diaspora of PhDs: Matthew Flattran a small workshop at his new home,the University of Utah.

This year, the workshops themselves wentnational. We ran workshops in four differ-ent states, training nearly a hundredteachers. The largest, hosting nearly fortyteachers, took place in July, 2001 atBrown, run jointly by my colleague, KathiFisler, of Worcester Polytechnic, and me.

CARPODACUS MEXICANUS ON 5TH FLOORGrad student Jasminka Hasic played host to a family of house finches during thespring/early summer. The pair began building their nest atop the open window of her5th floor office; consequently, the window had to remain open come wind and weather.In order to discourage the birds from flying into the room and becoming disoriented orhurt, Jasminka, herself a newly hatched American citizen, used her giant-sizedAmerican flag to cover the window and darken the room. The nest proved a remark-able, though messy, feat of engineering, teetering on the edge of oblivion. The femalefinch laid four eggs, all of which hatched. The highly vocal babies were pretty dis-tracting at feeding time, but very charming. Despite concern that the babies’ firstattempt at flight would be their last, each one fledged successfully.

conduit! 23

We, of course, merely soaked up thecredit; our helpers made it work smoothly.We had three great returning teachersand five superb students: three PhD stu-dents from Brown (Greg Cooper, RobHunter and Dave Tucker), and one eachfrom Cornell and Northeastern. This stu-dent group makes us feel like we’ve takenthe first step towards closing an educa-tional cycle.

One of the major attractions of our curric-ulum is that it costs teachers, studentsand schools nothing. We distribute soft-ware and support material free on theWeb. Our text, How to Design Programs,is published in hardback by the MITPress, but we asked to be able to distrib-ute it free on the Web, and they gener-ously agreed. Some poorer schools, both inthe U.S. and abroad, use the book entirelythrough this medium. Finally, with gener-ous support from the National ScienceFoundation, U.S. Department of Educa-tion, Exxon and CORD, an educationalnonprofit that’s adopted our curriculumfor its Academies of Information Technol-ogy, we pay expenses and stipends to vir-tually all teachers who attend workshops.

OOPWe’re (fortunately) lowering the exponentin our growth curve. More importantly,we’re seeing signs of the change we setout to accomplish. A growing populationof teachers recognizes that there’s an al-ternative to what they teach. Our “repeat

customer” ratios (teachers who use thecurriculum more than once) are huge. Al-most every week, we hear from either ateacher who struggled before who sayshe’s thriving now, or one who was comfort-able before but whose students are nowbuilding projects beyond her expectations.Perhaps one way to see that Teach-Scheme! has come of age is that promi-nent universities other than our own,both in the U.S. and abroad, have adoptedour curriculum.

One reason for TeachScheme!’s growinguniversity adoption is its superb conduitto true object-oriented programming.Corky Cartwright at Rice worked with usto revamp the second course to teach stu-dents algorithms and design pattern-based programming in Java. The beautyof this scheme is that once students haveinternalized the TeachScheme! designrecipe, it takes just about a day to startproducing authentic Java! Scheme easilymorphs into code obeying the Interpreterand Composite patterns; their Scheme ex-perience with higher-order functions hasprepared them well for inheritance-basedabstraction; even the Visitor pattern is aneasy step away. Students get throughmost of the Gang of Four book, in additionto the usual algorithmic material, in theirsecond semester.

A few high-school and college facultywho’ve adopted this route are finding thattheir students internalize OOP betterthrough this two-step approach than theydid starting with Java. By the time theyconfront their first ‘public staticvoid main’ the students are alreadymasters of useful data structures and sev-eral introductory algorithms. Having al-ready learned one language, the secondone comes a lot quicker—because theScheme we teach maps easily to Java. Thedesign recipes entirely take the mysteryout of recursion (sorry!) in a way that noprior approach we know of has. In short,there’s no excuse for not teaching chal-lenging and interesting topics.

People sometimes ask me whether func-tional programming isn’t unintuitive; per-haps students should really only see it as(college) juniors. My evidence to the con-trary is simple. I point to hundreds of stu-dents, starting from as young as the 7thgrade, at schools big and small, poor andrich, in the U.S. and in several foreigncountries. These students write rich, in-teresting and even fanciful programs. Wehope it isn’t just the language: the envi-ronment certainly helps, and teachers tellus the design recipe wins big, too. But thebottom line is, while functional program-ming may not be intuitive to professors, itsure is to kids. (And it seems to be in-creasingly intuitive to programmers in ev-erything from JavaScript to Python—astheir programmers hit on the same prob-lems functional programmers did two de-

“One reason for TeachScheme!’sgrowing university adoption is itssuperb conduit to true object-

oriented programming”

conduit! 24

cades ago, their languages have rapidlyexpanded in their functional offerings, asnumerous articles on these languages at-test.)

THE FUTUREWhat’s next? We’re building many moreextended exercises and improving thequality and features of DrScheme. Theseimprovements induce difficult researchproblems behind the scenes, with the ben-efit (or danger, if you don’t like the atten-tion!) that the solution will immediatelydownload to thousands of desktops in thenext release. The challenge we face as re-searchers and programmers is the sameone that actors face: always to stay incharacter. How do you build a powerfultype-inference engine that doesn’t baffle a10th grader? Stay tuned!

But I think much more is at stake. I’ve be-lieved for some years now that computerscience has come to reside at the heart ofa true liberal arts curriculum. What mat-ters isn’t the dust and noise: the details ofword sizes and endianness. Rather, it’sthe notion of computation, what Abelsonand Sussman called a procedural episte-mology. This is why computer science hasgone from bridesmaid (think of thoseearly number-crunchers) to guest of honor(think biology).

Our curricula must reflect this shift. Weneed to give students more than just the

syntactic details of some language; wemust train them to harness this beastcalled computation. If anything, the non-majors need it more than the majors. Inone semester (if that’s all they’ll give us),we must teach them principles they willrecognize for decades to come. To do this,our courses have to get at the heart ofcomputation as quickly as possible andstay there as long as they can. Peoplemust internalize these ideas so well thatthey recognize them in whatever theychoose as a concentration.

Brown is leading the way here. Last year,Lisa Cozzens ’01 completed a senior hon-ors thesis under my supervision in whichshe built several extended exercises im-plementing basic processes from biology.That material caught the eye of a new au-dience: biology teachers. Two of them at-tended the workshop at Brown this pastsummer. Now they’ve begun to teach pro-gramming to their students, too. The rev-olution has begun.

Acknowledgments: Thanks to my part-ners in crime, especially Matthias Fel-leisen, Robert Bruce Findler, MatthewFlatt and Kathi Fisler. Thanks also to theother TeachScheme! staff, and to theteachers who’ve endured us!URLs: The Project:http://www.teach-scheme.org/The Text:http://www.htdp.org/

fac.activities@cs.brown.edu

Michael Black. In the spring Michaelorganized an Industrial Partners Pro-gram Symposium on Vision-based Inter-faces (details in this issue).

He has recently been awarded two grants:an NSF ITR grant on “The Computer Sci-ence of Biologically Embedded Systems,”(a collaboration with Elie Bienenstock inApplied Math and John Donoghue inNeuroscience) and an ONR grant to study“Motion Capture for Statistical Learningof Human Appearance and Motion” aspart of the DARPA Human-ID project.

In July Michael attended the Interna-tional Conference on Computer Visionheld in Vancouver. He was one of the AreaChairs for the conference and his stu-dents presented two papers. HedvigSidenbladh presented a paper on “Learn-ing Image Statistics for Bayesian Track-ing,” while Fernando De la Torre talkedabout “Robust Principal Component Anal-ysis for Computer Vision.” Along withDavid Fleet from Xerox PARC, Michaelhad an invited paper, “Probabilistic De-tection and Tracking of Motion Bound-aries,” in the Distinguished Paper Track

conduit! 25

at the International Joint Conference onArtificial Intelligence held in Quebec dur-ing August.

During a busy spring of travel, Michaelgave an invited talk on human motiontracking at Workshop on the Convergenceof Vision, Video, and Graphics in Berkeleyand talks on brain-computer interfaces atthe Microsoft Research Vision Symposiumand at the Workshop on Vision-Based Per-ceptual Interfaces hosted by the Interac-tive Institute in Stockholm, Sweden. Hewas back in Sweden in August to give aninvited talk at the 2001 Stockholm Work-shop on Computational Vision, held on anisland in the Stockholm archipelago. Ad-ditionally, he gave colloquia on humanmotion analysis at the University of Roch-ester, the University of Western Ontario,the University of Pennsylvania, and NewYork University.

This fall Michael is teaching a new courseon brain-computer interfaces that isdrawing curious students from depart-ments such as physics, neuroscience, ap-plied math, cognitive and linguisticsciences, and engineering, as well as com-puter science.

Eugene Charniak. Eugene presentedhis paper “Immediate Head Parsers forLanguage Modeling” at the 2001 Associa-tion for Computational Linguistics (ACL)Conference, this year held in Toulouse.Eugene’s technical article in the last con-duit! was based on this paper. The ACLconference is generally considered to bethe top conference in this area, and thisyear for the first time it awarded a “bestpaper” award. Eugene’s paper shared theaward with one other group.

Tom Dean. Tom spent half of August inSeattle, attending meetings of the boardof trustees for IJCAI Inc., giving talks atIJCAI-related and satellite meetings andworkshops including an invited talk atATAL-2001 (Agent Theories, Architec-tures and Languages), and trying to run

the department and CIS from his hotelroom. (Tom was appointed interim VicePresident for Computing and InformationServices as of July 1; the appointmentruns through next June, at which point hehopes to turn over the job to a permanentCIO.)

Amy Greenwald. Like last year, oneof the highlights of Amy’s summer washer participation in the Distributed Men-tor Program, sponsored by the ComputingResearch Association. Her mentorees,Victoria Manfredi and Julia Farago, vis-ited from Smith College and HarvardUniversity, respectively. In addition, Amytraveled from eastern to western Canada.In May, she presented a paper at Autono-mous Agents in Montreal. In August, shechaired a workshop entitled “EconomicAgents, Models, and Mechanisms” in Se-attle, from where she took a backpackingholiday in British Columbia (a.k.a. BCa.k.a. Bug City). And in the middle of thesummer, Amy attended ICML (Interna-tional Conference on Machine Learning)at Williams College with David Gondekand Keith Hall, both PhD students intheir fourth year.

Shriram Krishnamurthi. In addition tohaving the usual amount of fun, Shriramran a successful TeachScheme! workshopat Brown (see article on page 19). He andKathi Fisler of WPI hosted nearly 40 highschool teachers from around the countryand abroad, training them in an innova-tive computer science curriculum. He hasmore at http://www.teach-scheme.org/.

David Laidlaw. David has been award-ed an NSF CAREER grant, the agency’smost prestigious award for junior faculty.His work under this grant will focus onshape modeling and its applications cou-pled with the development of a methodol-

▼▼▼

▼▼▼

▼▼▼

▼▼▼▼▼▼

conduit! 26

ogy for teaching the skills needed forsuccessful multidisciplinary researchprojects. The education plan consists ofDavid’s course, ‘Interdisciplinary Scien-tific Visualization,’ and a research group;both are aimed at undergraduates andgraduate students alike. The research ef-fort includes development of computa-tional tools for capturing geometry,representing it within the computer, andusing those representations for specificapplications in archaeology and biologicalmodeling.

Nancy Pollard. Nancy was recentlyawarded an NSF CAREER grant on“Quantifying Humanlike EnvelopingGrasps.” Nancy’s work under the grantwill focus on quantifying humanlike en-veloping grasps with the goal of creatingcredible hand use for digital characters.Producing realistic digital humans hasbeen called the last frontier in the marchtoward graphical realism, and Nancy be-lieves that the last frontier in creatingdigital humans is generating believablehand motion. In pursuit of this goal, sheproposes a tendon-based quality measurefor humanlike enveloping grasps, and sheplans to evaluate this quality measure (1)for ability to discriminate betweengrasps, (2) as a predictor of grasp forces,and (3) for use in modeling grasp acquisi-tion. Because of the strong emphasis onhuman anatomy, this research has the po-tential for additional impact outsidegraphics and animation in areas includ-

ing ergonomics (tool design), robotics (ro-bot hand design), and anthropology(research in human hand evolution andtool use).This summer she was invited to attendthe first Asia-Pacific Advanced StudiesInstitute, with the theme “New Frontiersof Intelligent Robotics,” in Tokyo, Japan.She is on junior sabbatical leave this year,and will be spending much of the year atCarnegie Mellon University in Pitts-burgh. She and her students will be usingthe motion-capture lab at CMU and work-ing with Jessica Hodgins’ group on prob-lems related to realistic animation ofhuman motion and control of robots frommotion-capture data.

John Savage. John chaired the Inau-gural Faculty Program Committee thatassembled 20 exciting panels reporting onfaculty research at President Ruth Sim-mons’ inauguration this October. He iscurrently Chair of the Faculty and is nowin his second year as an elected officer ofthe Faculty. As of January 1, John will bewithdrawing as Director of the IndustrialPartners Program to be replaced byMichael Black; however, he will workwith Michael over the next year to bringhim up to speed.

Eli Upfal. Eli was on the program com-mittee of the 13th International Sympo-sium on Fundamentals of ComputationTheory in Riga, Latvia, and the VIII In-ternational Colloquium on Structural In-formation and Communication Com-plexity in Spain. He participated in aDagstuhl meeting on “Design and Analy-sis of Randomized and Approximation Al-gorithms” in June and was an invitedspeaker at a workshop in honor of AllanBorodin’s 60th birthday at the Universityof Toronto, also in June.

▼▼▼

▼▼▼

▼▼▼

▼▼▼

The 40 TeachScheme! workshop participantsgather for a group shot outside the CIT building

conduit! 27

David Laidlaw, John Hughes andAndy van Dam. Brown presented fourpapers at this year’s Symposium on Inter-active 3D Graphics (John Hughes was theprogram co-chair with Carlo Sequin ofU.C. Berkeley, and Science and Technol-ogy Center collaborator Mary Whittonwas the general chair). Joe LaViola (Ph.D.student of Andy van Dam) presentedwork that he did with Daniel Acevedo Fe-liz and Daniel Keefe (Ph.D. students ofDavid Laidlaw) and Robert Zeleznik(graphics staff) presented work on“Hands-Free Multi-Scale Navigation inVirtual Environments.” Then Dan Keefepresented a paper on “Cave Painting,” ajoint project with Daniel Acevedo Feliz,Tomer Moscovich (Ph.D. student of JohnHughes), David Laidlaw and Joe LaViola.In Cave Painting, an artist uses virtualbrushes and paints to create an artworkthat lives in the 3D space of the four-wallCave.

Michael Kowalski (graphics staff) pre-sented a paper on “User-Guided Composi-tion for Art-Based Rendering,” joint workwith John Hughes, Cynthia Beth Rubin(of RISD) and Jun Ohya of ATR Researchin Japan.

Finally, Takeo Igarashi, a postdoc in thegraphics group, presented his work withDennis Cosgrove of CMU on “AdaptiveUnwrapping for Interactive TexturePainting” —work inspired by a desire topaint colors onto his now-famous “Teddy”models.

Andy van Dam. Andy participated intwo 60th birthday celebrations: On May29 he gave the keynote address “User In-terfaces: Disappearing, Dissolving, andEvolving” at the Celebration Colloquiumfor Prof. José Encarñaçao, founder and di-rector of Fraunhofer’s Graphics ResearchInstitute and a faculty member at theTechnical University of Darmstadt. In Oc-tober his paper “Reflections on Next-Gen-eration Educational Software” was pub-lished in a book honoring the career ofProf. Bernard Levrat, computer scienceprofessor at the University of Geneva andpresident of the Swiss Virtual Campus. Inaddition, he continued his own celebra-tion of time by hiking the heights and thedepths of the earth—the Alps and theGrand Canyon.

Stan Zdonik. Stan was co-presenter atthe 27th Very Large Database Conference(VLDB) in Rome of a half-day tutorial onData Management for Pervasive Comput-ing.

The National Science Foundation has an-nounced 309 awards designed to preserveAmerica’s position as the world leader ofcomputer science and its applications.CS’s five newly funded projects were se-lected from over 2000 competitive pro-posals.

Michael Black’s grant is for his work on“The Computer Science of Biologi-cally Embedded Systems.” Biologicallyembedded systems that directly couple ar-tificial computational devices with neural

systems are emerging as a new area of in-formation technology research. The physi-cal structure and adaptability of thehuman brain make these biologically em-bedded systems quite different from com-putational systems typically studied inComputer Science.

Fundamentally, biologically embeddedsystems must make inferences about thebehavior of a biological system based onmeasurements of neural activity that areindirect, ambiguous, and uncertain. More-over, these systems must adapt to short-and long-term changes in neural activityof the brain. These problems are ad-dressed by a multidisciplinary team inthe context of developing a robot arm thatis controlled by simultaneous recordings

FIVE CS FACULTY AWARDEDNSF’s ITR GRANTS

▼▼▼

▼▼▼

conduit! 28

from neurons in the motor cortex of a sub-ject. The goal is to model probabilisticallythe behavior of these neurons as a func-tion of arm motion and then reconstructcontinuous arm trajectories based on theneural activity. To do so, the project willexploit mathematical and computationaltechniques from computer vision, imageprocessing, and machine learning.

This work will enhance scientific knowl-edge about how to design and build newtypes of hybrid human/computer systems,will explore new devices to assist the se-verely disabled, will address the informa-tion technology questions raised by thesebiologically embedded systems, and willcontribute to the understanding of neuralcoding.

Eugene Charniak’s ITR grant is forwork on “Learning Syntactic/Seman-tic Information for Parsing.” The pro-ject is so named because the structural in-formation to be learned often falls at theboundary between syntax and semantics.For example, is the fact that “Fred” is typ-ically a person’s first name a syntactic orsemantic fact? Does the fact that the“New York Stock Exchange” has as part ofits name the location “New York” fall un-der syntax or semantics? What about thesimilarity between the expression “[to]market useless items” and “the marketfor useless items”? These are some of thetopics that come up in this research.

As for the “for parsing” portion of the ti-tle, the intention is to learn the abovekinds of information in a form that cur-rent statistical parsers can use so thatthey can output more finely structuredparses. However, this is not meant to sug-gest that parsing is the sole use for thissort of information—exactly the oppositeis the case. For example, more and moresystems for automatically extracting in-formation from free text use coreferencedetection and “named-entity recognition”(e.g., recognizing that “New York” is a lo-cation but “New York Stock Exchange” isan organization). There is evidence tosuggest that both coreference and named-entity recognition can be improved withthe finer level of analysis to be made pos-sible by this research. Or again, “lan-guage models” (programs that assign aprobability to strings in a language) are

standard parts of all current speech-rec-ognition systems. There is now evidencethat suggests that finer-grained syntacticanalysis can improve current languagemodels. Thus this research will enable awide variety of systems to make betteruse of language input and thus makethese systems more accessible to a diverseuser pool.

Eli Upfal has been awarded a five-yeargrant for his work with Harvard’s MichaelMitzenmacher on “Algorithmic Issuesin Large-Scale Dynamic Networks.”They will develop a theoretically well-founded framework for the design andanalysis of algorithms for large-scale dy-namic networks, in particular, for the Weband related dynamic networks, such asthe underlying Internet topology and In-ternet-based peer-to-peer ad hoc net-works. We plan to develop rigorous math-ematical models that capture key charac-teristics and can make reliable predic-tions about features such as connectivity,information content, and dynamic of thesenetworks. We plan to apply this frame-work to test existing algorithms and con-struct improved new algorithms.

The main benefits of developing the math-ematical models of the Web structure anddynamics will be the improved theoreticalfoundation for the design, analysis andtesting of algorithms that operate in theWeb environment. The tangible results ofthis work will be models that can be sub-jected to experimental verification, analy-ses of algorithms based upon thesemodels, new algorithms that benefit fromthese analyses, and, finally, proof-of-con-cept demonstrations and experimentalevaluations of such algorithms.

Pascal Van Hentenryck has beenawarded a large ITR grant, over fouryears, for work in conjunction with EliUpfal and researchers at MIT and Geor-gia Tech on “Stochastic combinatorialoptimization.” There are many real opti-mization problems for which no solutionsare known. For instance: a snowstorm isapproaching Chicago and United Airlines,at its Operations Center, must plan howto cancel and reroute its flights. Althoughsubstantial information on weather fore-casts, plane and crew status, passengeritineraries, and hotels is available elec-

conduit! 29

tronically, this information is not ex-ploited in a scientific way. Instead, hu-mans make ad-hoc decisions based onlyon their experience. Why is this the case,especially when the airline is a very so-phisticated user of optimization for plan-ning? Because the application is a large-scale stochastic combinatorial optimiza-tion problem for which no known algo-rithm produces good solutions in reason-able time. Our fundamental research inthis relatively unexplored area will havetwo complementary thrusts: (1) exploitingthe orthogonal strengths of constraintand mathematical programming to tacklethe hard combinatorial problems arisingin stochastic optimization (e.g., multi-stage or Monte Carlo approaches) and (2)studying stochastic combinatorial sub-structures that are amenable to efficientsolutions or approximations.

Andy van Dam’s three-year grant,“Electronic books for the tele-immer-sion age: a new paradigm for teach-ing surgical procedures,” will directlyinvolve researchers from the University ofNorth Carolina, Chapel Hill, as well asBrown and indirectly researchers fromUPenn, working with those from UNC onrealtime 3D model acquisition and recon-struction. Their work on tele-immersionwill provide a dramatic new medium forgroups of people remote from one anotherto work and share experiences in an im-

mersive 3D virtual environment, much asif they were co-located in a shared physi-cal space. Immersive “time machines” willadd a further important dimension, thatof recording experiences in which a view-er, immersed in a 3D reconstruction, canliterally walk through the scene or movebackward and forward in time. This workwill focus on a societally important andtechnologically challenging driving appli-cation, teaching surgical management ofdifficult, potentially lethal injuries.

They will develop a new paradigm forteaching surgical procedures: immersiveelectronic books that in effect blend atime machine with 3D hypermedia. Theirgoal is to allow surgeons to witness andexplore (in time and space) a past surgicalprocedure as if they were there, with theadded benefit of instruction from the orig-inal surgeon or another instructor as wellas integrated 3D illustrations, annota-tions, and relevant medical metadata.The trainee should be able freely and nat-urally to walk around a life-sized, high-fi-delity 3D graphical reconstruction of theoriginal time-varying events, pausing orstepping forward and backward in time tosatisfy curiosity or allay confusion. Theresearchers will bring together experts inthe respective disciplines and leveragetheir prior work in tele-immersion in or-der to achieve these goals.

Tele-immersion technology will let surgeons-in-trainingmove naturally within a life-sized, high-fidelity, 3D graphicalreconstruction of the surgery, pausing or stepping forward

and backward in time to assist in learning

conduit! 30

I was recently invited to participate in apanel discussion with the title “Is it ArtYet?” The main panelist was a composer,David Cope, who has written a computerprogram that composes music in stylesranging from Bach to Stravinsky that hasfooled experts. The question for the panelwas captured by the title.

I would say that my performance on thepanel was forgettable if I could only re-member what I said. Actually, most of theother panelists were only so-so as well.But fortunately our invited guest, Profes-sor Cope, was great. He is a funny, articu-late guy, with a combination of down-to-earth personality and controversial opin-ions that make him ideal for a panel dis-cussion. But what I found most intriguingwas the personal story behind the cre-ation of this program.

It seems that in the early eighties Prof.Cope received a commission for an opera.He received (and spent) the advance, butunfortunately suffered a major composer’sblock. Finally, after a long period of time,he hit upon the idea of creating a com-puter program to compose for him. So hetaught himself Lisp and AI (I was pleased

to learn that he used Drew McDermott’sand my textbook) but was not happy withthe result, which did not sound very muchlike his music.

He then did what any good scientist doeswhen confronted by a too-difficult prob-lem—he found a simpler one. He figuredthat while he was not sure what proper-ties made his music sound like “him”, hedid know the properties that made Bachsound like Bach and Mozart like Mozart.The program he eventually came up with,however, was not restricted to any partic-ular composer. Rather, one gives it a data-base of musical works, and the programtries to discover the similarities and dis-similarities and then use them to createmore music with the same features.

Eventually Professor Cope finished thecommission. The work, he says, is about60% him, 40% computer. However, fromhis point of view all of the work is his. In-deed, these days it can be hard to tellwhere one leaves off and the other begins.Now when he composes he gives the pro-gram a database of his own work and af-ter composing a bit asks the machine tosuggest, say, what the next two measuresshould be. If he does not like the sugges-tion he can keep asking for other alterna-tives.

EugeneCharniak

CHARNIAK UNPLUGGED

Takeo Igarashi, a postdoc in the Graphics Group here working with John Hughes, hit the worldwide news October18. The BBC News Online, no less (http://news.bbc.co.uk/hi/english/sci/tech/newsid_1606000/1606175.stm), picked upa system he’s to present at the ACM UIST conference in Orlando in November. What’s more, slashdot.org (http://slash-dot.org/article.pl?sid=01/10/18/1245232&mode=thread), which calls itself “News for Nerds” and is read, John Bazik

says, by “millions of geeks like me,” just grabbed the story and featured it; it’s now exciting lots ofcommentary, both insightful and amusing, in slashdot’s feedback section.

Takeo’s innovative idea is to use simple human sounds, like grunts and sighs, for controlling com-puters. Conventional voice-recognition software is still not accurate or efficient enough for generaluse in an interface. Takeo’s system works by measuring the pitch and duration of grunt-like soundslike “ah” and “umm.” Thus, Takeo suggests, in scrolling through a document on the Internet youcould say “move down, ahhhh”: the document would continue scrolling as long as the sound contin-ued. You could increase the scrolling speed by raising the pitch of your voice, and stop scrolling al-together by stopping speaking. It’s much easier to Undo a command with a quick “uh oh” than with

a mouse!

Said Takeo of his system, “I personally do not think this is useful in of-fice environments—it is really annoying for office mates! I basically de-signed it for isolated situations such as controlling computers whiledriving a car or rock climbing. I also think the technique is ideal for en-tertainment applications. I implemented a simple video game using thetechnique, and children love it.” You can find Takeo’s paper and demovideo at his homepage, http://www.mtl.t.u.-tokyo.ac.jp/~takeo/research/voice/voice.htm.

conduit! 31

His major work at the moment is not inhis own modern idiom but rather an op-era about the life of Mahler, written (withthe help of his program) in the style ofMahler. Mahler, besides being a composer,was an active conductor, and although hefrequently conducted operas, he neverwrote one. David Cope intends to correctthis historical mistake.

I teach CS002, our department’s com-puter literacy course. During the semes-ter the students, among other things,learn the basics of HTML, spreadsheets,etc. For the final project in the course thestudents have the option of a more exten-sive project in any of the above, or learn-ing a new software package. One of lastyear’s students briefly put her project onthe web. She and a friend bought the webaddress IllicitBundleOfJoy.com whereone was able to fill out a form specifyingthe sex, skin color, eye color, price one iswilling to pay, etc. However, all you got foryour pains is a notice that it is a joke. Ifshe didn’t mind getting the FBI after her,it would have been a better joke if she’dasked for MasterCard or Visa.

About 15 years ago or so, during the earlyspring, the tabloids had a field day whenit was discovered that several Brown stu-

dents (female) were involved in a prosti-tution ring. “Ivy League Madam!” or somesuch. Brown immediately went into full-court damage control, with multiple pressconferences all saying that what studentsdid on their own time was beyond our con-trol, that this was not exactly what wethought the life of the mind was all about,and that we did prefer students withoutcriminal records.

At the time I thought that Brown wasoverreacting, but now I am not so sure. Akey fact in the above is that it occurred inearly spring—the time when students(and their parents) are deciding where togo to college. I have reconsidered becausethis last spring I was one such parent. Inparticular, one of the schools my son wasconsidering was Hamilton, a small, selec-tive, liberal arts college in upstate NewYork. In early spring the New York Timespublished an article about cloning hu-mans prominent in which was one Profes-sor Boisselier, identified as a professor inthe chemistry department at Hamilton.Worse, the organization whose cloninglaboratory she runs, the Raelians, isheaded by a fellow named Rael, who wasabducted by aliens and seduced by femalealien robots, and who believes on the ba-sis of this that the destiny of the humanrace is to make itself perfect and immor-tal through the use of science.

After some investigation it turned out shewas a visiting faculty member, she re-signed from her position, my son decidedto go to Colby, and he thought the wholething was a hoot. But I mentioned this toa neighbor who is the spokesperson forthe Rhode Island School of Design. Sheresponded that a few years ago, again inthe spring, several RISD students werearrested by the Boston police after nearlyfinishing painting an entrance to anMBTA station pink. A picture appearedon the front page of the Boston Globe withthe headline ‘Caught Pink-Handed’.When the students were asked what theywere doing they responded that this wastheir ‘site-specific’ art work for one oftheir RISD courses. Naturally, this fell inmy neighbor’s lap. A reporter with whomshe had dealt before called to ask forRISD’s position. She responded that RISDdid not have a position, but would he takea comment off the record? Off the record,RISD’s position was ‘What shade of pink?’

No, Suzi Howe did not dress to match thetablecloths on IPP Symposium day, buteveryone who stepped off the elevator

thought she had!

Department of Computer ScienceBrown University

Box 1910, Providence, RI 02912, USA

conduit!

conduit! 32

NON-PROFIT

U.S. Postage

PAID

Providence, RI

Permit #202

Printed on recyled paper Address changes welcomed

conduit!A publication of

The Computer Science DepartmentBrown University

❦Inquiries to: conduit!

Department of Computer ScienceBox 1910, Brown University

Providence, RI 02912FAX: 401-863-7657

PHONE: 401-863-7610EMAIL: sjh@cs.brown.edu

WWW: http://www.cs.brown.edu/publications/conduit/

Jeff CoadyTechnical Support

Katrina AveryEditor

Suzi HoweEditor-in-Chief

Of the more than 5,000 feared dead as a result of the terror-ist attacks on September 11, at least six were Brownalumni.

As far as we know, none of them had connections with thisdepartment. Members of the Brown University communitypaused during three events in the October 12-14 celebrationof President Ruth Simmons’ inauguration to remember andhonor these alumni.

For details and a list of those lost, see:

http://www.brown.edu/Administration/News_Bureau/2001-02/01-041.html

Department of Computer ScienceBrown University

Box 1910, Providence, RI 02912, USA

conduit!

conduit! 33

NON-PROFIT

U.S. Postage

PAID

Providence, RI

Permit #202

Printed on recyled paper Address changes welcomed

conduit!A publication of

The Computer Science DepartmentBrown University

❦Inquiries to: conduit!

Department of Computer ScienceBox 1910, Brown University

Providence, RI 02912FAX: 401-863-7657

PHONE: 401-863-7610EMAIL: sjh@cs.brown.edu

WWW: http://www.cs.brown.edu/publications/conduit/

Jeff CoadyTechnical Support

Katrina AveryEditor

Suzi HoweEditor-in-Chief