A Brief History ofHuman Computer Interaction TechnologyBrad A. MyersCarnegie Mellon University School of Computer Science Technical Report CMU-CS-96-163anduman Computer !nteraction !nstitute Technical Report CMU-C!!-96-1"3 #ecem$er% 1996Please cite this work as:Brad A. Myers. A Brief History of Human Computer Interaction Technology.ACM interactions. !ol. "# no. $# March# %&&'. pp. (()"(. uman Computer !nteraction !nstitute School of Computer ScienceCarnegie Mellon University&itts$urgh% &' 1()13-3*91$am+a,gp,cs,cmu,eduA*stract This article summari-es the historical development of ma.or advances in human-computer interaction technology% emphasi-ing the pivotal role of university research in the advancement ofthe field, Copyright /c0 1996 -- Carnegie Mellon University' short e1cerpt from this article appeared as part of 2Strategic #irections in uman Computer !nteraction%2 edited $y 3rad Myers% 4im ollan% !sa$el Cru-% ACM Computing Surveys% $'/50% #ecem$er 1996 This research 6as partially sponsored $y 7CC8SC under Contract 7o, 766""1-95-C-6"39% 'rpa 8rder 7o, 33)6 and partially $y 7S: under grant num$er !R!-9319969, The vie6s and conclusions contained in this document are those of the authors and should not $e interpreted as representing the official policies% either e1pressed or implied% of 7CC8SC or the U,S, ;overnment, +eywords: uman Computer !nteraction% istory% User !nterfaces% !nteraction Techniits andinterface $uilders% concepts 6hich6eredeveloped first at universities, Aven the spectacular gro6th of the =orld-=ide =e$ is a directresult of C! researchB applying hyperte1t technology to $ro6sers allo6s one to traverse a lin>across the 6orld 6ith a clic> of the mouse, !nterface improvements more than anything else hastriggered this e1plosive gro6th, :urthermore% the research that 6ill lead to the user interfaces forthe computers of tomorro6 is happening at universities and a fe6 corporate research la$s, This paper tries to $riefly summari-e many of the important research developments in uman-Computer !nteraction /C!0 technology, 3y 2research%2 ! mean e1ploratory 6or> at universitiesand government and corporate research la$s /such as ?ero1 &'RC0 that is not directly related toproducts, 3y 2C! technology%2 ! am referring to the computer side of C!, ' companion articleon the history of the 2human side%2 discussing the contri$utions from psychology% design% humanfactors and ergonomics 6ould also $e appropriate, ' motivation for this article is to overcome the mista>en impression that much of the important6or> in uman-Computer !nteraction occurred in industry% and if university research in uman-Computer !nteraction is not supported% then industry 6ill .ust carry on any6ay, This is simply nottrue, This paper tries tosho6that manyof themost famous C! successes developed$ycompanies are deeply rooted in university research, !n fact% virtually all of todayCs ma.or interfacestylesandapplicationshavehadsignificant influencefromresearchat universitiesandla$s%often 6ith government funding, To illustrate this% this paper lists the funding sources of some ofthema.oradvances, =ithout thisresearch% manyoftheadvancesinthefieldofC!6ouldpro$a$ly not have ta>en place% and as a conseing these technologies ready for 6idespread use, The same 6ill $e true for theC! technologies that 6ill provide the interfaces of tomorro6, !t is clearly impossi$le to list every system and source in a paper of this scope% $ut ! have tried torepresent the earliest and most influential systems, 'lthough there are a num$er of other surveysof C! topics /see% for e1ample E1F E1"F E33F E3*F0% none cover as many aspects as this one% or tryto $e as comprehensive in finding the original influences, 'nother useful resource is the video2'll The =idgets%2 6hich sho6s the historical progression of a num$er of user interface ideasE)(F, The technologies covered in this paper include fundamental interaction styles li>e directmanipulation% the mouse pointing device% and 6indo6sG several important >inds of applicationareas% such as dra6ing% te1t editing and spreadsheetsG the technologies that 6ill li>ely have the$iggest impact on interfaces of the future% such as gesture recognition% multimedia% and 3#G andthe technologies used to createinterfaces using the other technologies% such as user interfacemanagement systems% tool>its% and interface $uilders, ,igure %B 'ppro1imate time lines sho6ing 6here 6or> 6as performed on some ma.or technologies discussed in this article, $. Basic Interactions -irect Manipulationof graphical o*.ectsB Theno6u$ietchpad E55F% 6hich 6as his 1963M!T&h#thesis, S>etch&adsupportedthemanipulationof o$.ectsusingalight-pen%including gra$$ing o$.ects% moving them% changing si-e% and using constraints, !tcontained the seeds of myriad important interface ideas, The system 6as $uilt at @incoln@a$s6ithsupportfrom the 'ir :orce and7S:, =illiam 7e6manCsReaction andlerE3"F% createdat !mperial College% @ondon/1966-690 provideddirect manipulationofgraphics% and introduced 2@ight andles%2 a form of graphical potentiometer% that 6aspro$a$ly the first 26idget,2 'nother early system 6as 'M3!TH; /implemented at M!TCs@incoln@a$s% 196*% 'R&'funded0, !t employed% amongother interfacetechni3audelaireCs #ra66hichaddedhandlingof lines andcurves E1"% p, 3)6F, Thefirstcomputer painting program 6as pro$a$ly #ic> ShoupCs 2Superpaint2 at &'RC /1995-9(0, Te1t 2ditingB !n196)at theStanfordResearch@a$% Angel$art proposed% andlaterimplemented% a6ordprocessor 6ithautomatic6ord6rap% searchandreplace% user-defina$lemacros% scrollingte1t% andcommandstomove% copy%anddeletecharacters%6ords% or $loc>s of te1t, StanfordCs TLAdit /196(0 6as one of the first CRT-$ased displayeditorsthat 6as 6idelyusedE5*F, Theyperte1t AditingSystemE("% p, 1"*F from3ro6nUniversityhadscreeneditingandformattingof ar$itrary-si-edstrings6ithalightpen in 1969 /funding from !3M0, 7@S demonstrated mouse-$ased editing in 196*,TAC8 from M!T 6as an early screen-editor /19690 and AM'CS E53F developed from itin1995, ?ero1&'RCCs3ravoE1"% p, )*5F6asthefirst=IS!=I;editor-formatter/19950, !t 6as designed$y3utler @ampsonandCharles Simonyi 6hohadstarted6or>ing on these concepts around 199" 6hile at 3er>eley, The first commercial=IS!=I; editors 6ere the Star% @isa=rite and then Mac=rite, :or a survey of te1teditors% see E))F E("% p, 1"*F, 3preadsheetsB The initial spreadsheet 6as LisiCalc 6hich 6as developed $y :ran>stonand 3ric>lin /1999-*0 for the 'pple !! 6hile they 6ere students at M!T and the arvard3usiness School, The solver 6as $ased on a dependency-directed $ac>trac>ing algorithm$y Sussman and Stallman at the M!T '! @a$, HyperTe1tB The idea for hyperte1t /6here documents are lin>ed to related documents0 iscredited to Lannevar 3ushCs famous MAMA? idea from 195( E5F, Ted 7elson coined theterm2hyperte1t2in196(E)9F, Angel$artCs7@SsystemE*F at theStanfordResearch@a$oratories in 196( made e1tensive use of lin>ing /funding from 'R&'% 7'S'% andRome '#C0, The 27@S 4ournal2 E1"% p, )1)F 6as one of the first on-line .ournals% and itincluded full lin>ing of articles /199"0, The yperte1t Aditing System% .ointly designed$y 'ndyvan#am%Ted7elson% andt6ostudentsat3ro6nUniversity/fundingfrom!3M0 6as distri$uted e1tensively E59F, The University of LermontCs &R8M!S /19960 6asthe first yperte1t system released to the user community, !t 6as used to lin> patient andpatient care information at the University of LermontCs medical center, The M8; pro.ect/19990fromCMU6asanotherearlyhyperte1tsystem% and6asfunded$y87Rand#'R&' E36F, 3en ShneidermanCs yperties 6as the first system 6here highlighted itemsinthete1t could$eclic>edontogotootherpages/19*3% Univ,ofMaryland0E19F,yperCard from 'pple /19**0 significantly helped to $ring the idea to a 6ide audience,There have $een many other hyperte1t systems through the years, Tim 3erners-@ee usedthehyperte1t ideatocreatethe=orld=ide=e$in199"at thegovernment-fundedAuropean &article &hysics @a$oratory /CAR70, Mosaic% the first popular hyperte1t$ro6ser for the =orld-=ide =e$ 6as developed at the Univ, of !llinoisC 7ational Centerfor Supercomputer 'pplications /7CS'0, :or a more complete history of yperTe1t% seeE31F, Computer Aided -esign 4CA-5B The same 1963 !:!&S conference at 6hich S>etchpad6as presented also contained a num$er of C'#systems% including #oug RossCsComputer-'ided #esign &ro.ect at M!T in the Alectronic Systems @a$ E39F and CoonsC6or> at M!T 6ith S>etch&ad E9F, Timothy 4ohnsonCs pioneering 6or> on the interactive3#C'#systemS>etchpad3E13F6ashis1963M!TMSthesis/funded$ythe 'ir:orce0, The first C'#HC'M system in industry 6as pro$a$ly ;eneral MotorCs #'C-1/a$out 19630, !ideo 6amesB The first graphical video game 6as pro$a$ly Space=ar $y Slug Russel ofM!T in 196) for the &-1 E19% p, 59F including the first computer .oystic>s, The earlycomputer 'dventuregame6as created$y=ill Cro6ther at 337% and#on=oodsdevelopedthisintoamoresophisticated 'dventuregameat Stanfordin1966E19% p,13)F, Con6ayCs game of @!:A 6as implemented on computers at M!T and Stanford in199", The first popular commercial game 6as &ong /a$out 19960, (. 7p)and)Coming Areas 6esture 8ecognitionB The first pen-$ased input device% the R'7# ta$let% 6as funded $y'R&', S>etchpad used light-pen gestures /19630, Teitelman in 1965 developed the firsttraina$le gesture recogni-er, ' very early demonstration of gesture recognition 6as TomAllisC ;R'!@ system on the R'7# ta$let /1965% 'R&' funded0, !t 6as of the 6"Cs - 9"Cs led the 6ay to ma>ing 3-# real-time6ith commercial systems from ;A% AvansDSutherland% SingerH@in> /funded $y 7'S'%7avy% etc,0, 'nother important center of current research in 3-# is :red 3roo>sCla$ atU7C /e,g, E)F0, !irtual 8eality and Augmented 8eality: The original 6or> on LR 6as performed $y!van Sutherland 6hen he 6as at arvard /196(-196*% funding $y 'ir :orce% C!'% and3ell @a$s0, Leryimportant early6or>6as$y Tom:urness6henhe6asat =right-&atterson':3, MyronJruegerCs early6or>at the Universityof Connecticut 6asinfluential, :red 3roo>sCand enry :uchCs groups at U7C did a lot of early research%including the study of force feed$ac> /1991% funding fromUS 'tomic AnergyCommission and 7S:0, Much of the early research on head-mounted displays and on the#ata;love 6as supported $y 7'S', Computer3upportedCooperati9e/ork, #ougAngel$artCs 196*demonstrationof7@S E*F included the remote participation of multiple people at various sites /fundingfrom 'R&'% 7'S'% and Rome '#C0, @ic>lider and Taylor predicted on-line interactivecommunities in an 196* article E)"F and speculated a$out the pro$lem of access $einglimited to the privileged, Alectronic mail% still the most 6idespread multi-user soft6are%6as ena$led $y the 'R&'net% 6hich $ecame operational in 1969% and $y the Athernetfrom ?ero1 &'RC in 1993, 'n early computer conferencing system 6as TuroffCs A!ASsystem at the 7e6 4ersey !nstitute of Technology /199(0, :atural language and speech:The fundamental research for speech and naturallanguage understanding and generation has $een performed at CMU% M!T% SR!% 337%!3M% 'TDT 3ell @a$s and 3ellCore% much of it government funded, See% for e1ample%E35F for a survey of the early 6or>, ". 3oftware Tools and Architectures The area of user interface soft6are tools is er% et al.% Aditors, 199(% Morgan Jaufmann &u$lishers% !nc,B San :rancisco, pp, 3(-59, ), 3roo>s% :, 2The Computer 2Scientist2 as Toolsmith--Studies in !nteractive Computer ;raphics%2 in I%I& Con'erence &roceedings. 1999, pp, 6)(-635, 3, 3urtny>% 7, and =ein% M,% 2Computer ;enerated Jey :rame 'nimation.2 (ournal )' the Society o' Motion &icture and Television $ngineers% 1991, '/30B pp, 159-1(3, 5, 3ush% L,% 2's =e May Thin>.2 The Atlantic Monthly% 195(, %>;/4uly0B pp, 1"1-1"*, Reprinted and discussed in interactions% 3/)0% Mar 1996% pp, 3(-69, (, 3u1ton% =,% et al. 2To6ards a Comprehensive User !nterface Management System%2 in &roceedings SI**RA&H+,-: Computer *raphics. 19*3, #etroit% Mich, %>, pp, 3(-5), 6, Card% S,J,% 2&ioneers and SettlersB Methods Used in Successful User !nterface #esign%2 in Human-Computer Inter'ace .esign: Success Stories# $merging Methods# and Real-/orld Conte0t% M, Rudisill% et al.% Aditors, 1996% Morgan Jaufmann &u$lishersB San :rancisco, pp, 1))-169, 9, Coons% S, 2'n 8utline of the Re% #,4, 2' User !nterface Management System%2 in &roceedings SI**RA&H+,!: Computer *raphics. 19*), 3oston% M', %;, pp, 99-1"6, 1(, Jay% ',% The Reactive $ngine. &h# Thesis% Alectrical Angineering and Computer Science University of Utah% 1969% 16, Jay% ',% 2&ersonal #ynamic Media.2 I$$$ Computer% 1999, %?/30B pp, 31-5), 19, Joved% @, and Shneiderman% 3,% 2Am$edded menusB Selecting items in conte1t.2 Communications o' the ACM% 19*6, (/)90B pp, 31)-31*, 1*, @evinthal% C,% 2Molecular Model-3uilding $y Computer.2 Scienti'ic American% 1966, $%(/60B pp, 5)-(), 19, @evy% S,% Hac5ers: Heroes o' the Computer Revolution. 19*5% ;arden City% 7IB 'nchor &ressH#ou$leday, )", @ic>lider% 4,C,R, and Taylor% R,=,% 2The computer as Communication #evice.2 Sci. Tech.% 196*, AprilB pp, )1-31, )1, @inton% M,',% Llissides% 4,M,% and Calder% &,R,% 2Composing user interfaces 6ith !nterLie6s.2 I$$$ Computer% 19*9, $$/)0B pp, *-)), )), Meyro6it-% 7, and Lan #am% ',% 2!nteractive Aditing SystemsB &art 1 and ).2 ACM Computing Surveys% 19*), %(/30B pp, 3)1-3(), )3, Myers% 3,',% 2The User !nterface for Sapphire.2 I$$$ Computer *raphics and Applications% 19*5, (/1)0B pp, 13-)3, )5, Myers% 3,',% 2' Ta1onomy of User !nterfaces for =indo6 Managers.2 I$$$ Computer *raphics and Applications% 19**, '/(0B pp, 6(-*5, )(, Myers% 3,',% 2'll the =idgets.2 SI**RA&H 1ideo Review% 199", "> )6, Myers% 3,',% 2User !nterface Soft6are Tools.2 ACM Transactions on Computer Human Interaction% 199(, $/10B pp, 65-1"3, )9, Myers% 3,',% et al.% The Amulet 1!." Re'erence Manual , Carnegie Mellon University Computer Science #epartment Report% 7um$er% :e$% 1996, System availa$le from httpBHH666,cs,cmu,eduHOamulet, )*, Myers% 3,',% et al.% 2;arnetB Comprehensive Support for ;raphical% ighly-!nteractive User !nterfaces.2 I$$$ Computer% 199", $0/110B pp, 91-*(, )9, 7elson% T, 2' :ile Structure for the Comple1% the Changing% and the !ndeterminate%2 in &roceedings ACM 7ational Con'erence. 196(, pp, *5-1"", 3", 7e6man% =,M, 2' System for !nteractive ;raphical &rogramming%2 in A%I&S Spring (oint Computer Con'erence. 196*, $', pp, 59-(5, 31, 7ielsen% 4,% Multimedia and Hyperte0t: the Internet and 8eyond. 199(% 3ostonB 'cademic &ress &rofessional, 3), &alay% ',4,% et al. 2The 'ndre6 Tool>it - 'n 8vervie6%2 in &roceedings /inter 9seni0 Technical Con'erence. 19**, #allas% Te1, pp, 9-)1, 33, &ress% @,% 23efore the 'ltairB The istory of &ersonal Computing.2 Communications o' the ACM% 1993, 0;/90B pp, )9-33, 35, Reddy% #,R,% 2Speech Recognition $y MachineB ' Revie6%2 in Readings in Speech Recognition% ', =ai$el and J,-:, @ee% Aditors, 199"% Morgan JaufmannB San Mateo% C', pp, *-3*, 3(, Reddy% R,% 2To #ream the &ossi$le #ream /Turing '6ard @ecture0.2 Communications o' the ACM% 1996, 0&/(0B pp, 1"(-11), 36, Ro$ertson% ;,% 7e6ell% ',% and Rama>rishna% J,% :)*: A Man-Machine Communication &hilosophy , Carnegie Mellon University Technical Report Report% 7um$er% 'ugust% 1999, 39, Ross% #, and Rodrigue-% 4, 2Theoretical :oundations for the Computer-'ided #esign System%2 in A%I&S Spring (oint Computer Con'erence. 1963, $0, pp, 3"(-3)), 3*, Rudisill% M,% et al.% Human-Computer Inter'ace .esign: Success Stories# $merging Methods# and Real-/orld Conte0t. 1996% San :ranciscoB Morgan Jaufmann &u$lishers, 39, Scheifler% R,=, and ;ettys% 4,% 2The ? =indo6 System.2 ACM Transactions on *raphics% 19*6, "/)0B pp, 99-1"9, 5", Shneiderman% 3,% 2#irect ManipulationB ' Step 3eyond &rogramming @anguages.2 I$$$ Computer% 19*3, %;/*0B pp, (9-69, 51, Smith% #,C,% &ygmalion: A Computer &rogram to Model and Stimulate Creative Thought. 1999% 3asel% StuttgartB 3ir>hauser Lerlag, &h# Thesis% Stanford University Computer Science #epartment% 199(, 5), Smith% #,C,% et al. 2The Star User !nterfaceB an 8vervie6%2 in &roceedings o' the 23,! 7ational Computer Con'erence. 19*), ':!&S, pp, (1(-()*, 53, Stallman% R,M,% $macs: The $0tensi;le# Customi