Department of Computer Science, Clemson University
Eye Tracking Research
Andrew T. Duchowski
Past Projects, Current Work, & Future Potential
2
Department of Computer Science, Clemson University
What is Eye Tracking?
• Method for calculating what user is looking at
• Accomplished through computer vision techniques imaging pupil and corneal reflection of infra-red light
• Can be done in real-time, hence eyes can be used to point to windows, icons, menus, in place of hand-controlled mouse
• Or…eye movements can be recorded for offline analysis
• Recent improvement in technology (cameras) have revitalized the field
3
Department of Computer Science, Clemson University
Old(er) Technology
• Windows 95 PC
• Requires chinrest
• Cumbersome and difficult to operate
Fig.1: VR and eye-tracking equipment
4
Department of Computer Science, Clemson University
State-of-the-Art Next Generation Technology
• Tobii ET-1750 eye tracker
• Apple Mac G5 (ET client)
• Dell PC (ET server)
• 3-headed display
Fig.3: Tobii setupFig.2: Tobii user
5
Department of Computer Science, Clemson University
Selected Past/Current Projects (~$1.3M)
• NASA ($265,742, with IE): Visual inspection training
– diagnostic eye tracking for aircraft inspection
• NSF CAREER ($212,022): Computer graphics, education, service
– real-time Level Of Detail, course development
• NSF ITR ($313,361, with IE): Eye tracked avatars in Virtual Reality
– eye tracked real-time “look at this” communication
• NSF ATE ($389,655, with IE): Tech. transfer to Greenville Tech
– aircraft inspection training and education
• DoD/Navy/SPAWAR Charleston ($115,528): Usability engineering
– interface design through eye movement analysis
6
Department of Computer Science, Clemson University
NASA: Visual Inspection
Fig.4: Real and virtual aircraft cargo bay (2 1/2 D and 3D models) and eye movements
• Visual inspection in VR cargo bay (VRST 2001)
• 21/2 D scanpaths (EuroGraphics 2002)
• Eye movement analysis in 3D (ETRA 2002)
• 3D scanpath comparison (VSS 2003)
7
Department of Computer Science, Clemson University
NASA: Visual Inspection
• Our most recent success: our first publication at SIGCHI (nominated for best paper, top 5% of 371 paper submissions)
QuickTime™ and aVideo decompressor
are needed to see this picture.
8
Department of Computer Science, Clemson University
NSF CAREER: Real-Time Graphics and Video
Fig.5: Gaze-contingent terrain, model, video
• Gaze-contingent terrain generation showed feasibility of real-time graphics changing with gaze direction (Smart Graphics 2000)
• Models gave 10-fold speedup in frame rate (EuroGraphics 2001)
• Video demonstrates in-hardware image processing on GPU-programmable graphics cards (ADC 04 invited talk)
9
Department of Computer Science, Clemson University
NSF CAREER: Course Development
0
100
200
300
400
500
600
No Cue Dot Arrow
Mean time
Both Male Female
• Quality of coursework is steadily improving
• Scanpaths showed user attention to exocentric map during navigation of a 3D maze (EGVE 04)
Fig.6: Example of student work: VR maze navigation experiment
10
Department of Computer Science, Clemson University
NSF CAREER: Course Development
• Our most recent success: undergraduate paper at Graphics Interface 2005 (DuPont award)
QuickTime™ and aVideo decompressor
are needed to see this picture.
11
Department of Computer Science, Clemson University
NSF CAREER: Service & Scholarship
• Eye Tracking Research & Applications conference brings together researchers from around the world, now in its 4th year
Fig.7: ETRA conference proceedings and text
12
Department of Computer Science, Clemson University
NSF ITR: Eye Tracked Avatars
• Research showed advantage of visual deictic reference in VR for training purposes (ETRA 04)
Fig.8: Avatars in Collaborative Virtual Environment (CVE)
13
Department of Computer Science, Clemson University
NSF ATE: Technology Transfer
• Currently in the process of transferring VR technology to Greenville Tech for aircraft inspection training
• Expanding training simulators to borescope inspection
Fig.9: Physical and virtual engine components for inspection
Fig.10: User working with WindowVR
14
Department of Computer Science, Clemson University
DoD/Navy/SPAWAR Charleston: Usability Engineering
• Collaboration between the military (Navy), industry (EMA), and academia (Clemson University)
• Using eye movements to design user interfaces (project just started)
• Navy expects to apply to design of Command & Control applications
Fig.11: Collecting scanpaths and “hot spots”
15
Department of Computer Science, Clemson University
Future Projects
• NSF / DoD: seeking funding for next generation eye tracking equipment
– currently only 1 tracker for entire class (and me!)
• Future projects:
– NSF?
– DoD?
– BMW / ICAR?
Fig.12: UAV navigation training
Fig.13: HUD design/analysis
16
Department of Computer Science, Clemson University
References
Danforth, R., Duchowski, A., Geist, R., McAliley, E. (2000). A Platform for Gaze-Contingent Virtual Environments. In Smart Graphics (Papers from the 2001 AAAI Spring Symposium, Technical Report SS-00-04), Menlo Park, CA, AAAI, pp. 66-70.
Duchowski, A., Cournia, N., and Murphy, H. (2004). Gaze-Contingent Displays: Review and Current Trends. In Adaptive Displays Conference, 7 August, Los Angeles, CA.
Duchowski, A., Medlin, E., Cournia, N., Gramopadhye, A., Melloy, B., and Nair, S. (2002). 3D Eye Movement Analysis for VR Visual Inspection Training. In Eye Tracking Research & Applications (ETRA), 25-27 March, New Orleans, LA, ACM, pp. 103-110, 155.
Duchowski, A., Medlin, E., Gramopadhye, A., Melloy, B., and Nair, S. (2001). Binocular Eye Tracking in VR for Visual Inspection Training. In VRST Proceedings, ACM, pp. 1-8.
17
Department of Computer Science, Clemson University
References
Duchowski, A., Marmitt, G., Desai, R., Gramopadhye, A., and Greenstein, J. (2003). Algorithm for Comparison of 3D Scanpaths in Virtual Reality. In VSS 03 Proceedings, Vision Sciences Society (posters).
Marmitt, G., Duchowski, A. (2002). Modeling Visual Attention in VR: Measuring the Accuracy of Predicted Scanpaths. In EuroGraphics 2002 (Short Presentations), 2-6 September, Saarbrucken, Germany, EuroGraphics.
Murphy, H. and Duchowski, A. (2002). Perceptual Gaze Extent & Level Of Detail in VR: Looking Outside the Box. In SIGGRAPH 02 Conference Abstracts & Applications, San Antonio, TX, ACM.
Murphy, H. and Duchowski, A. (2001). Gaze-Contingent Level Of Detail. In EuroGraphics (Short Presentations), Manchester, UK, EuroGraphics.
Vembar, D., Iyengar, N., Duchowski, A., Clark, K., Hewitt, J., Pauls, K. (2004). Effect of visual cues on human performance in navigating through a virtual maze. In EuroGraphics Symposium on Virtual Environments, 8-9 June, Grenoble, France, EuroGraphics.