Neural Eye Tracking for Human Computer Interaction

Presented By:Sibarama Panigrahi(11040056)VSSUT,Burla

2

Contents

3

4

Frame SearchRegion Pre-Processing Clustering

EyesLocated

Successfully?Update Means

And Covariance.Update frameSearch Region

Yes

Yes

Input Frame BayesianClassifier Post-Processing

5

Facts About Human Gaze

• One’s eye is rarely stationary

• Eye movement reflects a viewer’s visual information process

• The eye can move faster than the hand

• Eye movement consists of:• Saccades• Fixations

6

Why Use Eye Tracking?

•Gives an accurate measure of where one’s visual attention is directed

•Enhances or backs-up observations

•May reveal patterns which may become proof of effective or ineffective usability

•Can lead to many potentially useful applications

7

Types of Eye Tracking

•Skin Electrodes•Contact Lens•Head Mounted•Remote Systems

(Desk or Computer-based)

8

Types of Eye Tracking:Skin Electrodes

•Electrodes placed on the skin around the eye socket

•Measures electrical differences the between the retina and cornea

GOOD POINTS:•Both eyes can be recorded together •Least expensive•Simple to use

DOWNFALLS:•It is limited to horizontal and vertical movements•Poor accuracy for absolute positioning

9

Types of Eye Tracking:Contact Lens

•A non-slipping contact lens fits over corneal bulge.

•Tracking is recorded by affixing a magnetic coil or mirror to the lens.

•GOOD POINTS:•Provides accurate data about the nature of human eye movements.

•DOWNFALLS:•Extremely awkward, uncomfortable for the user•Interferes with blinking•Covers only a limited range of eye movements

10

Types of Eye Tracking: Head Mounted

• Small camera and light source mounted to users head via a headband or helmet

• Reports the angle of the user’s eye with respect to his or her head.

• Two data sources can determine the line of gaze in physical space.

GOOD POINTS:• Doesn’t restrict the user’s head movements

(including locations away from the computer screen)

DOWNFALLS:• More awkward to use than the desk-based

system as the user has to have instrument mounted to head.

11

Types of Eye Tracking:Remote Systems

• Most practical method of eye tracking

• Uses Illuminator/eye camera

• Tracking visible features of the eye

• Head movements can be distinguished from eye movements by tracking 2 points

GOOD POINTS:•Allows for a fair range of head movements•Accurate, fast and affordable•Not intrusive

DOWNFALLS:•Head still needs to stay within camera range.•Delicate to calibrate and operate

12

Processing Raw Data

•Data from eye tracking is too low level and voluminous for immediate interpretation.

•Software packages can combine data from eye tracking and interaction logs to produce a page-by-page reports.

13

Pitfall of Using Eye Tracking Technologies

•Equipment is still quite intrusive• Noisy• Restricted head movements• Requires more patience by user

•Fairly large pool of test subjects is needed• User vision must be considered

•How can we tell if and what info. is really being processed?

• eye fixations provide measurement of visual interest but do not provide a measure of cognitive interest.

•Often viewed as a time-consuming methodology returning questionable benefits.

14

Previous Research/Applications

2 Main Categories

•Interface Usability Studies

•Interactive Applications

15

Interface Usability StudiesOverview

•Challenging of the controversial Scanpath Theory(Noton and Stark, 1971)

The Scanpath Theory…• Argues that its improbable that the eye is controlled by specific

features of a visual display• Rejects the explanation that subjects are driven by habits

•Since Internet users are repeatedly exposed to certain visual displays of information, the Web is an ideal stimulus to test this theory.

•Many studies dispute the Scanpath Theory

16

Interface Usability Studies

Eye Tracking CAN…• Tell whether users are looking at the screen

• Tell whether users are reading or scanning.

• Learn the relative intensity of a user’s attention to various

parts of a web page

• Determine whether a user is searching for a specific item

• Compare user scan patterns

17

Interface Usability Studies

Eye Tracking CAN’T…• Let you know whether users actually “see” something

• Prove that users didn’t see something

• Determine why users are looking at something

• Test everybody

18

Interface Usability Studies

• Online News portal study done by Stanford University and The Poynter Institute in 2000

19

Research Applications/Studies: Online News Portals

• by Stanford University and The Poynter Institute (2000)• the first eyetracking study documenting the actual second-by-second

behavior of users of online news sites

• Tracked what subjects read and their movementsfrom page to page.

• Used 67 subjects in two cities• Subjects were readers of online news - people who visited news-

related sites at least three times a week.

• Each screen was analyzed by:• Format (what layout elements attracted users)• Topics• Actions• Time spent on page

20

Research Applications/Studies: Online News Portals

The numbers superimposed over the pink blocks show the order in which fixation clusters.

Follow the numbers and note the roaming pattern.

The dark "x" to the right of photo that indicates a mouse click for calling up a hyperlinked story.

21

Research Applications/Studies: Online News Portals

Return to main page after link click

22

Research Applications/Studies: Online News Portals

OVERALL CONCLUSIONS:

• A strong preference for text over graphics (briefs or captions)• The eyes of online news readers then come back to the photos and graphics

• Banner ads do catch online readers' attention• for an average of one second.• Long enough to perceive the ad (but what about animated ads?)

• Statistics of how different users (by age, sex) navigate through online news

23

Interactive Applications

•Accessibility

•System Enhancement

•Non-Command Based Systems

•Virtual Displays

24

Interactive Applications:Accessibility

• Eye tracking can allow people to use their eyes to communicate

• Its users include people with brain injuries, cerebral palsy, multiple sclerosis, muscular dystrophy, spinal cord injuries, spinal muscular atrophy, strokes, etc.

25

Interactive Applications:System Enhancement

•Aimed toward users whose hands are occupied

• Potential Problems:• Some will find the eye movement based interface better (faster, more

convenient and more natural) while others may feel uncomfortable.

• Too unnatural to use in critical situations?

•Eye tracking may be best used to act as a supplemental input or display method.

26

Interactive Applications:Non-Command Based Systems

•The computer passively monitors the user and responds as appropriate, rather than waiting for the user to issue specific commands.

•Eye-controlled paddleball game

•The Little Prince (interactive children’s story) The user controls the flow of the narration by where he or she looks without explicit commands (interactive fiction)

•Problem: How can we issue a command clearly?•Gaze duration and blnking can be ineffective or faulty

27

Interactive Applications:Virtual Displays

Making VR more realistic

• If accurate, the user cannot detect the difference between this arrangement and the large, high-resolution display it emulates.

28

Advantages of Using Eye Trackingas an Input Device

• Eye movement is faster than other current input media

• No training or particular coordination is required of normal users

• Can determine where the user’s interest is focused automatically

29

Disadvantages of Using Eye Trackingas an Input Device

• People are not accustomed to operating devices by simply moving their eyes.

• In interactive applications, using the eye as an input device, when does the system decide that a user is interested in a visual object?

30

Future of Eye tracking?

•Helpful for usability studies to understand users interact with their environments.

•Potentially could provide new and more effective methods of computer-human interaction.

Eye tracking equipment is still improving, and is “not quite there” yet – but has an exciting future!

31

VIDEOS

32

VIDEOS

33

VIDEOS

34

Links:RESEARCH LINKS

Gaze-Based Interaction Grouphttp://www.cs.uta.fi/research/hci/gaze/links.phpA page of links about various eye tracking projects and research studies

User Interface Engineeringhttp://world.std.com/%7Euieweb/index.html

Stanford Poynter Projecthttp://www.poynter.org/eyetrack2000/index.htm

Critique on Study: http://www.poynter.org/centerpiece/071200alan.htm

Technology News: New Software Replaces Computer Keyboards with Eyes: August 23, 2002http://www3.cosmiverse.com/news/tech/0802/tech08230201.htmlAn interesting article on a software has been named "Dasher", that claims to "exploit our eyes' natural ability to navigate and identify familiar patterns. It is the result of work done by a team at Cavendish Laboratory in Cambridge, UK. An eye-tracking device allows users to select letters from the screen.

The Little Prince (interactive children’s story experiment)I. Starker and R.A. Bolt, “A Gaze-Responsive Self-Disclosing Display,” Proc. ACM CHI ’90 Human Factors in Computing Systems Conference pp. 3-9, Addison-Wesley/ACM Press (1990).

35

Links :ANALYSIS SOFTWARE

SensoMotoric Instruments http://www.smi.de/specializes in the development and integration of eye movement measuring and evaluation systems

Seeing Machineshttp://www.seeingmachines.com/products.htmCreators of faceLAB™ -a head, face and gaze tracking technology that measures the position of a human head, gaze direction and blink events.

GazeTrackerhttp://www.a-s-l.com/gazetracker.htmSoftware that consolidates the information that researchers need to conduct their web design and software usability studies into one piece of software.

EYETECH DIGITAL SYSTEMS, INC.http://www.eyetechds.com/Software to operate your PC hands free

ARRINGTON RESEARCH, INC.http://www.arringtonresearch.com/-creators of eye tracking hardware and software

36

Links :EYE TRACKING EQUIPMENT

Eye Movement Equipment Database (EMED) http://ibs.derby.ac.uk/emed/Eye Movement Equipment Database, based in the Institute of Behavioural Sciences, University of Derby.

LC Technologieshttp://www.eyegaze.com/LC Technologies designs, manufactures, and distributes unobtrusive video eyetracking systems.

Metrovisionhttp://www.metrovision.fr/A company located in France that sells instruments for ophthalmologists including visual field, visual electrophysiology, contrast sensitivity, and eye movements.

Applied Science Laboratorieshttp://www.a-s-l.com/index.htmdevelops eye tracking technology and systems

SR Researchhttp://www.eyelinkinfo.com/Creators of EyeLink II –an eye tracking system that consists of three miniature cameras mounted on a comfortable leather-padded.

37

BibliographyCrowe, Eric C. and Narayanan, Hari. “Comparing interfaces based on what users watch and do”. In Proceedings of Eye Tracking Research & Applications Symposium 2000, pages 29 -- 36. ACM, 2000.

Hayhoe, Mary M. , Ballard,Dana H., Triesch, Jochen, Shinoda, Hiroyuki, Aivar, Pilar, and Sullivan, Brian. “Vision in Natural and Virtual Environments”. In Proceedings of Eye Tracking Research & Applications Symposium 2002, pages 7 -- 13. ACM, 2002.

Jacob. Robert J.K., "Eye Tracking in Advanced Interface Design," HCI Lab, Naval Research Laboratory http://www.eecs.tufts.edu/~jacob/papers/barfield.html

Josephson, Sheree (Weber State University, USA) and Holmes, Michael E. (Ball State University). “Visual Attention to Repeated Internet Images: Testing the Scanpath Theory on the World Wide Web”. In Proceedings of Eye Tracking Research & Applications Symposium 2002, pages 43 -- 49. ACM, 2002.

Description of a study that challenges the Scanpath Theory, concluding that other forces such as visual features or memory may be of importance.

Lewenstein, M., Edwards, G., Tatar, D., and DeVigal, A. Stanford-Poynter Project. See http://www.poynter.org/eyetrack2000/index.htm. 2000.

Schroeder, W. (1998, September/October). “What Is Eye Tracking Good for?” User Interface Engineering. [On-Line] Available URL: http://world.std.com/~uieweb/eyetrack2.htm

Schroeder, W. (1998, September/October). “Testing web sites with eye-tracking”. User Interface Engineering. [On-Line] Available URL: http://world.std.com/~uieweb/eyetrack1.htm

Vertegall, Roel. “Designing Attentive Interfaces”. In Proceedings of Eye Tracking Research & Applications Symposium 2002, pages 23 -- 30. ACM, 2002.