INFLUENCE OF PARIETAL CORTEX ON SALIENCY DRIVEN VISION IN NON HUMAN PRIMATESEshan GovilSURF Summer 2016Moore Labs at Stanford UniversityMentors: Xiaomo Chen, Mark Zirnsak, and Tirin Moore

Overview• 1 – Experimental Paradigm

• What was the experiment design• 2 – Basic Behavior

• Shift to the right• Main Sequence

• 3 – Optimization• IKN and GBVS model

• 4 – Saliency Driven fixations• Proportion as a function of Saliency Difference• Average saliency through the fixation sequence

• 5 – Future Direction

Key Terms• Saliency / Physical Salience

• The unique features of an image we subconsciously focus on when gauging dense sensory input to determine what we are looking at

• Saccade• Rapid, almost involuntary eye movements from one point to

another• Parietal Cortex

• Sensory input processing and integration• Frontal Eye Field

• Responsible for saccadic and voluntary eye movements, visual field perception, and awareness

• Ipsilateral/Contralateral• To the same side / to the other side of the body

Experimental Background and Design

EXPERIMENTAL GOALObserve and quantify the effects of reversible inactivation

of the parietal cortex on the processing of visual salience in the FEF.

IN GENERAL TERMSIf we switch off a region of our Parietal Cortex, how does that affect our vision? How does the Frontal Eye Field get

involved during inactivation?

Experimental Background and Design

http://www.vanderbilt.edu/exploration/text/index.php?action=view_section&id=1192&story_id=289&images=

Fixation

Time

5 s

Fixation

Reward

Free viewing task

Control Block (30 min)

Inactivation Block (30 min)

Fixation

Time

5 s

Fixation

Reward

Left visual hemifieldRight visual hemifield

Behavioral biases induced by parietal inactivation

Bias for eye movementsaway from the inactivatedhemifield

1. Free viewing task2. Two-target free-choice task

Two behavioral tests:

e.g.,Lynch & McLaren 1989Karnath et al. 1998

Intraparietal Sulcus (IPS)Inactivation

ControlNumber of Fixations

Left: 5 Right: 3Number of Fixations

Left: 1 Right: 10

Inactivation

Free viewing task: Example image

Scanpath of the eyes Scanpath of the eyes

Basic Behavior• Observed: IPS Inactivation in the right hemisphere of the

brain -> eye movements biased away from contralateral field• Ipsilateral shift in horizontal fixations• Both in free viewing and two target free choice task

2D Density Heatmap - Control

X axis (dva)

Y a

xis

(dva

)

Den

sity

X axis (dva)

Den

sity

Y a

xis

(dva

)

2D Density heatmap - Experimental

X axis (dva)

Den

sity

(Inc

or D

ec)

Y a

xis

(dva

)

2D Density Heatmap – Exp v Control Difference

Magnitude (degrees)

Dur

atio

n (m

s)

Magnitude (degrees)

Pea

k Ve

loci

ty (d

egre

es /

sec)

Optimization – GBVS toolboxIn order to compare fixation data to calculated visual saliency that, through optimization, will be accurate to how our vision typically identifies saliency

• Parameters optimized: • IKN - center and surround levels• GBVS – feature map resolution

• More parameters to choose from but did not optimize

Optimization• Default

• p.levels = [2 3 4];• p.ittiCenterLevels = [2 3 4];• p.ittiDeltaLevels = [3 4];

• Optimized• p.levels = [2 3 5 6 7 8 9];• p.ittiCenterLevels = [2 5 6];• p.ittiDeltaLevels = [1 3];

• Optimized across random set of images across all experiments

Pre-optimization

Post-optimization

Saliency Driven Vision• Previously observed: following inactivation, responses to

visual stimuli are reduced compared to control• Currently observed: responses to visual stimuli are about

the same in some cases

Fixations in order

Average Saliency (Left) per Fixation in order of Fixations across all ExperimentsA

vera

ge S

alie

ncy

Average Saliency (Right) per Fixation in order of Fixations across all Experiments

Fixations in order

Ave

rage

Sal

ienc

y

Saliency Difference (Right field – Left field)

Pro

porti

on o

f Fix

atio

ns in

Rig

ht fi

eld

GBVS - % Fixations to the Right as a function of Saliency

Future Direction• Run my code on future data sets

• Validate my results and look for greater significance• Improve and formalize the figures created and findings

from this data set• Separate images used in optimization from the data set• Run all tests for both GBVS and IKN

Acknowledgements• My mentor Dr. Xiaomo Chen• Dr. Tirin Moore, Dr. Marc Zirnsak, and all other members

of the Moore lab• Stanford University• the Caltech SFP program