aks-memory across eye-movements 1f dynamic in visual search

8/12/2019 Aks-Memory Across Eye-Movements 1f Dynamic in Visual Search

1/111

Memory across Eye-Movements:1/f Dynamic in Visual Search

Aug 5, 2001Madison, Wisconsin

---------------------------------- Society for Chaos Theory in

Psychology & the Life Sciences

Deborah J. AksUW-Whitewater

Gregory ZelinskySUNY- Stonybrook

Julien C. Sprott UW-Madison


2/111

What guides eye-movements during complicated

visual search?


3/111

Deterministic rules?

Simple set of neuronal interaction rules (e.g.,SOC) ?

Memory?

Are there correlations across sequence of

fixations?


4/111

Find:


5/111


6/111


7/111

Do we keep track of where we look?

Is there memory in search?


8/111

Horowitz, T.S. & Wolfe, J. M. (1998). Visual Search has no memory . Nature, 357, 575-577.

Finding: Random repositioning of stimulidoes not affect search RTs

No memory?


9/111

Overview QUESTIONS .

What guides complicated eye movements? Random or non-random process? Is there memory across fixations?

METHOD OF TESTING . Challenging visual search task

KEY ANALYSES Power law relation? Coloring of noise --> Memory across eye-movements

Fourier analysis Iterated Functions Systems (IFS) Test


10/111

RESULTS

Raw fixations --> short term memory (1/f 2 brown noise) Fixation differences --> long term memory (1/f pink noise)

MODEL . Self-organized criticality (SOC)(Bak, Tang, & Wiesenfeld, 1987)

CONCLUSION There is memory across eye-movements! SOC model predicts relative eye movements.


11/111

Horowitz, T.S. & Wolfe, J. M. (1998). Visual Search has no memory . Nature, 357, 575-577.

Finding: Random repositioning of stimulidoes not affect search RTs

Key Press RTsvs.Eye Movements


12/111

What does visual search teach us? Cognitive processes!

Speed & Accuracy

Mechanisms Automatic or Attention

Search strategy Parallel, Serial, random or ?


13/111

Features... Find the odd item Discriminate by..

Color xxxx xxx

Size xxxxxxx Orientation ------l---

Depth

Movement xxxx--> x


14/111

Look for the red L

L

L

L L

L


15/111

L

L

L L

L

L L L

L


16/111

Conjunction Search

Find...combination of features

2 orientations (particular arrangement) Find: L among Ts

T

T

T

L T


17/111

T

T

T

T

T

T T

L T

T T

T

T


18/111

+


19/111

T

T

T

T

T

T T

T

T T

T

T

L


20/111

300

400

500

# of items

R T ( m s e c

)

5 10 15

0 ms/item

Feature search is easy!

Fast (300ms) Parallel (0-10ms/item) No attention needed


21/111

300

500

700

# of items

R T ( m s e c )

5 10 15

40 ms/item

0 ms/item Feature

Conjunction

Conjunction search is hard! Slow er Sequential Focused attention needed


22/111

Feature search is easy Fast (300ms) Parallel (500ms) Serial (>10ms/item)

Attention needed


23/111


24/111


25/111


26/111


27/111

What guides search?

Environmental information.

Internal cognitive process Attention.

Memory? Deterministic Process:Self-Organized Criticality (SOC)?


28/111

Memory in visual search?

Memory for locations in

search (Kristjansso,2000)

Identity of objectsaccumulates over time(Treisman & Gelade, 1980)

Random repositioning ofstimuli does not affectsearch RTs(Horowitz & Wolfe, 1998)

Inattentional amnesia insearch (Wolfe, 1999)

Evidence for Evidence against

We are able to keep track of

where we look! Inhibition of return(Klein, 1982)

Failure to replicate

inhibition of return(Wolfe & Pokorny, 1990)


29/111

Non-systematic eye-movements

Engle, 1977; Ellis & Stark, 1988;Scinto & Pillalamarri, 1986;

Krendel & Wodinsky, 1960; Groner & Groner, 1982


30/111

Visual Search Task

T

T

T

T

T

T

T

Find the upright T

T

T

T

T T

T

T

T

T

T

T

T T

T

T

T

T

T

T

T

T

TT

T

T

T

T

T


31/111


32/111

Map trajectory of eyes :

Duration & x,y coordinates for each fixation.----------------------------------------------------------

Differences between fixationsxn xn+1 & yn yn+1

Distance = (x2 + y 2)1/2

Direction = Arctan (y/x).


33/111

Analyses

Power spectra (FFT)

Descriptive & Correlational Statistics

Iterated Function Systems (IFS) test


34/111

Results 24 fixations per trial (on average)

7.6 seconds (SD =6.9 sec) per trial (316 ms/item).

Mean fixation duration = 212 ms (SD = 89 ms)

10,215 fixations across complete search experiment.


35/111

Series of Fixation Differences(yn+1- yn)


36/111

Scatter plot of 10,215 eye fixations forthe entire visual search experiment.

Eye Fixations


37/111

Delay Plot of Fixationsyn -vs- y n+1


38/111

Across 8 sessions we see scaling properties:

Fixation frequency decreased from 1888 to 657

Fixation duration increased from 206 to 217 ms.

Fixation differences xn xn+1 decreased yn yn+1 increased


39/111

Spectral analysisFast-Fourier Transform (FFT)

Power vs. FrequencyRegression slope = power exponent

f a

f -2 = 1/ f 2

Brown noise


40/111

Power law indicates

Adaptive fractal properties: Scale invariance Flexible system

Strength of memory Steepness of the slope (on a log-log scale) reflects..

correlation across data points duration of memory


41/111

White Noise

Pink Noise

Brown Noise

1/f 0 noise -- flat spectrum= nocorrelation across data points

1/f noise --shallow slope =extremely long term correlation

1/f 2 noise-- steep slope = short-term correlation.


42/111

Power Spectra on raw fixations

a


43/111

Power Spectra of first differencesacross fixations

a = -.6

Di fi i


44/111

Distance across eye fixations(x2 + y 2) 1/2

a = -.47

a = -0.3

a = -1.8

Iterated Function Systems


45/111

Iterated Function Systems--IFS Test--

(Peak & Frame, 1994 ; Stewart, 1989 ).Fixation Series


46/111

1

2 3

4

Start


47/111


48/111

1

2 3

4


49/111

1

2 3

4


50/111

1

2 3

4


51/111

1

2 3

4


52/111

1

2 3

4


53/111


54/111

1

2 3

4


55/111

1

2 3

4


56/111

1

2 3

4


57/111

1

2 3

4


58/111

1

2 3

4


59/111

1

2 3

4


60/111

1

2 3

4


61/111


62/111

1

2 3

4


63/111


64/111


65/111

1

2 3

4


66/111

1

2 3

4


67/111

1

2 3

4


68/111

1

2 3

4


69/111

1

2 3

4


70/111

1

2 3

4


71/111

PinkNoise

Brown Noise

White Noise

1/f

1/f

1/f

Raw Fixations


72/111

Brown noise pattern

Clustering along diagonals reveals short-term, highly correlated consecutive data points

Fixation differences


73/111

Fixation differences

Pink noise

Triangular microstructure associated w/ long-term, loosely correlated consecutive data points

IFS Test: Fixation direction


74/111

Clustering indicates correlated fixations. Direction of fixations show

anti-correlated movements a indicated by absence of main diagonals.

IFS Test: Fixation direction


75/111

IFS tests yields patternsconsistent w/ FFT results


76/111

Summary of results :

Sequence of

Absolute eye positions --> 1/f brown noise Short-term memory.

Differences-between-fixations --> 1/f pink noise Longer-term memory.


77/111

ModelHebb, 1969; Rummelhardt & McClelland, 1985

Neuronal interactions --->

implicit guidance

Could eye movements be described by a simple set ofneuronal interaction rules (e.g., SOC) that produce 1/f

behavior?


78/111

Increasing Neural Activation --->

0 4

SOC Network(Adapted from Bak, Tang, & Wiesenfeld, 1987)


79/111

Stimulate 1 neuron


80/111

Z(x,y)= initiallystimulated site

As individual neurons are activated beyond a threshold (of 3), activity (4) is

dispersed to surrounding cells.

Threshold rule: For Z(x,y) > Zcr =3


81/111

Activity in theoriginal site is

depleted to zero.

Z(x,y) -> Z(x,y) - 4


82/111


83/111


84/111


85/111

0 4

Neural SOC

l SOC il


86/111

Neural SOC w/ eye movements trails

Eye movements are pulled to the


87/111

site(s) of greatest activation


88/111


89/111


90/111


91/111


92/111


93/111

For Z(x,y) > Zcr


94/111

Complex & effective search

( ,y) Z(x,y) -> Z(x,y) - 4 Z(x + 1,y)-> Z(x + 1,y) + 1

Z(x,y+

1) -> Z(x,y+

1) + 1Simple set of SOC rules..

{

can produce:


95/111

CONCLUSIONS

There is memory across eye-movements!

Neural SOC model --> 1/f relative eye-movements.

Simple self-organizing system--> effective search


96/111

http://psychology.uww.edu/Aks/papers/AZS01.ppt

Aks, D. J. Zelinsky G. & Sprott J. C. (2002). Memory Across Eye-Movements: 1/f Dynamic in Visual Search. Nonlinear

Dynamics, Psychology and Life Sciences, 6 (1).


97/111

Bluebird contributed bywww.Sierra foothill.org


98/111

And thanks to Bob Goodman for,among a # of things,

getting me to reduce the # of slides in this talk.

Phew, Debs down to 93 slides


99/111


100/111


101/111


102/111


103/111


104/111


105/111


106/111


107/111


108/111


109/111


110/111

References

Aks, D.J., Nokes, T. Sprott, J.C. & Keane, E. (1998). Resolvingperceptual ambiguity in the Necker Cube: A dynamical systemsapproach Abstracts of the Psychonomics Society 3 38


111/111

approach. Abstracts of the Psychonomics Society, 3, 38.Allport, D.A. (1987). Selection-for-action: Some behavioral and

neurophysiological considerations of attention and action. In H.Heuer & A.F. Sanders, (Eds.), Perspectives on Perception andAction. Hillsdale, N.J.: Erlbaum.

Bak, P. (1996). How nature works: the science of self-organizedcriticality. New York: Springer-Verlag.

Bak, P., & Tang, C. (1989). Earthquakes as a self-organizedcritical phenomenon. Journal of Geophysics Research - Solar.Earth Planet, 94, 15635-15637.

Bak, P., Tang, C., & Wiesenfeld, K. (1987). Self-organizedcriticality: An explanation of 1/f noise. Physical ReviewLetters, 59, 381-384.

Bak, P., Tang, C., & Wiesenfeld, K. (1988). Self-organizedcriticality. Physical Review A, 38, 364-374.

Daugman, J. G. (1991). Self-similar oriented wavelet pyramids:Conjectures about neural non-orthogonality. In A. Gorea (Ed.),Representations of Vision. Cambridge: Cambridge University Press.

Ellis, S. R., & Stark, L. (1986). Statistical dependency in visualscanning. Human Factors, 28(4), 421-438.

Gilden, D. L. (1996). Fluctuations in the time required forelementary decisions. Psychological Science, 8 (4), 296 302.

Gilden, D. L., Thornton, T., & Mallon, M. (1995). 1/f noise inhuman cognition. Science, 267, 1837-1839.

Hilborn, R. C. (1994). Chaos and Nonlinear Dynamics: Anintroduction for Scientists and Engineers. Oxford: OxfordUniversity Press.

Hochberg, J. (1968). In the minds eye. In R. N. Haber (Ed.),

aks-memory across eye-movements 1f dynamic in visual search

Documents