1

Frog Vision

Template matching as a strategy forseeing (ok if have small number ofthings to see)Template matching in spiders?Template matching in frogs?The frog’s visual parameter space

PSY305 Lecture 4 JV Stone

2

Primate Vision

The richness of a representation depends onwhat it is to be used for.Simple organisms construct simple representations.

3

Jumping SpiderRecall from PSY241 that the jumping spider has eight eyes, each of which is alens/camera type eye such as our own. Most of these eyes have fish-eye like lenses,giving them a wide field of view but blurry vision. The two forward looking eyes,though, have very high resolution (almost as good as a cat).

The jumping spider uses its low-resolution, wide field-of view eyes to detect thepresence of moving objects and prey, and then it quickly reorients its body to image theobject of interest with its high-resolution eyes (akin to foveating an object).

The jumping spider is capable of discriminating its prey (flies) from mates (otherspiders) using vision alone.

4

Jumping Spider• Mate recognition is achieved via template matching. Eyes

are fixed in carapace, and cannot move, but retinae can.• Each retina has )-shaped region, so that two retina together

yield )( or X-shaped region in visual field. The two retinaemove in a conjugate manner (i.e. movements of both eyes arethe same).

• Retinae are scanned over image to see if object is a mateLand (1969).

5

Jumping Spider

Evokescourtship

EvokesPrey capture

Drees (1952) cited in Land (1969)

6

What the frog’s eye tells thefrog’s brain (Lettvin Maturana,

McCulloch and Pitts, 1959)• Seminal paper.• McCulloch and Pitts produced seminal 1943

connectionist paper, so were accustomed tothinking about building models.

McCulloch, W. S. and Pitts, W. H. (1943).A logical calculus of the ideas immanent in nervous activity.Bulletin of Mathematical Biophysics, 5:115-133.

7

Frog Brain

Front Tail

Optic tectum

Optic tectum

View from above

Side vew

8

Frog Visual System

• No fovea.• Retina projects to superior colliculus, also called

tectum.• Requires movement to see food.• 1 million rods and cones project to 0.5 million

retinal ganglion cells. These project to tectum.– (human has 126M receptors and 1M ganglion cells

which project to LGN)

9

Frog Visual System

Eyes have overlapping views in tiny region of scene (unlike humans).Complete cross-over of optic nerve at chiasm.

EyeOptic tectumorSuperior colliculus

Spinal chord

Optic nerve

10

Method• Frog faces inside of hemisphere 14 inches

in diameter.• Stimulus moved about with magnets on

outside of hemisphere.Stimuluson inside of hemisphere

11

Optic angle and image size• The size of the image of an object on the retina is proportional

to the angle subtended by that object.

• An optic angle of one degree can be subtended by a smallnearby object or by a large distant object.

• Sun or moon = 0.5 degree• Fingernail at arm’s length = 1 degree.

1 degree

Eye

Retinal image

Same sizedretinal images

12

Four Concentric Ganglion CellReceptive Field (RF) Types

1 Sustained contrast detectors (2 degrees, unmyelinated)2 Net convexity detectors (1-3 degrees, unmyelinated)

3 Moving edge detectors (12 degrees, myelinated)4 Net dimming detectors (15 degrees, myelinated)

30 times as many of (1,2) as (3,4) - reflects RF size.Types 1-4 reflect depth in tectum (with 1 at surface).Myelinated fibres transmit information more rapidly than

unmyelinated fibres - consistent with detection ofstatic/changing contrast above.

13

1 Sustained Contrast Detectors

• Figure 2.• Unmyelinated, 2 degrees RF size.• Sustained response to any static contrast

edge.• Response to moving edge in one direction

only (2a top trace).• Not respond to general changes in

illumination.

14

1 Sustained Contrast Detectorsare Sensitive to Motion Direction

LuminanceWithin RF

Action potentials

Direction of motion of disc across RF

3 degree disc moved across RF in one direction, then the other.

Figure 2a in paperTime

Motion

15

1 Sustained Contrast DetectorsResponse to Static Edge

3 degree disc moved into RF and stopped => sustained response.

Timebase = 50ms

Luminance Within RF

Sustainedfiring

Figure 2b in paperTime

16

2 Net Convexity Detectors: bugdetectors?

• Figure 3. So-called bug detectors.• Unmyelinated.• Transient response to <3 degree moving disc.

Response small if <1 degree.• Response large if motion is ‘jerky’ (like a fly?).• No response to moving straight edge.• No response to whole-field motion of array of

dots, unless one dot moves wrt whole-fieldmotion (i.e. responds to moving dot only if motionis relative to whole-field). Whole-field motion canbe caused by frog moving.

17

2 Net Convexity DetectorsResponses to Moving Disc and Edge

Response to 1 degree disc at 3 speeds (left to right).

Response to straight edgeat 2 speeds (no response).

Luminance in RF (up => dim, here)

Also respond to static 1 degree disc (figure 3a).

Figure 3e and 3f in paper.

Time

18

3 Moving-Edge Detectors

• See figure 4.• 12 degree RF.• Responds only if edge (bar) is moving.• Firing rate increases with speed of edge.

19

3 Moving-Edge Detectors

Luminance within RF (up => dim).

Time

TransientFiring

From figure 4e in paper.

20

4 Net Dimming Detectors• See Figure 5 in paper.• 15 degree RF size• Sustained response to general (lack of)

illumination

Dark

IlluminationLevel(up => dimmer)

Firing

Figure 5d in paper.

21

Recoding the Retinal Image

• Different cell types encode different types ofinformation.

• Output of any single cell is inherently ambiguousin terms of what it signals about the retinal image(NEXT SLIDE).

• For example, may need population of bug detectorcells to signal exact location of moving dot - seepopulation coding.

• May also need to take account of output ofmoving edge detectors - these provide contextualinformation …

22

Cell’s Output Does Not SpecifyExactly What is in Cell’s RF

A spot of light here or here gives identicalinput to neuron and identical firing rate.

Edge of retina

23

What a tectal neuron ‘sees’

• “Consider the dendrite of a tectal cellextending up through four sheets” (p257),where each sheet of neurons encodes one offour different types of feature (i.e.parameter) in the same retinal location.

• Each retinal stimulus generates a differentsignature, depending on how it stimulatesneurons in each of the of 4 sheets.

24

What a tectal neuron ‘sees’ iftectum had only two layers

Bug detector sheet

Edge detector sheetPath of tectal dendritethrough tectum

Receptive field of bug detector

Image on retina

Retinotopic map

25

Definition: Population andAssembly Coding

• Population coding: If a population of cells allrespond to the same parameters then theircollective output is called a population code.– For example, all cells within a tectal sheet respond to

same parameters => population code.• Assembly coding: If two cell populations respond

to different parameters then their collective outputis called an assembly code.– For example, cells in different tectal sheets respond to

different parameters => assembly code.

26

Recoding the Image

• (A little speculative connectionism).

27

What a tectal neuron would ‘see’if tectum had only two layers

Bugdetectingneuron

Edge detecting neuron

To spinal chord

Tectal neuron

-1+1

Retinal image

Synapticstrength

28

What a tectal neuron would ‘see’if tectum had only two layers

• Assume that the tectal neuron has a connection strength of +1 to thebug detector and -1 to the edge detector.

• If bug detector neuron is firing (denote this as +9) and edge detector istoo (also +9) then may not infer a bug is present (because bugdetector’s output is a ‘false alarm’).

• This would yield a total input to the tectal neuron of 1x9 + (-1x9) = 0.• But if bug detector neuron is firing but edge detector is not firing then

may infer a bug is present.• This would yield a total input to the tectal neuron of

+1x9 + (-1x0) = +9, and the tectal neuron would fire,and the frog would stick out its tongue.

-1+1

29

From Image Space to Feature(or Parameter) Space

• Degree of bugness == output of bug detector• Degree of edgeness == output of moving-edge

detector• Any image stimulates each detector to different

extent, and therefore represents different point in2D bug-edge parameter space.

• Thus detectors implement recoding of retinalimage space into useful 2D feature space. If havethree detector types then parameter space is 3D.

30

2D Parameter Space

Degree of Edgeness

Deg

ree

of b

ugne

ss x

x

x

Very like a contrast edgeUnlike a bug

Very like a bugUnlike an edge A bit like an edge and a bug

Each point in parameter space represents the shape of a retinalstimulus presented to the same retinal location.

31

Tectal Neuron’s RF in 2D Parameter SpaceVery like a bugUnlike an edge

Degree of Edgeness

Deg

ree

of b

ugne

ss x

x

x

Each tectal neuronresponds according tooutput of retinal neuronswhich detect ‘lower order’parameters (bug andedges), so the RF of thetectal neuron can bedefined as a region orparameter field (yellowdisc) in parameter spacewhich corresponds to bug-and-not-edge features.

32

Two types of feature detectorneurons define a 2D feature space

• Require M=Nk neurons to tile 2Dparameter space. If k=2 and N=7(as here) then need M= N2=49tectal neurons to tile 2Dparameter space.

• Can’t draw 4D feature space, butwe know that M=Nk neurons arerequired to code each of kparameters over N values of eachparameter.

• Therefore require M=Nk totile 4D parameter space.

• Would need N4=74=2401 tectal cellsper retinal location to ‘tile’ 4D space.

Degree of Edgeness

Deg

ree

of b

ugne

ss

x

x

x

N=7

1 2 3 4 5 6 7

33

Summary 1• The four tectal sheets of neurons essentially

provide a recoding of the retinal image.• The retinal image is specified in terms of

luminance at each receptor - this description isredundant and not useful to frog.

• Tectal neurons recode each small region on retinain terms of 4 basic features or parameters, so wehave 4 different average firing rates per retinallocation.

• Unlike the retinal receptor outputs, the outputs ofthese four feature detectors are useful. Takentogether they make up the frog’s representation ofits visual world.

34

Summary 2

• Each point on retina represented in 4Dparameter space.

• Bug detector probably insufficient to signalpresence of bug, but requires contextualinformation from other 3 cell types(parameters).

• Bug detector is easily fooled …

35

ReferencesEssential• Lettvin, J.Y., Maturana, H.R., McCulloch, W.S., and Pitts, W.H.,

“What the Frog’s Eye Tells the Frog’s Brain”, Proc. Inst. Radio Engr.47:1940-1951, 1959 (supplied as handout).

Background Reading:• Land, MF, Structure of the retinae of the principal eyes of jumping

spiders (Salticidae: dendryphantinae) in relation to visual optics, J ExpBiol 1969 51: 443-470.

• Land, MF, Movements of the retinae of jumping spiders (Salticidae:dendryphantinae) in response to visual stimuli, J Exp Biol 1969 51:471-493.

• Land, MF, Animal Eyes, Oxford University Press, 2002.