A model for detecting illusory contours

Caveat: Response decreases gradually withincreasing intermeshing of gratings. This does notcorrespond with subjective percepts.

(Peterhans and von der Heydt, 1989)

Convergence of cues in contour coding(Studies with Tilt AfterEffect)

Adaptingstimulus

Teststimulus

TAE is orientation specific and exhibits inter-ocular transfer.Therefore, TAE is likely to be of cortical origin.

Does the TAE transfer from one kind of contour (say ‘real’) to another(say ‘illusory’, or ‘motion defined’)?

Yes.(Berkley, DeBruyn & Orban, 1994;Paradiso, Shimojo & Nakayama, 1989)

Convergence of cues in contour coding

From zero-crossings to a raw primal sketch (collection of tokens)

Zero crossingsat a coarse scale

Zero crossingsat a fine scale

‘Blobs’Regions whereZero crossings

Form closed contours

‘Edge segments’Regions of coincidenceof ZCs acrossscales

From a raw primal sketch to a full primal sketch

Requires contour linking and extracting globally salient structures

What assumptions does the visual system use to linkEdge fragments and identify ‘important contours’?

1. Collinearity (aligned fragments typically belong to the same contour)2. Length (longer contours are more salient)3. Low curvature4. Completion (closed contours are more salient)5. Object recognizability (controversial)

Ideas from psychophysical studies

Computing saliency

How to assign a saliency measure to each edge fragment?

S (p) = w iii

i = 1 if element present= 0 otherwise

w = ei-c i Ci is the total curvature

of the contour from pupto the ith element

p

i1

2

3

Wi is 1 for a straight line and decreases as the curvature increases-penalizes high curvature-Weights distant elements less than closer ones

[Ullman and Shashua, 1988]

Final saliency at p

S(p) = max S (p)

Very high computational requirements, but can be implemented Highly efficiently in a locally connected network. [Ullman, 1995]

p

i1

2

3

Computing saliency – some results

Psychophysical and physiological correlatesOf saliency computations

-Fields et al-Polat and Sagi

Facilitation ofdetection by

aligned neighboringelements

Gilbert and Wiesel found long range connections between similarlytuned hypercolumns that might subserve the perceptual facilitation.

From a raw primal sketch to a full primal sketch

Requires segregating regions with different textures

Moving beyond the first stages of image-processing…

Information about object shape and surface properties

Open questions about early processes

Do texture segregation processes work on the 2D image or at the level ofSurface representation?

Processing Framework Proposed by Marr

Recognition

ShapeFromstereo

ShapeFromstereo

Motionflow

ShapeFrom

motion

Colorestimation

ShapeFrom

contour

ShapeFrom

shading

ShapeFrom

texture

3D structure; motion characteristics; surface properties

Edge extraction

Image

Emphasis on‘Bottom-up’processing

Stereopsis

Challenges:1. Trigonometric calculations2. Correspondence problem

The process of perceiving the relative distance to objects based on theirLateral displacement in the two retinal images (aka binocular disparity).

Other potential cues to depth (convergence and accommodation) do not seem to be tooImportant for humans.

The Stereo Correspondence Problem

“During binocular regard of an objective image, each uniocularMechanism develops independently a sensual image of considerableCompleteness. The singleness of binocular perception results from The union of these elaborated uniocular sensations. The singleness idTherefore the product of a synthesis that works with already elaboratedSensations contemporaneously proceeding.”

- Sherrington, 1906

The Stereo Correspondence Problem

Hand Hand

However, different looking images could also be fused…

“Although a perfect stranger to you, and living on the reverse side of the globe,I have taken the liberty of writing to you on a small discovery I have made inBinocular vision in the stereoscope. I find by taking two ordinary photos of twoDifferent persons’ faces, the portraits being about the same sizes, and looking About the same direction, and placing them in a stereoscope, the faces blend intoOne in a most remarkable manner, producing in the case of some ladies’Portraits, in every instance, a decided improvement in beauty.”

- From a letter to Charles Darwin by A. L. Austin of New Zealand

Is monocular shape analysis a necessary pre-requisite forstereo correspondence?

Open questions:

1. Does RDS stereopsis conclusively prove that theshape-first theory is incorrect?2. Are RDSs representative of real-world scenes?3. Are RDSs completely devoid of monocular shape cues?