Effects of Viewing Geometry on Combination of Disparity and Texture Gradient Information

Michael S. Landy

Martin S. Banks

James M. Hillis

Outline

• Background: Optimal cue combination

• Methods: slant discrimination

• Single-cue results

• Two-cue results: perceived slant

• Two-cue results: JNDs

• Conclusions

Outline

• Background: Optimal cue combination

• Methods: slant discrimination

• Single-cue results

• Two-cue results: perceived slant

• Two-cue results: JNDs

• Conclusions

Sources of Depth Information

• Motion Parallax

• Occlusion

• Stereo Disparity

• Shading

• Texture

• Linear Perspective

• Etc.

Depth Cues

• Motion Parallax

• Occlusion

• Stereo Disparity

• Shading

• Texture

• Linear Perspective

• Etc.

Optimal Cue Combination:Statistical Approach

If the goal is to produce an estimate with minimal variance, and the cues are uncorrelated, then the optimal estimate is a weighted average

where

ˆ ˆ ˆ ,t t d dS w S w S

2 2

2 2 2 2

1/ 1/and .

1/ 1/ 1/ 1/t d

t dt d t d

w w

Optimal Cue Combination:Bayesian Inference Approach

From the Bayesian standpoint, the measurements D and T each result in a likelihood function

( | ) and ( | ).p T S p D S

These are combined with a prior distribution

( ).p S

Optimal Cue Combination:Bayesian Inference Approach

( | , ) ( | ) ( | ) ( ).p S T D p T S p D S p S

From Bayes rule, and assuming conditional independence of the cues, the posterior distribution satisfies:

Optimal Cue Combination:Bayesian Inference Approach

ˆ ˆ ˆ ,t t d d p pS w S w S w S

where p stands for the prior which acts as if it were an additional cue, and the weights are again proportional to inverse variance.

Finally, assuming Gaussian likelihoods and prior, it turns out that the maximum a posteriori (MAP) estimate satisfies:

Previous Qualitative Tests that Cue Weights Depend on Reliability

• Young, Landy & Maloney (1993)

• Johnston, Cumming & Landy (1994)

• Rogers and Bradshaw (1995)

• Frisby, Buckley & Horsman (1995)

• Backus and Banks (1999)

• etc. etc.

Previous Quantitative Tests that Cue Weights Depend on Reliability

• Landy & Kojima (2001) – texture cues to location

• Ernst & Banks (2002) – visual and haptic cues to size

• Gepshtein & Banks (2003) – visual and haptic cues to size

• Knill & Saunders (2003) – texture and disparity cues to slant

The Current Study

• Texture and disparity cues to slant

• Vary reliability by varying base slant (as in Knill & Saunders, 2003) and distance

• Measure single-cue reliability

• Compare two-cue weights to predictions

• Compare two-cue reliability to predictions

Outline

• Background: Optimal cue combination

• Methods: slant discrimination

• Single-cue results

• Two-cue results: perceived slant

• Two-cue results: JNDs

• Conclusions

Types of Stimuli

• Disparity-only: sparse random dots

• Texture: Voronoi textures viewed monocularly

• Two-cue stimuli: Voronoi texture stereograms, both conflict and no-conflict

Stimuli – Disparity-only

Stimuli – Voronoi textures

Cue Conflict Stimuli

Methods

• Task: 2IFC slant discrimination• Single-cue and two-cue blocks• Opposite-sign slants mixed across trials in a

block to avoid slant adaptation• One stimulus fixed, other varied by

staircase; several interleaved staircases• Analysis: fit psychometric function to

estimate PSE and JND

Outline

• Background: Optimal cue combination

• Methods: slant discrimination

• Single-cue results

• Two-cue results: perceived slant

• Two-cue results: JNDs

• Conclusions

Single-cue JNDs: Texture

Single-cue JNDs: Disparity

Single-cue JNDs: Disparity

Predicted Cue Weights

Outline

• Background: Optimal cue combination

• Methods: slant discrimination

• Single-cue results

• Two-cue results: perceived slant

• Two-cue results: JNDs

• Conclusions

Cue Conflict Paradigm

Determination of PSEs

Determination of Weights

Full Two-Cue Dataset

ACH JMH

Effect of Viewing Distance

Effect of Base Slant

Outline

• Background: Optimal cue combination

• Methods: slant discrimination

• Single-cue results

• Two-cue results: perceived slant

• Two-cue results: JNDs

• Conclusions

Improvement in Reliability with Cue Combination

If the optimal weights are used:

then the resulting variance

is lower than that achieved by either cue alone.

2 2

2 2t d

t d

2 2

2 2 2 2

1/ 1/and

1/ 1/ 1/ 1/t d

t dt d t d

w w

Improvement in JND with 2 Cues

Conclusion

• The data are consistent with optimal cue combination

• Texture weight is increased with increasing distance and increasing base slant, as predicted

• Two cue JNDs are generally lower than the constituent single-cue JNDs

• Thus, weights are determined trial-by-trial, based on the current stimulus information and, in particular, the two single-cue slant estimates

Are Cue Weights Chosen Locally?

Are Cue Weights Chosen Locally?