Upload File

psy 5018h: math models hum behavior, prof. paul schrater, spring 2004 vision as optimal inference...

  • Home
  • Documents
  • PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as
41
PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference • The problem of visual processing can be thought of as computing a belief distribution • Conscious perception better thought as a decision based on both beliefs and the utility of the choice.

Post on 19-Dec-2015

216 views

Category:

Documents


0 download

Report

Tags:

TRANSCRIPT

Page 1: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Vision as Optimal Inference• The problem of visual processing can be thought of as

computing a belief distribution• Conscious perception better thought as a decision

based on both beliefs and the utility of the choice.

Page 2: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Hierarchical Organization of

Visual Processing

Page 3: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Visual Areas

Page 4: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Circuit Diagram of Visual Cortex

Page 5: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Motion Perception as Optimal Estimation

QuickTime™ and aVideo decompressorare needed to see this picture.

Page 6: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Local Translations

OpticFlow:(Gibson,1950)

Assigns local image velocities v(x,y,t)

Time ~100msecSpace ~1-10deg

Page 7: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Measuring Local Image VelocityReasons for Measurement Optic Flow useful:

Heading direction and speed, structure from motion,etc. Efficient:

Efficient code for visual input due to self motion (Eckert & Watson, 1993)

How to measure? Look at the characteristics of the signal

Page 8: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

X-T Slice of Translating Camera

x

t

x

yQuickTime™ and aVideo decompressorare needed to see this picture.QuickTime™ and aVideo decompressorare needed to see this picture.

Page 9: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

X-T Slice of Translating Camera

x

t

x

y

Local translation

Page 10: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Early Visual Neurons (V1)

QuickTime™ and aVideo decompressor

are needed to see this picture.

Ringach et al (1997)

y

x y

x

t

x

QuickTime™ and aVideo decompressor

are needed to see this picture.

Page 11: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

What is Motion?

As Visual Input: Change in the spatial

distribution of light on the sensors.

Minimally, dI(x,y,t)/dt ≠ 0

As Perception: Inference about causes of

intensity change, e.g.I(x,y,t) vOBJ(x,y,z,t)

QuickTime™ and aVideo decompressor

are needed to see this picture.

QuickTime™ and aVideo decompressor

are needed to see this picture.

Page 12: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Motion Field: Movement of Projected points

Page 13: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Page 14: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Basic Idea• 1) Estimate point motions• 2) use point motions to estimate camera/object motion• Problem: Motion of projected points not directly measurable.• -Movement of projected points creates displacements of image

patches -- Infer point motion from image patch motion– Matching across frames– Differential approach– Fourier/filtering methods

Page 15: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Page 16: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Page 17: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Problem: Images contain many edges-- Aperture problem

Normal flow:Motion component in the direction of the edge

Page 18: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Page 19: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Page 20: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Aperture Problem (Motion/Form Ambiguity)

Result: Early visual measurements are ambiguous w.r.t. motion.

QuickTime™ and aVideo decompressor

are needed to see this picture.

QuickTime™ and aVideo decompressor

are needed to see this picture.

Page 21: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Aperture Problem (Motion/Form Ambiguity)

However, both the motion and the form of the pattern are implicitly encoded across the population of V1 neurons.

QuickTime™ and aAnimation decompressor

are needed to see this picture.

Actual motion

QuickTime™ and aAnimation decompressor

are needed to see this picture.

Page 22: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Plaids

QuickTime™ and aGraphics decompressor

are needed to see this picture.

QuickTime™ and aGraphics decompressor

are needed to see this picture.

This pattern was created by super-imposing two drifting gratings, one moving downwards and the other moving leftwards.

Here are the two components displayed side-by-side.

Rigid motion

Page 23: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004Find Least squares solution for multiple patches.

Page 24: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Motion processing as optimal inference• Slow & smooth: A Bayesian theory for the combination of local motion signals

in human vision, Weiss & Adelson (1998)

Figure from: Weiss & Adelson, 1998

Page 25: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Page 26: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Modeling motion estimation

From: Weiss & Adelson, 1998

L(v)∝ e− w(r )(I xvx+ I yvy+ It)2 / 2σ 2

r∑Local likelihood:

Lr (v)→ p(I |θ) ∝ Lr(θ)r∏Global likelihood:

P(V )∝ e− (Dv)t (r)(Dv)(r )/ 2

r∑Prior:

P(V )→ P(θ)

Posterior: P(θ |I ) ∝ P(I |θ)P(θ)

Page 27: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Figures from: Weiss & Adelson, 1998

Page 28: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Figure from: Weiss & Adelson, 1998

Page 29: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Figure from: Weiss & Adelson, 1998

Page 30: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Figure from: Weiss & Adelson, 1998

Page 31: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Lightness perception as optimal inference

Page 32: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Illuminant

surface reflectances

( )λxyS

€

L λ( )

Surface normal NLight dir. L

€

Simple rendering model

I(x,y) = S xy λ( )r L λ( ) ⋅

r N (x,y)[ ]

I(x,y)

Page 33: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

400 500 600 700

400 500 600 700 400 500 600 700 400 500 600 700

( L , M , S )

REFLECTANCE ILLUMINANT SIGNAL

CONES

?

X =

Page 34: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Land & McCann’s lightness illusion

Page 35: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Neural network filter explanation

.

Differentiate(twice) byconvolvingimage with

"Mexican hat filter"

Lateral inhibition

Integrate

Threshold small values

Page 36: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Apparent surface shape affects

lightness perception

• Knill & Kersten (1991)

Page 37: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Inverse graphicssolution

What model ofmaterial reflectances, shape,and lightingfit the image data?

ImageLuminance

Reflectance

Shape

Illumination

ambient

point

Shape

Reflectance

Illumination

point

ambient

x

different "paint" same "paint"

Page 38: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Page 39: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Page 40: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Page 41: PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Vision as Optimal Inference The problem of visual processing can be thought of as

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004


Hum Quiz Assist

HUM Customizing

Segmentation via Generative Models - Vision Labsvision.psych.umn.edu/users/schrater/schrater_lab/courses/CompVis09/... · CSCI 5561: Computer Vision, Prof. Paul Schrater, Spring 2005

Hum w25 hum-w25

CSCI 5521: Pattern Recognition - University of …vision.psych.umn.edu/users/schrater/schrater_lab/courses/PattRecog...Prof. Paul Schrater Pattern Recognition CSCI 5521 20 Preprocessing

Series (Hum)

Hum 200 w7

Hum Exper Toxicol

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Decision Theory: Action Problems

Do Hummingbirds Hum

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2005 Making Decisions: Modeling perceptual decisions

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004

Comp. Hum. Org

Hum Acshumen1502 Retpacksd

Hum honge kamyab

Fcv hum mach_t_berg

Bancure hum

1b_Huwa Hum

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater ...vision.psych.umn.edu/users/schrater/schrater_lab/courses/...PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring

Hum t19 hum-t19

2 Vision, Psychophysics and Bayesvision.psych.umn.edu/users/schrater/SchKer00MIT.pdf · 2 Vision, Psychophysics and Bayes Paul Schrater and Daniel Kersten Introduction Neural information

Hum matamkiyoonkartayhain

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2005 Decision Theory II

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2005 Game Theory I Decisions with conflict

PSY 5018H: Math Models Hum Behavior, Prof. Paul Schrater, Spring 2004 Measurement Theory

HUM Consulting

March2013 hum

Gov Hum 04

OmManiPadme Hum

HUM Modulation

Hum Renaissance

HUM. PROJECT.docx

Katja Doerschner (1) , Paul Schrater (1,,2) , Dan Kersten (1)

Hum Miraflores

Hum Acshumen1502 Retpackmm