Bayesian integration in sensorimotor learning

Introduction

● Learning new motor skills○ Variability in sensors and task

● Tennis: Velocity of ball○ Not all are equally probable over time○ Increased uncertainty: fog -> rely on prior knowledge

● Bayes strategy○ Prior belief & uncertainty

Apparatus● 6 male & 4 female subjects● Reach a visual target with the finger

○ Projection mirror prevented direct view

○ Reaching movements on a table with an Optotrak 3020

tracking system measuring the position of their right index finger

○ Showing projection of finger -> displace feedback of finger location

○ Feedback midway: white sphere○ Green: target - Blue: starting point

Procedure● Feedback given four different ways

○ ○○○

: No added uncertainty

● Each subject: 2000 trials:○ Only last 1000 trials for the analysis○ Shift was randomly drawn from gaussian

distribution (True prior)■ Mean 1 cm to right■ Standard deviation 0.5

○

: Medium uncertainty: Large uncertainty: Feedback withheld

Giving the final deviation at the end of only clear feedback trials

Possible Computation Models

● Full compensation model

● Bayesian probabilistic model

● Mapping model

Full Compensation Model

● Full compensation by subjects for the visual estimate of the lateral shift

● The pointing variability but not the average location affected by increasing the lateral shift feedback uncertainty

● No requirement of estimation for the visual uncertainty or the prior distribution of shifts.

Bayesian Model

Optimal use of information about the prior distribution and the uncertainty of the visual feedback to estimate the lateral shift

Sense for Bayesian statistics

● Sensed a lateral shift of 2 cm due to any of the many true lateral shifts● True lateral shift: 1.8 cm (Error of +0.2 cm) vs 2.2 cm (Error of -0.2 cm)● Errors equally probable for gaussian visual feedback noise ● 1.8 cm lateral shift more probable than a 2.2 cm shift given the prior distribution

having mean of 1 cm. ● Depends on two factors:

○ Prior distribution ○ Degree of uncertainty in the visual feedback.

Mapping model

● Learn mapping from the visual feedback to an estimate of the lateral shift

● Error minimized over repeated trials and without any explicit representation of the prior distribution or visual uncertainty

● Requires knowledge of the error at the end of the movement to learn mapping

● The shifted finger position revealed at the end of the movement on the clear feedback trials ( ). The same mapping applied to the blurred conditions ( , ).

● The average shift of the response towards the mean of the prior same for all amounts of blur

Comparison between 3 models● Basis: The effect of the visual feedback on the final deviation from the target

○ Model 1: The average cursor lateral deviation from the target should be zero for all conditions.

○ Model 2: The estimated lateral shift should move towards the mean of the prior by an amount that depends on the sensory uncertainty.

○ Model 3: Predicts that subjects should compensate for the seen position independently of the degree of uncertainty.

● Comparing theoretical MSE for the three models shows that it is minimal for model 2. Even though model 1 is on average on target, the response variability is higher than in model 2 , leading to a larger MSE

Bayesian Model Analysis

● Prior

● Likelihood

● Posterior

Prior

Sensed observation

Combined estimate

Given that we know ; we can estimate the uncertainty in the feedback by linear regression

MSE

Model 1:

Model 2:

Visual feedback noise variation

● At minimal noise, the estimate distribution follows the sensory distribution.● As noise increases, the estimate distribution starts to approach the prior

distribution.

Slope variations for individual subjects

● The slopes for the linear fits are shown for the full population of subjects.

● Planned comparisons of the slopes between adjacent uncertainty levels were all significant

Bias vs slope

● No deviation from the target if the true lateral shift is at the mean of the prior

● Predicts that the sum of the slope and offset should be zero

● x(estimated) = w1*1 + w2*x(sensed)● Bias = E[x(estimated)-x(true)] = w1[1-x(true)]● Offset = w1, Slope = -w1

Inferring Prior

The derivative of this posterior with respect to x(true) must vanish at x(estimated).

Using derivative of log(p(x(true))) and integrating with certain operations, the true prior can be obtained

Bimodal distributions

● New prior● Is learning complex distributions possible?

○ Two gaussians

Bimodal distributions

● Dashed line: Single gaussian● Solid line: Two gaussian fit

○ Shows a nonlinear relationship

● Showing one subject

Bimodal distributions

● Trying to fit data with linear regression○ Showing the non-linear relationship

● Shows average over subjects

Conclusion

● Proposed: Nervous system fits with bayesian○ Model: Sensorimotor consistent with neurophysiological

studies analyzing that estimating reward has uncertainty

● Only tested on visuomotor displacement○ Expected to work with all sensorimotor control & learning○ Tennis match : Prior knowledge about other player