visual feedback in the control of reaching movements david knill and jeff saunders
TRANSCRIPT
Visual feedback in the control of reaching movements
David Knill
and
Jeff Saunders
Two types of motor control
• Ballistic
• Feedback control
Motor planning
Physicalplant
Target state
Initial system state
Motorcommands New System
states
Ballistic control
Feedback control
Motor planning
Physicalplant
Target state
Initial System state
Motorcommands New System
states
Sensorysystem
Baseball examples
• Ballistic control– Hitting– Throwing
• Feedback control– Running to catch a ball
Target
Target
Target
Target
Target
Target
Target
Target
Target
Target
Target
Target
Target
Target
Target
Target
Target
Target
Target
Target
Target
0 100 200 300 400 500 600 700 8000
100
200
300
400
500
600
700
Time (msecs.)
X position (mm)
EOG signal
Finger position
Eye-hand coordination
0 100 200 300 400 500 600 700 8000
100
200
300
400
500
600
700
Time (msecs.)
X position (mm)
EOG signal
Finger position
Movement start
0 100 200 300 400 500 600 700 8000
100
200
300
400
500
600
700
800
900
1000
Time (msecs.)
Speed (mm/sec)
Speed profile for pointing movement
Motor planning
Physicalplant
Target state
Initial system state
Motorcommands New System
states
Ballistic control
Questions
• Does the visuo-motor system use visual information about target on-line to update motor program?
Feedback control
Motor planning
Physicalplant
Target state
Initial System state
Motorcommands New System
states
Sensorysystem
Questions
• Does the visuo-motor system use visual information about target on-line to update motor program?
• Does the visuo-motor system use continuous feedback from the hand during a movement to control the movement
Questions
• Does the visuo-motor system use visual information about target on-line to update motor program?
• Does the visuo-motor system use continuous feedback from the hand during a movement to control the movement– What visual information is used?
Question
• Does the visuo-motor system use visual information about target on-line to update motor program?
Question
• Does the visuo-motor system use visual information about target on-line to update motor program?
• Yes - for detectable target motion (e.g. catching a moving object)
Question
• Does the visuo-motor system use visual information about target on-line to update motor program?
• Yes - for detectable target motion (e.g. catching a moving object)
• ?? - for imperceptible changes in target position
Experiment
• Perturb position of target during a saccade (imperceptible change)
• Does motor system correct for change in target position?
infrared markerson finger
tabletop aligned to virtual targets
monitor
mirror
0 50 100 150 200 250 300 350 400-20
-15
-10
-5
0
5
10
15
X (mm)
Y (mm)
Perturbed trials
Unperturbedtrials
0 50 100 150 200 250 300 350
-100
-50
0
50
100
X (mm)
Y (mm)
Results
• Automatically correct for imperceptible target perturbations.
• Correct for perturbations – Perpendicular to movement– In direction of movement
• Reaction time = 150 ms
• Smooth corrections
Question
• Does the visuo-motor system use continuous feedback from the hand during a movement to control the movement?
Hypotheses
• Classic model – Ballistic control during “fast” phase of motion– Feedback control during end, “slow” phase of
motion
• Continuous model– Feedback control throughout movement
Arguments against continuous feedback
• Visuo-motor delay (~100 ms) is too large for effective control during fast phase.
• Removing vision of hand early in motion does not affect end-point error.
• Corrections to target perturbations are just as strong with or w/o vision of hand.
Experiment
• Imperceptibly perturb the position of the hand during a movement and measure motor response.
• Add perturbations early and late in pointing movement.
• Measure reaction time to perturbations.
infrared markerson finger
tabletop aligned to virtual targets
monitor
mirror
(c)
(b)
target
unseenhand
virtual fingertip
(a)
Reaction time predictions
End-phasefeedback
Continuousfeedback
Lateperturbation
Earlyperturbation
X(cm)
0 5 10 15 20 25 30
2
0
-2
X(cm)
0 5 10 15 20 25 30
2
0
-2
X(cm)
0 5 10 15 20 25 30
2
0
-2X(cm)
0 5 10 15 20 25 30
2
0
-2
X(cm)
0 5 10 15 20 25 30
2
0
-2X(cm)
0 5 10 15 20 25 30
2
0
-2
Fast reaches (~450ms)
Fast reaches (~450ms) Slow reaches (~600ms)
Slow reaches (~600ms)
Early perturbation
Mid-reach perturbation
Early perturbation
Mid-reach perturbation
Early perturbation
Mid-reach perturbation
Early perturbation
Mid-reach perturbation
X(cm)
0 5 10 15 20 25 30
2
0
-2
X(cm)
0 5 10 15 20 25 30
2
0
-2
-5 0 5 10 15 20 25 30-8
-6
-4
-2
0
2
4
6
X position (cm)
Y position (cm)
Sample finger paths
Autoregressive model
• Baseline (unperturbed) trajectories
• Perturbed trials
y(t) =w8y(t−8)+w7y(t−7)+L +w1y(t−1)
y(t) =w8y(t−8)+w7y(t−7)+L +w1y(t−1)+wP (t)ΔY
0
0 100 200 300 400 500time after perturbation (ms)
0
0 100 200 300 400 500time after perturbation (ms)
0 100 200 300 400 500time after perturbation (ms)
0
0 100 200 300 400 500time after perturbation (ms)
0
0 100 200 300 400 500time after perturbation (ms)
0
0 100 200 300 400 500time after perturbation (ms)
Fast reaches (~450ms)
Fast reaches (~450ms) Slow reaches (~600ms)
Slow reaches (~600ms)
Early perturbation
Mid-reach perturbation
Early perturbation
Mid-reach perturbation
Early perturbation
Mid-reach perturbation
Early perturbation
Mid-reach perturbation
0
0 100 200 300 400 500time after perturbation (ms)
0
0 100 200 300 400 500time after perturbation (ms)
0
Early Mid-reach0
20
40
60
80
100
120
140
160
180
200
Fast reaches
Early Mid-reach
Slow reaches
N = 6 N = 6
0 100 200 300 400 500 600 700 800 900 10000
5
10
15
20
25
30
Time (ms)
X position (cm)
Subject 1: Trajectories for early perturbed trials
Positive perturbations
Negative perturbations
0 100 200 300 400 500 600 700 800 900 100020
21
22
23
24
25
26
27
28
29
30
Time (ms)
X position (cm)
Subject 1: Trajectories for early perturbed trials
Positive perturbations
Negative perturbations
0 100 200 300 400 500 600 700 800 900 10000
5
10
15
20
25
30
Time (ms)
X position (cm)
Subject 1: Trajectories for late perturbed trials
Positive perturbations
Negative perturbations
0 100 200 300 400 500 600 700 800 900 100020
21
22
23
24
25
26
27
28
29
30
Time (ms)
X position (cm)
Subject 1: Trajectories for late perturbed trials
Positive perturbations
Negative perturbations
0 100 200 300 400 500 600 700 800 900 10000
5
10
15
20
25
30
Time (ms)
X position (cm)
Subject 2: Trajectories for early perturbed trials
Positive perturbations
Negative perturbations
0 100 200 300 400 500 600 700 800 900 10000
5
10
15
20
25
30
Time (ms)
X position (cm)
Subject 2: Trajectories for late perturbed trials
Positive perturbations
Negative perturbations
0 50 100 150 200 250 300 350 400 450 500-3
-2
-1
0
1
2
3x 10-3
Time (ms)
Perturbation weight
Perturbation weight function for in-line perturbations
0 50 100 150 200 250 300 350 400 450 500-4
-3.5
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1x 10-3
Time (ms)
Perturbation weight
Perturbation weight function for perpendicular perturbations
Conclusions
• Visuomotor system uses directional error signal for feedback control?
• Position / speed error in direction of movement is not effective feedback signal?
• Why?– Position along path blurred by motion– Insensitivity to acceleration along direction of
motion
Question
• What visual information about hand does visuomotor system use– Position error?– Motion error?– Position and motion?
(c)
(b)
target
unseenhand
virtual fingertip
(a)
(c)
(b)
target
(a)
0 50 100 150 200 250 300 350 400 450-12
-10
-8
-6
-4
-2
0
2
4x 10-4
Time (msecs.)
Perturbation weight
0 50 100 150 200 250 300 350 400 450-4
-3.5
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1x 10-3
Time (msecs.)
Perturbation weight
0 50 100 150 200 250 300 350 400 450-3.5
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5x 10-3
Time (msecs.)
Perturbation weight
0 50 100 150 200 250 300 350 400 450-2.5
-2
-1.5
-1
-0.5
0
0.5x 10-3
Time (msecs.)
Perturbation weight
0 50 100 150 200 250 300 350 400 450-4
-3.5
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1x 10-3
Time (msecs.)
Perturbation weight
0 50 100 150 200 250 300 350 400 450-3.5
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1x 10-3
Time (msecs.)
Perturbation weight
Conclusions
• Visuomotor system uses continuous visual feedback to control reaching movements.
• Feedback signals include positional error.
• Feedback signals include motion error.
• System is approximately linear.
0 50 100 150 200 250 300 350 400 450-6
-4
-2
0
2
4
6
8x 10-3
Time (msec.)
Perturbation weight
0 50 100 150 200 250 300 350 400 450-0.06
-0.05
-0.04
-0.03
-0.02
-0.01
0
0.01
0.02
Time (msec.)
Cumulative perturbation weight
0 50 100 150 200 250 300 350 400 450-16
-14
-12
-10
-8
-6
-4
-2
0
2
4x 10-3
Time (msec.)
Perturbation weight
0 50 100 150 200 250 300 350 400 450-0.35
-0.3
-0.25
-0.2
-0.15
-0.1
-0.05
0
0.05
Time (msec.)
Cumulative perturbation weight
0 50 100 150 200 250 300 350 400 450-12
-10
-8
-6
-4
-2
0
2
4
6x 10-3
Time (msec.)
Perturbation weight
0 50 100 150 200 250 300 350 400 450-0.2
-0.15
-0.1
-0.05
0
0.05
Time (msec.)
Cumulative perturbation weight
0 50 100 150 200 250 300 350 400 450-12
-10
-8
-6
-4
-2
0
2
4
6x 10-3
Time (msec.)
Perturbation weight
0 50 100 150 200 250 300 350 400 450-0.25
-0.2
-0.15
-0.1
-0.05
0
0.05
0.1
Time (msec.)
Cumulative perturbation weight
0 50 100 150 200 250 300 350 400 450-10
-8
-6
-4
-2
0
2
4
6
8x 10-3
Time (msec.)
Perturbation weight
0 50 100 150 200 250 300 350 400 450-0.02
-0.015
-0.01
-0.005
0
0.005
0.01
Time (msec.)
Perturbation weight