moving through the brain: a study of movement …shenoy/theses/kalmar.pdf · each has a special...
TRANSCRIPT
MOVING THROUGH THE BRAIN: A STUDY OF MOVEMENT
PREPARATION IN THE OCULOMOTOR AND REACH SYSTEMS
A DISSERTATION
SUBMITTED TO THE NEUROSCIENCES PROGRAM
AND THE COMMITTEE ON GRADUATE STUDIES
OF STANFORD UNIVERSITY
IN PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
Rachel Stern Kalmar
August 2010
http://creativecommons.org/licenses/by-nc/3.0/us/
This dissertation is online at: http://purl.stanford.edu/sk937qn7081
© 2010 by Rachel Stern Kalmar. All Rights Reserved.
Re-distributed by Stanford University under license with the author.
This work is licensed under a Creative Commons Attribution-Noncommercial 3.0 United States License.
ii
I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.
William Newsome, Primary Adviser
I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.
Krishna Shenoy, Primary Adviser
I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.
Tirin Moore
I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.
Brian Wandell
Approved for the Stanford University Committee on Graduate Studies.
Patricia J. Gumport, Vice Provost Graduate Education
This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file inUniversity Archives.
iii
Abstract
Movement preparation allows the rapid and accurate execution of voluntary move-
ments, and can be influenced by factors that may change from moment to moment,
such as attention and differences in stimulus properties. Consequently, movement
preparation unfolds differently across many repetitions of the same movement. Av-
eraging neural responses across many repetitions is necessary to interpret single-cell
recordings, but diminishes our ability to characterize the dynamics of the underlying
process. A central question in neuroscience, and also of fundamental clinical impor-
tance, is to understand how these plans develop in the brain. Several research groups
are starting to build prosthetic devices that are controlled directly by neural activity
in motor areas of the brain (Nicolelis, 2001; Donoghue, 2002; Musallam et al., 2004;
Schwartz, 2004; Santhanam et al., 2006; Hochberg et al., 2006; Mulliken et al., 2008;
Andersen et al., 2010), but the extent to which these can be developed may hinge crit-
ically upon our understanding of the neural basis of motor preparation. Simultaneous
recording from populations of neurons allows dynamics of movement preparation to
be estimated on single trials. Our goal is to characterize these dynamics, to gain
insight into the process underlying movement preparation.
Here, we recorded peri-saccadic activity from ensembles of neurons in an oculo-
motor area, prearcuate cortex, in two monkeys. While monkeys performed visually-
guided eye movements, we measured firing rates of a population of neurons using a
96-electrode array. We studied (1) the responses from a heterogeneous population
of prearcuate cortex neurons involved in decision-making and movement preparation,
(2) the relationship between saccade initiation times and responses from the neural
population, and (3) how these responses compared to those recorded in PMd, a cor-
tical area involved in arm movements. The array data from prearcuate allowed us to
compare responses from individual neurons with previous findings, but also allowed
iv
us to analyze the population dynamics of movement planning, by using techniques
applied to the reach system.
We found that ensemble responses from diverse populations of prearcuate neurons
(1) can be visualized as trajectories moving through a low-dimensional neural state
space, (2) reflect visual, decision-, and movement-related aspects of the task, and
(3) correlate with a monkey’s reaction time on a trial-by-trial basis. Further, the
single-trial relationship between ensemble activity in prearcuate cortex and saccadic
reaction times was qualitatively and quantitatively very similar to the relationship
between ensemble activity in PMd and corresponding reach reaction times. This
framework for analyzing neural population activity and dynamics should permit new
extensions of single-neuron-level models, and may offer further insight into general
mechanisms of movement preparation across motor systems.
v
Acknowledgements
Having done neuroscience research as an undergraduate, I had some idea of what
graduate research would be like. However, what I learned during my six years at
Stanford was more multidimensional than I could have guessed, and it far surpassed
my expectations. I credit this to the amazing people who have been my mentors,
colleagues, classmates and friends during my graduate career.
First and foremost, I want to thank my advisors, Bill Newsome and Krishna
Shenoy. I appreciate their guidance and support, but also that they allowed me the
space to figure things out on my own, to follow my intellectual curiosity, and pushed
me to learn how to make and stick to a timeline. Not only were Krishna and Bill
excellent to work with individually, but working with both together was even better.
Each has a special talent for communicating complex concepts clearly, boundless
enthusiasm and courage for trying new things, and the critical insight needed to find
flaws in experimental design or analyses. I benefited greatly from our joint weekly
meetings. I thank my advisors for their collaborative mentorship, for their patience,
and for their friendship. My thesis committee members, Tirin Moore and Brian
Wandell, have also been valuable mentors, providing constructive feedback about my
research plan in its many, many iterations. I left each of my committee meetings
feeling energized and excited about science.
The Shenoy and Newsome labs are stimulating and encouraging environments,
and I feel privileged to have been a part of both. In particular, I owe thanks to
John Reppas in the Newsome Lab. John taught me how to do awake, behaving
monkey experiments, mentored me through my rotation and throughout my time in
lab, generously shared his array data with me, gave critical feedback when needed,
and always had time to chat about nitty gritty or big-picture questions. In the
Shenoy Lab, discussions with Mark Churchland about monkey training and the motor
vi
control literature, and conversations with Byron Yu, John Cunningham, Afsheen
Afshar and Zuley Rivera about dimensionality reduction techniques and data analyses
were incredibly useful. I am also grateful for the expert care and indispensable training
assistance of the technicians in both labs: Jessica Powell, Stacy Rosenbaum, Jamie
Sanders (Newsome Lab) and Mackenzie Risch (Shenoy Lab). I am indebted to Tex
and Vito for the data in this thesis, and Ham, Larry and Isaac not only for providing
data but also for teaching me how to work with non-human primates. The lessons I
learned about behavior have proved invaluable in working with primates, macaques
and humans alike.
The ‘Neurosci-X’ community in the Clark Center and the greater Stanford neuro-
science community have provided rich intellectual—and fun—environments in which
to learn and grow. During my time here, Bill Newsome and John Huguenard had the
leadership and vision to steer the Neurosciences Program to where it is today, and
Rebecca Wyse and Ross Colvin worked tirelessly to keep all the gears well-oiled and
running. In addition to the neuroscience community, I would like to acknowledge the
Bio-X community as well. I am thankful for the Bio-X Graduate Fellowship I was
awarded, and for the ample opportunities in the vibrant Bio-X community to learn
about research outside my field as well as to present my own work. Jill Sakata and
Heideh Fattey have been especially helpful coordinators of the fellowship program
and the Bio-X endeavor overall. I feel honored to have been a member of the Bio-X
and neuroscience communities at Stanford, and to have been surrounded by so many
interesting, well-rounded, happy and motivated colleagues.
Outside of my thesis work, I have been extremely fortunate to have had the
opportunity to follow other passions at Stanford. Taking classes through the Hasso
Plattner Institute of Design (better known as the d.school) was my first introduction
to “design thinking”, with which I instantly fell in love. Thanks to everyone I’ve
worked with at the d.school, and particularly to Jim Patell and Dave Beach for having
the vision to see how a neuroscientist could fit into the Entrepreneurial Design For
Extreme Affordability class. It was a life-changing experience. Thanks also to Dan
Klein, for putting improvisation in my creative toolbox. Since my first d.school class,
I have integrated design thinking into every aspect of my work and my life.
I also feel privileged to have had access to wonderful support resources at Stanford.
Without the Hume Writing Center, the Center for Career Development (especially
vii
Chris Pohalski), and programs supported by the Vice Provost for Graduate Edu-
cation like WISE (Women in Science and Engineering), my graduate career would
undoubtedly have been far less pleasant.
Several other people have played pivotal roles in my career. When I was an
impressionable college freshman, Ken Reisman and Matt Simon took me under their
wings, introducing me to scientific research and setting me on the path that ultimately
brought me to graduate school. I will be forever grateful to E.J. Chichilnisky, for
taking a chance and letting a completely inexperienced and naive undergraduate join
his lab. Working with E.J. completely changed the trajectory of my career. Over the
nearly 5 years I spent in his lab, E.J. was a remarkably supportive mentor, teacher
and friend.
I would also like to thank my friends and family for their tireless support and
encouragement. Whether collaborating on projects, traveling together to destinations
near and far, or sitting through yet another round of practice talks, it has been great
to have such an extraordinary network of people I can rely on. Of my friends, I owe
special thanks to Jim Sowers. Jim pushed me to achieve my goals in school and out
and also shared some of my favorite moments during my time at Stanford, including
unicycling across Nova Scotia! Finally, I would like to acknowledge the unconditional
love of my immediate and extended family, and in particular of my parents, my own
special forces team. I am very lucky to have such a caring, unique and interesting
family. It has been a joy and a privilege to share my graduate school journey with so
many wonderful people.
viii
Contents
Abstract iv
Acknowledgements vi
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 The study of movement preparation . . . . . . . . . . . . . . . . . . . 2
1.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Neural basis of movement preparation . . . . . . . . . . . . . . . . . . 4
1.3.1 Early work about movement centers in the brain . . . . . . . . 4
1.3.2 Evidence for movement preparation . . . . . . . . . . . . . . . 5
1.3.3 Movement centers of the frontal lobe . . . . . . . . . . . . . . 8
1.4 Somatic movement in the frontal lobe . . . . . . . . . . . . . . . . . . 9
1.5 Eye movement centers in the frontal lobe . . . . . . . . . . . . . . . . 10
1.5.1 Arcuate concavity . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.5.2 FEF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.5.3 8Ar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.6 Linking movement preparation to behavior . . . . . . . . . . . . . . . 23
1.6.1 Models and paradigms for studying movement preparation in
the oculomotor system . . . . . . . . . . . . . . . . . . . . . . 23
1.6.2 Extending single-neuron models to a population . . . . . . . . 25
1.7 Dynamics of movement preparation . . . . . . . . . . . . . . . . . . . 26
1.7.1 Motivation to study dynamics . . . . . . . . . . . . . . . . . . 26
1.7.2 Open questions: using state-space approaches in the oculomo-
tor system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
1.8 Questions addressed here . . . . . . . . . . . . . . . . . . . . . . . . . 29
ix
2 Response properties of prearcuate cortex 30
2.1 Background & Motivation . . . . . . . . . . . . . . . . . . . . . . . . 30
2.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.2.1 Subjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.2.2 Task design and training . . . . . . . . . . . . . . . . . . . . . 32
2.2.3 Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.2.4 Array locations and neural recordings . . . . . . . . . . . . . . 35
2.2.5 Dimensionality reduction of neural data . . . . . . . . . . . . 39
2.2.6 Computing the ROC predictive index . . . . . . . . . . . . . . 41
2.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.3.1 Tuning properties of neurons / cell classification . . . . . . . . 41
2.3.2 Diversity of PSTHs . . . . . . . . . . . . . . . . . . . . . . . . 45
2.3.3 Visualizing population activity . . . . . . . . . . . . . . . . . . 45
2.3.4 Decision-related activity . . . . . . . . . . . . . . . . . . . . . 46
2.3.4.1 Visualizing the effect of coherence on population activity 49
2.3.4.2 Dimensionality of data . . . . . . . . . . . . . . . . . 52
2.3.5 Effects of choice and reward history: this trial . . . . . . . . . 54
2.3.6 Effects of choice and reward history: subsequent trials . . . . . 56
2.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.4.1 Bias in population measures of choice predictivity . . . . . . . 59
2.4.2 Dimensionality vs. epoch . . . . . . . . . . . . . . . . . . . . . 61
2.4.3 History effects: rethinking the idea of ‘baseline’ . . . . . . . . 62
2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3 Population activity in prearcuate cortex accounts for variance in
saccadic latencies 65
3.1 Chapter Intro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.2 Background & Motivation . . . . . . . . . . . . . . . . . . . . . . . . 66
3.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.3.1 Subjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.3.2 Task design and training . . . . . . . . . . . . . . . . . . . . . 68
3.3.3 Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.3.4 Array locations and neural recordings . . . . . . . . . . . . . . 69
3.3.5 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
x
3.3.6 Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.3.7 Dimensionality reduction of neural data . . . . . . . . . . . . 70
3.3.8 Analysis and presentation . . . . . . . . . . . . . . . . . . . . 70
3.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.4.1 Diverse neural responses in prearcuate . . . . . . . . . . . . . 71
3.4.2 Neural trajectories evolve along a stereotyped path . . . . . . 72
3.4.3 Trial-by-trial relationship between neural dynamics and RT . . 74
3.4.4 RT correlation varies with a number of factors . . . . . . . . . 79
3.4.5 Does simultaneity matter? Comparing results for single cells
and populations . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.4.6 Dimensionality of neural data . . . . . . . . . . . . . . . . . . 83
3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
3.5.1 Saccadic preparation in prearcuate . . . . . . . . . . . . . . . 89
3.5.2 Comparison between dynamics in prearcuate and PMd . . . . 90
3.5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4 PMd experiments 93
4.1 The speed limits of planning . . . . . . . . . . . . . . . . . . . . . . . 93
4.1.1 The prosthetic cursor task . . . . . . . . . . . . . . . . . . . . 93
4.1.1.1 Tuning curves are modulated by chain position . . . 96
4.1.1.2 Inter-monkey differences . . . . . . . . . . . . . . . . 98
4.1.1.3 Gain field analogy . . . . . . . . . . . . . . . . . . . 100
4.1.1.4 State-space model for investigating cortical differences 101
4.1.2 Future directions . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.2 Planning with uncertainty . . . . . . . . . . . . . . . . . . . . . . . . 103
4.3 Neural dynamics and muscle activity . . . . . . . . . . . . . . . . . . 111
4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
5 Conclusion 116
5.1 Summary & Contributions . . . . . . . . . . . . . . . . . . . . . . . . 116
5.2 Array recording . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.2.1 Pros and cons of array recording . . . . . . . . . . . . . . . . . 119
5.2.1.1 Advantages . . . . . . . . . . . . . . . . . . . . . . . 119
5.2.1.2 Challenges of array placement . . . . . . . . . . . . . 120
xi
5.2.1.3 Other challenges of array recording . . . . . . . . . . 122
5.2.2 What can and cannot be addressed by multielectrode recording
and state-space approaches? . . . . . . . . . . . . . . . . . . . 124
5.3 Open questions & Future directions . . . . . . . . . . . . . . . . . . . 125
5.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
xii
List of Tables
5.1 PMd arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
5.2 Prearcuate arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
xiii
List of Figures
1.1 Mean RT vs. delay period duration (reaches) . . . . . . . . . . . . . . 3
1.2 Mean RT vs. delay period duration (saccades) . . . . . . . . . . . . . 4
1.3 Motoric areas of the brain . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 Organization of frontal cortex . . . . . . . . . . . . . . . . . . . . . . 9
1.5 The arcuate concavity contains FEF and 8Ar . . . . . . . . . . . . . 12
1.6 Prearcuate recordings . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.7 Eye movements evoked by microstimulation . . . . . . . . . . . . . . 20
1.8 FEF connectivity schematic . . . . . . . . . . . . . . . . . . . . . . . 21
1.9 Oculomotor connectivity schematic . . . . . . . . . . . . . . . . . . . 22
1.10 The rise-to-threshold hypothesis . . . . . . . . . . . . . . . . . . . . . 26
1.11 Optimal subspace hypothesis . . . . . . . . . . . . . . . . . . . . . . . 28
2.1 Behavioral tasks: delayed saccade and direction discrimination . . . . 33
2.2 Array location in prearcuate and PMd . . . . . . . . . . . . . . . . . 36
2.3 Array placement, with respect to arcuate and principal sulci . . . . . 36
2.4 Diversity of waveform signal-to-noise ratios . . . . . . . . . . . . . . . 38
2.5 Diversity of inter-spike interval statistics . . . . . . . . . . . . . . . . 39
2.6 Polar tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.7 Tuning during the delayed saccade task . . . . . . . . . . . . . . . . . 44
2.8 Average activity of population: low-dimensional representation . . . . 46
2.9 Coherence effects: individual unit PSTHs . . . . . . . . . . . . . . . . 47
2.10 Coherence effects: individual unit ROC . . . . . . . . . . . . . . . . . 48
2.11 ROC predictive index comparison between prearcuate and LIP . . . . 50
2.12 Low-dimensional representation of population activity during dots task. 51
2.13 Measuring distance between mean trajectories as a function of coher-
ence and time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
xiv
2.14 Relative proportion of shared variance explained varies across epochs 53
2.15 Behavioral task: direction discrimination, hold period . . . . . . . . . 54
2.16 Hold period effects represent previous action or previous choice . . . . 55
2.17 Low-dimensional representation of population activity during hold period 56
2.18 History effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.1 Behavioral task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.2 PSTHs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.3 Neural trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.4 Schematic of projection along mean trajectory . . . . . . . . . . . . . 75
3.5 Reaction time correlation . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.6 Correlation coefficients vs. reference offset . . . . . . . . . . . . . . . 78
3.7 Correlation coefficients are influenced by many factors . . . . . . . . . 81
3.8 Measuring curvature . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
3.9 Does simultaneity matter? One example dataset . . . . . . . . . . . . 82
3.10 Does simultaneity matter? Looking across datasets . . . . . . . . . . 83
3.11 Time-courses of GPFA projections . . . . . . . . . . . . . . . . . . . . 85
3.12 Relative proportions of variance explained . . . . . . . . . . . . . . . 86
3.13 Hypotheses relating neural preparatory activity to reaction time . . . 87
3.14 Comparison of example prearcuate and FEF neurons . . . . . . . . . 89
4.1 The chain task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
4.2 Tuning does not depend on position in chain . . . . . . . . . . . . . . 96
4.3 Chain position modulates firing rate . . . . . . . . . . . . . . . . . . . 97
4.4 Gain histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
4.5 Differences in placement of electrode arrays . . . . . . . . . . . . . . . 99
4.6 Trial structure and reward schedule . . . . . . . . . . . . . . . . . . . 100
4.7 Gain analogy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
4.8 Gain in state space . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
4.9 Schematic of a proposed task for investigating neural dynamics of rapid
plan-switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.10 Delayed reach and distractor tasks . . . . . . . . . . . . . . . . . . . 105
4.11 Trajectories during the delay period, with and without a distractor
present . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
xv
4.12 State-space hypotheses for representing plans with uncertainty . . . . 107
4.13 Sample trajectories from the ambiguous period of the distractor task 108
4.14 Shuffling test and occupancy rate . . . . . . . . . . . . . . . . . . . . 109
4.15 Bias represented in trajectories . . . . . . . . . . . . . . . . . . . . . 110
4.16 Low-dimensional neural state-space related to muscle activity . . . . 112
4.17 Illustration of neural curvature axis (NCA) computation . . . . . . . 113
4.18 NCA projections predict EMG features . . . . . . . . . . . . . . . . . 114
5.1 Comparison of waveforms found using different spike-sorting techniques 124
xvi
Chapter 1
Introduction
1.1 Motivation
Fast, accurate movements are critically important to survival. An animal waiting
to catch its prey may only have one chance—a poorly executed pounce could mean
no dinner, or worse. Neural circuits in the brain must evaluate and choose between
multiple motor plans, using incoming sensory information to decide where and when
to move. The brain’s capacity to integrate sensory information with prior information
and statistics about the world allows us to prepare movements in advance. This
planning is critical for executing fast and accurate movements. A central question
in neuroscience, and also of fundamental clinical importance, is to understand how
these plans unfold in the brain. This is an active area of research both in the context
of eye and arm movements. As an example of the potential translational importance
of understanding planning in the arm system, several research groups are starting to
build prosthetic devices that are controlled directly by neural activity in motor areas
of the brain (Nicolelis, 2001; Donoghue, 2002; Musallam et al., 2004; Schwartz, 2004;
Santhanam et al., 2006; Hochberg et al., 2006; Mulliken et al., 2008; Andersen et
al., 2010). However, the extent to which these can be developed may hinge critically
upon our understanding of the neural basis of motor preparation. This thesis aims
to further our understanding of movement preparation on timescales relevant for
behavior by employing array recordings in prearcuate cortex, an oculomotor area.
1
2
1.2 The study of movement preparation
In early studies of motor control, researchers catalogued classes of standard move-
ments. Comparing the amount of time needed for movements in each class allowed
evaluation of different models of movement processing (Keele, 1968). From these
studies, researchers observed that subjects’ speed and accuracy depends on many
variables, including direction and extent of an instructed movement, and how far in
advance of a go cue the target was specified (Keele, 1968). Subsequent behavioral
studies provide strong evidence that voluntary movements are prepared before they
are executed (Rosenbaum, 1980; Riehle and Requin, 1989; Crammond and Kalaska,
2000; Churchland et al., 2006). The process of movement preparation takes time,
and without adequate time to plan (∼150-250 ms), subjects take longer to initiate
the movement once instructed to do so. With adequate time to plan, subjects can
initiate their movements 30-40 ms faster. One way to determine how long the motor
system needs to prepare an instructed movement is to measure the latency between
when subjects are given a ‘go cue’ and when they start moving. This latency before
movement initiation is what we will refer to as a subject’s reaction time (RT).
Instructed delay vs. RT tasks
Studies of movement preparation come in two general categories: reaction time tasks
and instructed delay tasks. In the first category of tasks, information about the target
location is presented simultaneously with the go cue. In these reaction time tasks,
subjects are allowed to move to the target as soon as they are ready, thus subjects’
reaction times include the time needed to plan a movement, in addition to how long
it takes to trigger the movement after it is fully planned. By contrast, instructed
delay tasks allow separation of the processes of movement preparation and movement
initiation. In these tasks, information about an upcoming movement is separated
from the go cue by a delay. Thus, subjects are able to use the delay period to plan
movements before they are allowed to move. In this paradigm, reaction times measure
to what extent the process of movement preparation was complete before the go cue,
giving us insight into how long it takes for the motor system to translate a plan into
a movement.
Several studies show that the length of the delay period between the instruction
3
cue and the go cue influences subjects’ reaction times. This effect is consistent across
variations in tasks, subjects, and target locations (Pare and Munoz, 1996). Figure 1.1
illustrates this trend for three monkeys performing a reaching task in our lab: the
shorter the delay period preceding the movement cue, the longer was the animals RT
(Churchland et al., 2006). Figure 1.2 shows the same relationship between saccadic
reaction times and delay period duration in a delayed saccade task (data from Mark
Churchland, unpublished). This effect has also been described in the saccade system
by Schall and Hanes (1993).
Subjects’ reaction times are faster not only when given more time to plan a move-
ment, but also when given more information about the movement to perform. Ambi-
guity or incomplete information about the movement instruction increased subjects’
reaction times (Posner et al., 1980; Rosenbaum, 1980). Rosenbaum (1980) presented
some, none, or all of the information about the required movement in advance of
the go cue, and measured the related changes in reaction time. Providing subjects
with information about the direction and extent of a movement in advance of a go
cue allowed them to initiate movements more rapidly. This suggested that reaction
time reflects the time needed to specify parameters of the upcoming movement that
were not known in advance. Churchland et al. (2006) use this evidence to support
their hypothesis that there is some time-consuming preparatory process that starts
happening during the delay.
30 430
220
260
monkey H
delay period (ms)
RT
(m
s)
Figure 1.1: Mean reach RT vs. delay period duration. Data from monkeys A and Bare from Churchland et al., 2006, Fig. 4. Reprinted with permission.
1.2.1 Overview
Building on this behavioral evidence for movement preparation, we will give an
overview of the motor systems of the frontal lobe, focusing on the role of specific areas
4
220
260
RT
(m
s)
0 300delay period (ms)
Figure 1.2: Mean saccadic RT vs. delay period duration. Data from Mark Churchland(unpublished).
in preparing and initiating movements. We will review the somatic motor system,
focusing on the role of premotor cortex in movement preparation. Drawing parallels
between the somatic and oculomotor systems, we will then describe the premotor
areas of the frontal lobe involved in saccade planning and execution.
A number of models have been proposed to explain variability in RT as it relates
to movement preparation. In the second half of this chapter, we will discuss some of
these models with respect to the neural mechanisms of movement preparation.
1.3 Neural basis of movement preparation
1.3.1 Early work about movement centers in the brain
The neural basis of how movements are prepared and executed has captured the
interest of scientists for well over a century. In 1870, Gustav Fritsch and Eduard
Hitzig pioneered the field of modern electrophysiology with their finding that electrical
stimulation of the motor cortex evoked contralateral movements (Taylor and Gross,
2003). Four years later David Ferrier built upon this work, more finely mapping
out the movement centers of the brain. Injecting current into different regions, he
catalogued the eye, limb, and body movements that were evoked (Ferrier, 1874).
The next century of research on the motor system built on these foundations,
using electrical stimulation and lesion studies to refine Ferrier’s maps of cortical and
subcortical motor areas. However, the first experiments that connected recordings
of individual neurons in these regions with voluntary movement of monkeys’ eyes or
limbs did not take place until nearly 100 years later.
5
Using the same microelectrode recording techniques that were being used to study
cells’ responses in anesthetized animals, Edward Evarts made the first connection be-
tween single-unit physiology and behavior in awake animals. Evarts measured the
activity of individual pyramidal tract neurons in precentral motor cortex while a
monkey made hand movements (Evarts, 1966). Evarts’ series of experiments demon-
strated that activity of pyramidal tract neurons preceded hand movements (Evarts,
1966) and was also related to the force and displacement of a load the monkey con-
trolled with a handle (Evarts, 1968). Around the same time, Emilio Bizzi (1967)
identified oculomotor frontal lobe neurons using antidromic stimulation, and mea-
sured the timing of these neurons’ responses with respect to different types of eye
movements.
Since these early electrophysiology studies, much more has been learned about
the areas of the brain involved in preparing and executing movements. Studies of
cytoarchitecture, connectivity, electrophysiological recordings, electrical stimulation,
and changes in behavior following lesions have helped identify which cortical and sub-
cortical areas contribute to which aspects of behavior (see comprehensive reviews by
Wise, 1985; Schall, 1997). In particular, electrophysiological recordings have allowed
researchers to probe, on fine spatial scales, how individual neurons respond to aspects
of stimuli and upcoming movements. Recording from different collections of individual
cells across the brain, a number of areas involved in eye movements and arm move-
ments have been discovered and characterized (Figure 1.3). Figure 1.3a illustrates
regions of the brain involved in arm movements. Figure 1.3b shows a schematic of
the eye movement areas, and their connections to other visual and oculomotor areas.
Thus, we have a wealth of empirical knowledge about the major movement centers of
the brain.
1.3.2 Evidence for movement preparation
Since much research about movement preparation has been done in the context of
reaching tasks (reviewed by Wise, 1985; Kalaska, 1991; Kalaska et al., 1997), we will
first focus on areas of the frontal lobe involved in arm movements. Then we will
move to a system where the effector dynamics are simpler and discuss the oculomotor
regions of the frontal lobe.
As described in Section 1.2, psychophysical evidence suggests that movements are
6
a
b
Figure 1.3: (a) Regions of the brain involved in arm movements (modified fromAchtman et al., 2007, with permission). AIP, anterior interparietal cortex; cPMd,caudal dorsal premotor cortex; cPMv, caudal ventral premotor cortex; FEF, frontaleye fields; LIP, lateral interparietal cortex; preSMA, pre-supplementary motor area;M1, primary motor cortex; MIP, medial interparietal cortex; rPMd, rostral dorsalpremotor cortex; rPMv, rostral ventral premotor cortex; S1, primary somatosensorycortex; SMA, supplementary motor area; VIP, ventral interparietal cortex. (b) Cor-tical and subcortical oculomotor areas (from Fecteau and Munoz, 2003, adapted bypermission from Macmillan Publishers Ltd: Nature). CN, caudate nucleus; DLPFC,dorsolateral prefrontal cortex; LIP, lateral intraparietal cortex; MRF, medullary retic-ular formation; PPRF, paramedian pontine reticular formation; SEF, supplementaryeye fields; SNp, substantia nigra pars reticulata
prepared before they can be executed. This suggests that movement preparation is a
process unfolding in the brain, that is necessary for the rapid and accurate execution
7
of movements. Where and how does this process occur in the brain? A number of
motor and premotor areas show signals correlated with an upcoming action or motor
plan. Using instructed delay tasks in which the instruction is separated from the go
cue by a variable delay period, collecting neural activity during this delay allows the
study of the neural basis of movement preparation.
One of the first pieces of physiological evidence supporting movement preparation
for specific movements came from Kutas and Donchin (1974). In this study, Kutas
and Donchin instructed subjects with which hand to perform their task, and found
that scalp potentials were greater over the hemisphere contralateral to the respond-
ing hand. The potential was associated with a specific movement, supporting the
hypothesis that movement preparation is specific for an upcoming action. Neural sig-
nals correlated with an upcoming action or motor plan have since been found in many
of the areas shown in Figure 1.3 (e.g., Snyder et al., 2006). This delay-period activity
reflecting motor preparation is common to frontal, premotor, and motor cortex, su-
perior colliculus, and basal ganglia (discussed in Snyder et al., 2006 and Churchland
et al., 2006).
Neural signals related to movement preparation are tuned for parameters of a
specific upcoming action (Tanji and Evarts, 1976; Stoet and Snyder, 2003; Church-
land et al., 2006; Snyder et al., 2006). In the supplementary motor area (SMA),
neural movement preparation signals reflect the dynamics, but not kinematics, of an
upcoming movement (Padoa-Schioppa et al., 2002). Movement preparation signals
can be predictive of accuracy and reaction time for arm movements (Lecas et al.,
1986; Riehle and Requin, 1993; Snyder et al., 2006) and signals related to movement
initiation also predict saccadic reaction time (Hanes and Schall, 1996; Dorris et al.,
1997; Everling and Munoz, 2000; Snyder et al., 2006).
Evidence for a causal role of delay period activity in movement initiation
in PMd
Behavioral and physiological studies provide strong evidence that voluntary move-
ments are prepared before they are executed, and that neural activity represents
advance preparation for specific actions. Churchland et al. (2006) show that, pre-
ceding movements, the variance of neural activity in dorsal premotor cortex (PMd)
decreases and stabilizes, across trials. This supports the hypothesis that the goal of
8
motor planning is to move neural activity into a prepared ‘state’, preceding a mon-
key’s reach in a given direction. In this hypothesis, the across-trial drop in variance
corresponds to the point in time when preparation for this upcoming movement is
complete. The similar timescales for movement preparation found from the time
taken for variance to decrease and the monkey’s RTs on a delayed reach task provide
additional support for this hypothesis. But does delay period activity play a causal
role in movement initiation? Churchland et al. tested this hypothesis by injecting
current into PMd during the delay period when the monkey was planning a reach.
When putative planning activity in these neurons was disrupted with microstimula-
tion, monkeys’ RTs were longer than would be predicted from Fig. 1.1 (Churchland
et al., 2006; Churchland and Shenoy, 2007). These two pieces of evidence, combined
with behavioral results, support the idea that movements are prepared before they
are executed, and that PMd is involved in this movement preparation.
1.3.3 Movement centers of the frontal lobe
To investigate more deeply the neural mechanisms underlying movement preparation,
we will focus on the somatotopic and oculomotor areas of the frontal lobe. The
confluence of visual, cognitive, and motoric signals make the motor and dorsolateral
regions of the frontal lobe a rich area in which to study movement preparation.
The primate frontal lobe can be separated into two general divisions: one motor
and one prefrontal. The motor division contains primary motor cortex (M1) and
several premotor areas, including premotor cortex and SMA (Preuss, 2007). The
prefrontal division is generally split into three parts: a medial region, an orbital
region, and a dorsolateral region. The medial region corresponds to anterior cingulate
cortex, and the orbital region contains orbital frontal cortex. The dorsolateral region,
which contains the area around the arcuate cortex, is involved in the highest levels of
cognitive function, and malfunctions in this area are tied to neuropsychiatric disorders
like schizophrenia (Preuss, 2007). This region is also involved in selective attention,
working memory, and programming behavior in novel tasks. The frontal eye field
(FEF), which is involved in controlling the direction of eye movements and attention,
and the principalis cortex, which is involved in spatial working memory are also
contained in this dorsolateral region (Preuss, 2007).
9
a
b
Figure 1.4: Organization of frontal cortex. (a) From Preuss and Goldman-Rakic(1991), with permission. (b) From Romanski (2004).
1.4 Somatic movement in the frontal lobe
The frontal lobe contains at least 6 premotor areas, each of which projects directly
to primary motor cortex (M1) and to the spinal cord, and each of which has some
degree of somatotopic organization (Dum and Strick, 2002). We will focus here on
premotor cortex, which contains neurons responsive to behaviorally-relevant stimuli
and are involved in movement preparation (Wise, 1985).
10
PMd
Dorsal premotor cortex (PMd) is involved in planning reaching movements (Weinrich
and Wise, 1982; Weinrich et al., 1984; Godschalk et al., 1985). Individual neurons in
PMd are active before movements are initiated, and are tuned for the direction (Geor-
gopoulos et al., 1982; Crammond and Kalaska, 1989), distance (Messier and Kalaska,
2000), and speed (Churchland et al., 2006) of an upcoming arm movement. Addi-
tionally, PMd projects directly to the spinal cord and primary motor cortex (M1),
receives input from a number of parietal areas, and is connected with frontal areas,
via primary, supplementary, and cingulate motor cortices (Wise et al., 1997). Impor-
tantly, neural activity in PMd is causally related to motor preparation (Churchland
and Shenoy, 2007), as well as lower-level aspects of motor control, such as distance
and speed. Being involved in both high level planning and lower-level motor con-
trol makes PMd an ideal place for studying how cognitive and behavioral signals are
integrated to form movement plans.
Many studies of population coding have been done in the somatotopic motor
system. However, it is not known to what extent the principles that govern movement
preparation in somatotopic areas will generalize to the oculomotor system. Eye and
arm movements differ in many ways (e.g., effector speed, weight, number of muscles
required), so one might expect oculomotor and reach systems to utilize different
strategies for motor preparation. By performing our experiments both systems, we
can compare our results in the reach system with populations of oculomotor neurons
with similar premotor properties (e.g., higher stimulation thresholds, fewer direct
connections to movement generators). This allows us to test whether the dynamics of
motor planning generalize to many movement preparation systems in the brain. The
next section will give an overview of the oculomotor signals found in the frontal lobe.
We place a particular emphasis on the motoric signals around the arcuate gyrus, the
area we targeted in our experiments.
1.5 Eye movement centers in the frontal lobe
The oculomotor region of the frontal lobe is involved in visual, motor, spatial and
cognitive functions. The general region around the arcuate concavity, 8A, is involved
in varying degrees of movement preparation (Fig. 1.5) (Bruce and Goldberg, 1985;
11
Seidemann et al., 2002). This area contains FEF and area 8Ar, as illustrated in
Figure 1.5. The functional and anatomical distinctions within the arcuate concavity
have evolved as technological advances have allowed finer mapping and enhanced
resolution of microstimulation, electrophysiological measurements and imaging. It is
clear that we are recording from the arcuate region with our electrode arrays. But
are our arrays in 8Ar or FEF? Understanding the historical and current literature
about this region is crucial for accurate interpretation and context of our results. We
first review what is known about the arcuate concavity and its general neighborhood,
then give an overview of the distinction between FEF and area 8Ar.
In this premotor region, FEF (also called 8Ac (Schall, 1995) or 8Ac and 8Am
(Preuss and Goldman-Rakic, 1991)) is the area most tightly connected to eye move-
ments and participates significantly in the network mediating gaze control. FEF is
located at the posterior aspect of the arcuate concavity and is involved in transform-
ing visual computations guided by cognitive processes into saccade motor commands
(reviewed by Schall, 1997). Just anterior, area 8Ar has some similar response prop-
erties to FEF. Microstimulation in 8Ar evokes eye movements, though with higher
stimulation thresholds than FEF (Bruce and Goldberg, 1985). Responses of neurons
in this area also reflect metrics of eye movements (Seidemann et al., 2002). How-
ever, anatomical and cytoarchitectural studies show that 8Ar is less connected to
direct motor output than FEF (Preuss and Goldman-Rakic, 1991; Schall, 1995; Ger-
bella et al., 2007). Like the distinction between premotor and primary motor cortex
(Wise, 1985), the boundary between 8Ar and FEF cannot be fully distinguished by
cytoarchitectural differences. Rather, FEF and 8Ar are distinguished by microstimu-
lation. Functional relationships between the premotor and motor functions may also
be analogous in these areas.
Anterior of area 8Ar, the cortex surrounding the principal sulcus (PS) plays a
role in spatial working memory (Fuster, 1973; Goldman-Rakic, 1987; Funahashi et
al., 1990). Dorsal to the PS, dorsolateral prefrontal cortex (DLPFC) is involved
in spatial aspects of working memory (Goldman-Rakic, 1987, 1988; Funahashi et
al., 1989; Petrides, 1994; Levy and Goldman-Rakic, 2000). Ventrolateral prefrontal
cortex, ventral to the PS, also plays a role in short-term memory but unlike DLPFC is
not spatially selective. The supplementary eye field (SEF) is another eye-movement
related region near the arcuate sulcus. The SEF is a second order motor area, and
12
Arcuate Concavity
8Ar
FEF
Figure 1.5: The arcuate concavity region contains FEF and 8Ar.
neurons are more responsive to stimuli in the context of behavior than when presented
with the stimulus alone. This area is less understood than FEF. Finally, posterior to
the arcuate sulcus, premotor cortex represents movement preparation or ‘set’, as well
as the visual aspects of the stimulus in the context of movement. Premotor cortex
is activated by sensory stimuli that guide movements, and contains both neurons
that respond to eye movements (more ventrally) and others that respond to arm
movements (more dorsally).
The area from the caudal terminus of the principal sulcus to the caudal bank of
the arcuate sulcus, known as the arcuate concavity (generally corresponding to area
8A, including both FEF/8Ac and 8Ar), is tightly linked to eye movement prepa-
ration. Microstimulating in this region, nearly all locations in principal sulcus and
all neurons in the arcuate sulcus elicit contralateral eye movements (Wagman et al.,
1961; Robinson and Fuchs, 1969; but see Boch and Goldberg, 1989). Additionally, a
high percentage of neurons in this area respond in relation to saccadic eye movements
(Wagman et al., 1961; Funahashi et al., 1991). Many respond before only purposive
saccades, with or without visual guidance (Wagman et al., 1961; Bruce and Goldberg,
1985; Funahashi et al., 1991).
13
14
Figure 1.6: 120 years of prearcuate studies. Overview of areas in frontal cortex relatedto eye movement generation (from Schall, 1997, with permission). (A) Original mapfrom Ferrier (1875). Eye movements were evoked from the shaded region. (B) Mapfrom Mott and Schaefer (1890). Zones from which different directions of eye move-ments were evoked are indicated. (C) Map from Levinsohn (1909). Eye movementswere evoked from the shaded region; the darker shading indicates the lower thresholdregion. (D) Map from Smith (1940). The regions from which the different types ofresponses were observed are indicated. (E) Map from Crosby et al. (1952). The dif-ferent directions of eye movements evoked are indicated. (F) Map from Wagman etal. (1961). Regions from which different directions of eye movements were evoked areindicated by different stipples and line widths. (G) Map from Robinson and Fuchs(1969). Zones are shown from which different amplitude saccades were evoked. (H)Map of visual response field eccentricity from Suzuki and Azuma (1983). (I) Map ofvisual response field size from Suzuki and Azuma (1983). (J) Map of frontal eye fieldfrom Bruce et al. (1985) and Gottlieb et al. (1994). The arcuate sulcus is representedas opened: the thick line is the lip, and the thin dashed line is the fundus. The low-threshold zone is indicated by the shaded region. (K) Location of supplementary eyefield from Schlag and Schlag-Rey (1987); Schall (1991); Parthasarathy et al. (1992).
Historically, the region considered the frontal eye field included the entire anterior
convexity of the arcuate, from the caudal end of the principalis sulcus to the arcu-
ate sulcus (Funahashi et al., 1991). In 1985, Bruce et al. systematically measured
the minimum current needed to evoke an eye movement. They found that, although
saccades could be evoked from this entire region, the region in which saccades were
elicited with small amounts of current (<50 µA) was restricted to the arcuate sulcus
and the adjacent arcuate lip. This smaller region characterized by microstimulation
thresholds is now agreed upon as ‘core’ FEF. The rostral aspect of the arcuate con-
cavity, on the cortical surface adjacent to the arcuate sulcus and bounded rostrally by
the principal sulcus was excluded from FEF, based on higher stimulation thresholds.
This part of the arcuate concavity with higher stimulation thresholds corresponds to
area 8Ar (Preuss and Goldman-Rakic, 1991) or 8r (Gerbella et al., 2009).
We will first review some of the older work that lays the groundwork for current
studies of FEF and 8Ar, which describes response properties from the arcuate con-
cavity (the pre-1985 definition of FEF, roughly area 8A), which includes both areas
(see Fig. 1.5). In the sections that follow, we will describe properties of “core” FEF
(post-1985 distinction) and 8Ar individually.
15
1.5.1 Arcuate concavity
Ferrier’s demonstration that electrical stimulation of the prearcuate gyrus causes con-
traversive eye movements has been reproduced in subsequent experiments in many
neocortical and subcortical sites, including the whole prefrontal convexity. Stimula-
tion in this area has also been shown to evoke movements of the ears, head and body
(Rosenkilde, 1979).
In more recent experiments, contralateral eye movements have been evoked by
stimulating in sites in nearly all of the principal sulcus and all arcuate sulcus (Robin-
son and Fuchs, 1969; Gottlieb et al., 1993). Robinson and Fuchs (1969) showed that
electrical stimulation in this area elicited fixed-vector saccadic eye movements that
were similar to natural, voluntary saccades. This suggests that these microstimula-
tion studies are activating the same circuitry for saccade generation that is used in
the context of voluntary behavior. Therefore, these microstimulation studies allow us
to learn about natural saccade maps.
The caudal prearcuate concavity is a source of polysynaptic projections to ex-
traocular muscles (Moschovakis et al., 2004; Gerbella et al., 2007). This area also has
a functional mapping onto superior colliculus. One piece of evidence for this comes
from anatomic studies of topographic connectivity (Schall, 1997). Also, simultane-
ous stimulation of the arcuate concavity and superior colliculus evokes saccades with
amplitudes and directions that are the vector sum of individual saccades at the site
of stimulation in either area, weighted by stimulation current (Schiller et al., 1979;
Schall, 1997).
Lesion studies have indicated that the arcuate concavity is sufficient but not nec-
essary for saccades (Schall and Thompson, 1999). In experiments where superior
colliculus is lesioned, eye movements can still be made (e.g. Schiller et al., 1980;
Collin et al., 1982). In other experiments, the effects of arcuate concavity lesions
were assessed. Bilateral lesions including the arcuate concavity caused slight visual
discrimination deficits when the whole sulcus was ablated (Rosenkilde, 1979). These
animals were not blind but showed decreased sensitivity to stimuli in the entire visual
field (Rosenkilde, 1979). Though the lesioned animals showed slight abnormalities
in fixation and needed more time to find a circular target among other geometrical
shapes, saccadic eye movements were not impaired (Rosenkilde, 1979). Importantly,
animals had time to recover after lesions, allowing time for alternate eye-movement
16
strategies to develop. This suggests a distributed or redundant mechanism of saccade
generation. This differs from the somatic motor system, where significant movement
deficits are seen with lesions of primary motor cortex.
Physiological recordings from the arcuate concavity have demonstrated that this
area has visual and motor-related signals. A large proportion of these units (∼45%)
have visual receptive fields (Wurtz and Mohler, 1976; Schall, 1997). Suzuki and
Azuma (1983) showed maps of visual receptive field (RF) eccentricity in this area
(Fig. 1.6I), in which RF size increased from lateral to medial, and caudal to rostral.
The activity of visual cells in the arcuate concavity is enhanced if the stimulus is the
target for a saccade (Goldberg and Bushnell, 1981).
The motoric signals in the arcuate concavity are diverse. Neurons in this area
respond in the context of saccades, smooth pursuit, fixation (Bizzi, 1968; Bizzi and
Schiller, 1970), vergence, divergence, and ocular accommodation (Gamlin and Yoon,
2000). Passing current into this area of cortex elicits saccades, ordered topographically
by saccade amplitude (Beevor, 1888). Fig. 1.6 (from Schall, 1997, Fig. 7) summarizes
over a century of research describing the visual and motoric properties of the arcuate
concavity.
After Bruce et al.’s redefinition of FEF as the region of the arcuate concavity
in which eye movements could be stimulated with less than 50 µA of current, most
studies of this area focused on this region of core FEF (primarily within the arcuate
sulcus), excluding the region on the arcuate gyrus, 8Ar.
In addition to differences in stimulating current needed to evoke eye movements,
several lines of evidence speak against 8Ar being part of FEF. First, saccades are
elicited with lower thresholds from sites on the rostral bank of the arcuate sulcus,
in the region corresponding to areas 8Ac, 45a and 45b; higher thresholds are needed
for sites in the anterior convexity in the region corresponding to 8Ar (Bruce et al.,
1985; Stanton et al., 1989; Schall et al., 1995b). Second, there is a higher density of
large layer V pyramidal cells that project to the superior colliculus and brainstem in
FEF than in 8Ar (Stanton et al., 1989). Compared to FEF, 8Ar also has a thicker,
more clearly defined granular layer/layer 4 (Gerbella et al., 2007) and lacks the thick
fascicles of 8Ac (which is included in FEF) (Schall, 1997), and has more conspicuous
bands of Baillarger and radial fibers (Schall, 1995). Finally, 8Ar differs from FEF in
its strength of connections with LIP (Schall, 1997).
17
Despite these findings, two recent studies indicate that the part of the arcuate
cortex directly involved in eye movements occupies a region larger than the classic,
electrical stimulation-defined, core frontal eye field. Using voltage-sensitive dye, Sei-
demann et al. (2002) demonstrated that the area of prearcuate cortex activated by eye
movements extended beyond the region of core FEF and onto the gyrus around area
8Ar. Similarly, Moschovakis et al. (2004) showed that the area in which metabolic
activity is increased around the time of an eye movement extends beyond the smaller
zone corresponding to FEF based on electrical stimulation. The region they describe
includes not only a large part of the anterior bank of the arcuate sulcus, but also
the caudal prearcuate concavity and part of premotor cortex in posterior bank of
the arcuate sulcus (Moschovakis et al., 2004; Seidemann et al., 2002). Are FEF and
8Ar both part of the same oculomotor area, or are they distinct? Barbas’s studies
of projections show that projections to FEF from 8Ar are relatively rare (Barbas
and Mesulam, 1981). This would be more consistent with FEF being the same area,
rather than a different one (Barbas and Mesulam, 1981). These studies, along with
pre-1985 FEF recordings, support a role for both FEF and 8Ar in processing eye
movements. However, to what extent these two areas overlap in their functional roles
is still an open topic of research.
1.5.2 FEF
Since Bruce et al. (1985), the region from which saccadic eye movements can be
evoked with currents of less than 50 µA has been used as the functional definition of
FEF (Tehovnik et al., 2000). Much smaller than the general arcuate concavity, this
area is confined to the anterior bank of the arcuate sulcus (Funahashi et al., 1991).
Earlier work showed that FEF generates signals sufficient but not necessary to
generate saccadic eye movements (reviewed by Schall and Thompson, 1999). However,
more recent studies have shown that reversible inactivation of the FEF impairs the
ability of monkeys to make saccades (Dias et al 1995, Sommer & Tehovnik 1997). This
is consistent with previous observations that initial saccade impairments following
FEF ablation recovered over time (Schall and Thompson, 1999).
Physiological studies show that, similarly to neurons in the broader arcuate con-
cavity, FEF neurons discharge in relation to saccadic eye movements and have visual
responses (Bruce and Goldberg, 1985; Schall et al., 1995a). Roughly half of the
18
neurons respond to visual stimuli, and FEF response fields (RFs) are large and gen-
erally emphasize the contralateral hemifield (Bruce and Goldberg, 1985; Schall et al.,
1995a). Visual responses are also modulated by the presence of stimuli outside their
receptive field and by animals’ readiness to make a saccade (Schall et al., 1995a).
For motoric FEF neurons, responses before saccade initiation reflected whether
or not the stimulus in the response field was the target for the eye movement. Schall
et al. (1995a) demonstrated significant variation in the level of activity in the 50
ms immediately preceding a saccade to the search target, in 96% of their sample of
neurons. This movement-related activity reflected the saccade produced, regardless
of the visual stimulus. This suggests that this activity represents the outcome of the
subjects’ decision process (Funahashi et al., 1991; Schall et al., 1995a).
Saccade amplitude is mapped topographically mapped in the FEF (Schall and
Thompson, 1999). Bruce et al. (1985) found that not only is saccade amplitude
orderly arranged, but also that saccade direction varied with depth of stimulation in
the arcuate sulcus (Tehovnik et al., 2000). Both for stimulation-evoked and natural
saccades, shorter saccades are represented ventrolaterally, and increasingly larger-
amplitude saccades represented dorsomedially (Bruce et al., 1985). Maps of visual
RF size and eccentricity in the rostral bank of the arcuate roughly correspond to the
map of saccade amplitude (Schall, 1997).
Connections of FEF with extrastriate visual cortex and with superior colliculus are
topographically organized (Schall, 1997; Schall and Thompson, 1999). FEF receives
afferents from most prestriate visual cortical areas and most areas in PFC, including
SEF, areas 46 and 12, and anterior cingulate area 24 (Barbas and Mesulam, 1981;
Stanton et al., 1993; Schall et al., 1995a). Projecting to oculomotor structures in-
cluding the caudate nucleus, the deep layers of the superior colliculus and brainstem
pre-oculomotor nuclei (Schall et al., 1995a), place FEF in a position to directly trigger
saccades.
The current definition of FEF includes an area much smaller than the general
arcuate concavity, and corresponds to areas 8Ac, 8Am, 45a and 45b (Preuss and
Goldman-Rakic, 1991; Schall, 1995). FEF lies within the rostral bank of the posterior
curve of the arcuate in frontal cortex of macaques. It extends anteriorly over the lip of
the arcuate a few millimeters onto the surface of the prearcuate gyrus, and posteriorly
a few millimeters onto the caudal bank (Bruce et al., 1985; Tehovnik et al., 2000).
19
Mediolaterally, it extends about 10 mm, centered approximately on the arcuate’s
midpoint as defined by the intersection of the straight extension of the principal
sulcus with the arcuate. However, the ventral and dorsal boundaries of FEF are less
clear (Schall et al., 1995b). The overall area of the FEF is approximately 100 mm2
(Tehovnik et al., 2000).
In terms of cytoarchitecture, area 8 extends only to the lip of the arcuate sulcus,
which excludes part of FEF included in the functional definition (Tehovnik et al.,
2000). However, Stanton et al. (1989) showed that though the location of FEF does
not line up with previously mapped cytoarchitectonic areas, it does coincide with
an area of increased concentration of large layer V pyramidal cells. Along the lip of
the prearcuate gyrus and at dorsomedial and ventrolateral locations outside of the
FEF, there is a marked decrease in number of large pyramidal cells in layer V. Given
that the layer V pyramidal cells are thought to be the major conveyors of oculomotor
signals from the FEF to the superior colliculus and the pons (Tehovnik et al., 2000),
this is consistent with lower thresholds for saccade initiation being found here.
1.5.3 8Ar
Much of what we know about the area immediately anterior to the FEF (current
definition) comes from pre-1985 studies of the arcuate concavity, as well as studies
of the PFC that recorded from this area (e.g., Zaksas and Pasternak, 2006; Kim and
Shadlen, 1999). Two recent studies have also reinforced the evidence that this area
is part of the frontal lobe oculomotor region (Seidemann et al., 2002; Moschovakis et
al., 2004).
8Ar is a transition zone located in the caudalmost prearcuate concavity and is
distinct from the FEF and the caudal part of area 46 (Gerbella et al., 2007). This
area includes the cortical surface between the rostral boundary of 8Ac and area 45a,
and the caudal boundary of area 46 (Schall, 1997). Preuss and Goldman-Rakic (1991)
defined this myeloarchitectonic area 8Ar as shown in Figs. 1.4 and 1.8. Gerbella et
al. (2007) describe 8r as an architectonic area just rostral to the FEF, correspond-
ing to Preuss & Goldman-Rakic’s area 8Ar. Like FEF, this area has high prefrontal
connectivity (88% of labeled cells). However, unlike FEF, this area’s only robust
extrafrontal connection is with dorsal and ventral LIP (Medalla and Barbas, 2006).
20
Figure 1.7: Eye movements evoked by microstimulation along the bank of the arcuatesulcus (from Bruce et al., 1985, with permission).
Less-robust connections are also found with areas MT, FST and V4, however (Ger-
bella et al., 2009). 8Ar also receives input from V2, V3 (very light), V4 (moderate),
TEO (heavy) (Schall, 1995).
As described above, several lines of evidence suggest that 8Ar and FEF are distinct
areas. However, despite differences in microstimulation thresholds, connectivity, and
21
a
b
Figure 1.8: FEF connectivity schematic, from Schall et al. (2002). (a) Frontal lobeoculomotor areas. (b) Relationships between input and output in FEF, SEF andACC. (Reprinted from Neuron, Vol. 36, Schall et al., Monitoring and control of actionby the frontal lobes, p. 309-22, Copyright 2002, with permission from Elsevier.)
neuronal density, much less is known about physiological differences between these
regions.
In general, this region has been shown to play a role in visuospatial functions and in
spatial working memory (e.g., see Levy and Goldman-Rakic, 2000; Romanski, 2004).
Zaksas and Pasternak (2006) described here a class of direction-selective visually
22
responsive neurons, which play an executive role while monkeys performed a direction
discrimination task in which the sample and test stimuli were separated by a brief
memory delay.
Kim and Shadlen (1999) also showed this area to be involved in decision-making.
They recorded from FEF and the posterior third of principal sulcus region (roughly
corresponding to areas 8Ar & 46), that responded selectively when the monkey
planned a saccade to within neurons’ RFs.
Gamlin and Yoon (2000) also showed evidence that this region plays a role in
vergence and ocular accommodation. The recent studies by Seidemann et al. (2002)
and Moschovakis et al. (2004) demonstrated a clear role for this area in oculomotor
preparation. Within the caudal prearcuate concavity, Gerbella et al. (2007) also found
that, similarly to 45B, this area was activated by execution of saccadic eye movements.
Looking across these studies, we see that within this arcuate region, there are areas
specialized for all classes of voluntary eye movements: saccades, smooth pursuit, and
vergence. This area also represents complex cognitive and motor signals.
The bottom line is that without microstimulation or histology, it is difficult if not
impossible to determine whether recordings are from FEF or 8Ar. What is clear is
that both areas contain signals tightly connected to eye movements, along with other
cognitive signals.
a b
Figure 1.9: Oculomotor connectivity schematic (from Schall, 1997, with permission).(a) Projections connecting oculomotor areas across the brain. (b) Schematic ofoculomotor functional connections.
23
1.6 Linking movement preparation to behavior
As we described in the previous section, several brain areas in the frontal lobe are
involved in the preparation and control of somatic and eye movements. In addition,
over the past few decades, much progress has been made in understanding how the
brain uses sensory input to guide behavior. Studies of neural circuitry in monkeys have
revealed a number of interconnected cortical and sub-cortical areas that participate
in sensory-guided decisions and movement planning (Shadlen and Newsome, 1996;
Horwitz and Newsome, 1999; Kim and Shadlen, 1999; Schall, 2001; Coe et al., 2002).
As a monkey receives sensory signals, neural activity in different areas of the brain
reflects aspects of the upcoming action, such as weighing sensory evidence (Newsome
et al., 1989; Shadlen and Newsome, 2001; Romo et al., 2004), comparing the value
of different actions (Barraclough et al., 2004; Sugrue et al., 2004; McCoy and Platt,
2005; Louie and Glimcher, 2010, etc.), or preparing movements (Evarts and Tanji,
1974; Tanji and Evarts, 1976; Wise, 1985). However, comparatively little is known
about how the dynamics of these computations play out across neural circuits or
neural populations.
In the beginning of this chapter, we discussed work supporting the idea that
voluntary movements are prepared before they are executed, and that neurons in
several premotor areas reflect this process. In the next two sections, we focus more
closely on how the state of movement preparation is reflected in neural activity.
1.6.1 Models and paradigms for studying movement prepa-
ration in the oculomotor system
A number of studies addressing the relationship between single-unit neurophysiol-
ogy and reaction time have explained variability in reaction time based on features
of individual neurons’ responses (Hanes and Schall, 1996; Dorris et al., 1997; Schall
and Thompson, 1999). These studies have been conducted both in the context of
instructed delay and RT tasks (introduced in Section 1.2). In RT tasks the processes
of movement preparation and initiation are difficult to distinguish. Therefore, the
studies employing RT tasks generally focus on the process of movement initiation
by measuring neural activity in the few hundred milliseconds preceding a saccade or
reach. In the context of RT tasks, neurons’ firing rates increase rapidly preceding
24
a given movement in motoric cells in many areas of the brain. In many models of
movement preparation, sets of cells must increase their activity toward their indi-
vidual thresholds in order to trigger a movement. These firing rate thresholds are
idiosyncratic to each neuron. Hanes and Schall (1996) discuss two classes of models
that could account for variability in RT given this rise of firing rate to a threshold.
In one class of models, the level of the threshold varies from trial to trial, causing
the increase in firing rate to trigger movements at different times depending on the
trigger’s level. In another class of model, the threshold is fixed and variations in the
speed of firing rate increase account for RT variability. For the FEF neurons they
recorded from, Hanes and Schall’s experiments supported the second class of mod-
els. In this thesis, we will refer to such fixed-threshold, variable-increase models as
rise-to-threshold.
In Hanes and Schall’s formulation, rise-to-threshold models describe the relation-
ship between RT and neural activity correlated with movement initiation. However,
variations of a rise-to-threshold hypothesis have been extended to investigate the
relationship between neural activity and behavior in many different contexts. This
extended class of rise-to-threshold hypotheses has been used to describe RT variabil-
ity in terms of individual cells’ variations in firing rate in many oculomotor areas of
the brain (e.g., Dorris et al., 1997; Roitman and Shadlen, 2002).
In instructed delay tasks, an extended version of the rise-to-threshold hypothesis
predicts that trials with short RTs should correspond to trials in which neurons’
firing rates were higher at the time of the go cue, the subject’s instruction to move.
In the reach system, several groups have also associated higher firing rates with faster
RTs (Lecas et al., 1986; Riehle and Requin, 1993; Bastian et al., 1998, 2003), even
for neurons that were not directionally selective (Bastian et al., 2003). A study
in humans describes a similar finding, demonstrating the relationship between the
latency of subjects’ response and the magnitude of the lateralized readiness potential,
a signal from the scalp that precedes movements (Kutas and Donchin, 1974). This
body of work supports the idea that by measuring neural activity during movement
preparation, we can gain insight into the processes that control timing of behavioral
output. Understanding how temporal agreement about when to move is coordinated
across areas is an open question of interest to many.
Many neurons in many brain areas contribute to each movement (Georgopoulos,
25
1996; Schall and Thompson, 1999). This suggests that the activity of any single neu-
ron is not necessary for movement production. Although Hanes and Schall (1996)
showed that using parameters from a single neuron a simple linear rise to threshold
simulation could accurately predict saccadic reaction time, it is unclear how this re-
lationship holds up for neurons less directly involved in movement initiation, or how
it extends to populations of neurons. Also in the context of a RT task, Thompson et
al. (1996) describe how presaccadic activity in FEF coincides with similar activation
throughout the oculomotor system in a winner-take-all race, and that the ultimate
saccade is made into the movement field of the set of neurons in which the movement
activity first reaches threshold. However, this assumes that each neuron is well de-
scribed by a rise-to-threshold model (Fig. 1.10). The Thompson model describes how
a population of motoric neurons increases its activity right before a saccade. Suppose
that all of the individual neurons did in fact act like rise-to-threshold neurons directly
preceding a saccade. However, it is not at all obvious that we could gain any insight
into movement preparation by studying this activity related to movement initiation
(Churchland et al., 2010). What is the best way to relate behavior to movement
preparatory responses in populations of neurons that may have different tuning and
response properties? This is one of the central questions of our research.
1.6.2 Extending single-neuron models to a population
Though much of the research about the neural basis of movement preparation has
been done in the oculomotor system, it has been in the context of single-electrode
recordings, where the properties of the stimulus were tailored to maximize individual
neurons’ responses. However, it is not clear whether our conceptual understanding of
movement initiation derived from single unit recordings will work when population ac-
tivity related to movement preparation is considered. By performing our experiments
in an oculomotor area, but recording across a population of neurons with different
tuning properties, we can build on previous research to develop more complete models
of movement preparation.
26
go cuetargets on
Ac
tiv
ity
(s
pik
es
/se
c)
Ac
tiv
ity
(s
pik
es
/se
c)
RT
FR - FR0
Extended rise-to-threshold hypothesis
Higher firing rates -> faster RTs
Figure 1.10: The rise-to-threshold hypothesis predicts that the higher the firing rateat the time of the go cue, the faster a saccade can be triggered.
1.7 Dynamics of movement preparation
1.7.1 Motivation to study dynamics
Whether making a decision about noisy sensory input, or trying to use a snapshot
of activity in the brain to control a prosthesis, we want to be able to understand the
dynamics of movement preparation on single trials. Though this work focuses on basic
science, our motivation for studying the dynamics of movement preparation is also
to help validate some of the tools which are important for prosthetics development.
Our overall goal is to understand how movement preparation unfolds on the timescale
relevant for behavior. The brain takes in input from many neurons at once, and uses
whatever signals are available on a given trial to produce a movement. If there is
uncertainty about where to move, the brain needs to find a way to continue planning
the movement, though perhaps leaving the plan more flexible. In the context of
neural prosthetics, understanding how plans are formed on single trials is particularly
important; averaging across trials is not an option.
Individual neurons in different areas of the brain reflect movement preparation in
different ways, but little is known about how these diverse responses fit together as
part of a larger dynamical system. To study the dynamics of movement preparation
at a fine temporal scale, we need to examine how the underlying signals of the neu-
ronal population evolve from moment to moment. Difficulty in interpreting single-cell
responses is often solved by averaging neural activity across many trials; however, this
would possibly obscure idiosyncrasies of neural dynamics that vary from trial to trial.
27
The way we are studying the dynamics underlying movement preparation is to look
at many cells on single trials.
The reason we want to take this approach is that it gives us a richer look at
the system than looking at single units. As the monkey performs his task, there
is a state the system is in. Multielectrode recording gives us a better look at this
hypothesized state, and provides a rich look at the system on a trial-by-trial basis.
This gives us the leverage we need for looking at how the state of the system evolves
on a given trial. Our goal is to trace the state of the underlying system. Is the state
of the system related to eye-movement parameters? In order to convince ourselves
that the state reflects underlying dynamics, we need to confirm that it relates to
behavior. What we want to know is whether the state of the system is tightly linked
to the trial-by-trial parameters of eye movements. This has been looked at in FEF and
superior colliculus in the context of single-electrode recording, but not with resolution
multielectrode recording now offers. Additionally, much of the previous work in the
oculomotor system was framed in the context of studying movement initiation, rather
than movement preparation.
Churchland et al. (2006) explored this in PMd, evaluating how an extension of
the rise-to-threshold hypothesis applies to populations of neurons. They found that
shorter RTs corresponded not to trials when neurons’ firing rates were highest, but
rather, when cells’ firing rates were closest to their cross-trial mean. In the state-
space model they propose, moving close to the cross-trial mean is represented by the
population activity entering an “optimal subspace” (Fig. 1.11). On trials when the
monkey has sufficient time to prepare his movement, the population activity enters
the optimal subspace, and RTs are shorter on these trials.
Afshar et al. (2010) extended the optimal subspace hypothesis, demonstrating that
while RTs are fastest on trials when PMd population activity is close to its mean,
that there is also structure within this optimal subspace. In the optimal subspace
hypothesis, the only factor that affects RT on a given trial is how close the neural
activity is to the heart of the optimal subspace (Fig. 1.11). In contrast, in the extended
rise-to-threshold hypothesis, the only relevant factor is how quickly cells’ firing rates
increase, or in some versions, how high the firing rate is before an upcoming saccade.
Afshar et al. draw on both of these concepts, putting forth their “initial condition
hypothesis.” This hypothesis predicts that within an optimal subspace RTs are even
28
Figure 1.11: Optimal subspace hypothesis (from Churchland et al., 2006, with permis-sion). Each axis in this state space corresponds to a neuron’s firing rate. As neurons’firing rates evolve over the course of a trial, this activity is represented as a trajectorymoving through the state space. Distinct regions of the state space correspond toplans for different movements (illustrated by shaded regions). RTs are shortest whenneural trajectories reach these optimal subspaces by the time the go cue is given.
faster on trials when trajectories are closest to the direction they need to move during
movement initiation. Here, we bring the initial condition hypothesis to the oculomotor
system. We build on this model to explore the relationship between RT and individual
neurons of the population in greater detail.
1.7.2 Open questions: using state-space approaches in the
oculomotor system
The initial condition hypothesis is not incompatible with the rise-to-threshold hy-
pothesis, but rather it aims to describe the neural basis of movement preparation at
the system level. Though the activity of particular neurons may be explained from
single-neuron frameworks like rise-to-threshold, intuition about the dynamics of the
system cannot be built up from individual neurons. In our work, we will also be able
to compare our results to Afshar’s recent study in PMd, in the context of planning
arm movements (Afshar, 2008). This will allow us to test whether the dynamics of
motor planning generalize to many movement preparation systems in the brain.
29
1.8 Questions addressed here
In this chapter, we gave an overview of behavioral and physiological evidence for
movement preparation. We focused on a premotor area related to arm movements
and a premotor area related to eye movements, and asked how movement preparation
is reflected in neural activity of these areas. Discussing some of the models used to
describe movement preparation, we speculated about mechanisms underlying this
process.
In this thesis, we focus our oculomotor studies on prearcuate cortex, an area
involved in eye movement preparation and decision-making. Combining a rich behav-
ioral paradigm with heterogenous populations of neurons recorded on our arrays gave
us the opportunity to: (1) explore this area’s role in movement preparation, which
has been implicated but not addressed directly, (2) look at how movement prepara-
tion is represented in a oculomotor population, (3) compare responses in dots epoch
from population to single cells elsewhere, and (4) compare systems-level descriptions
of movement preparation in prearcuate and PMd.
Chapters 2 and 3 focus on the oculomotor system, but also compare some results
to the reach system.
Chapter 2 describes basic response properties from prearcuate cortex, and covers
the period of the trial preceding the saccade, in which the neurons need to integrate
sensory information in order to decide where to saccade. It also explores the effect
of past events on the neurons’ firing rates, both within a trial and into subsequent
trials.
Chapter 3 describes in detail how we can evaluate the behavioral relevance of the
neural dynamics we see in Chapter 2.
Chapter 4 reviews some of the experiments we proposed in the reach system, and
approaches to including some of the behavioral elements (e.g. decision-making) that
are part of the oculomotor studies.
Chapter 2
Response properties of prearcuate
cortex
2.1 Background & Motivation
Prefrontal cortex (PFC) plays a significant role in the organization of movement and
has been described as “the peak of the hierarchy of anterior neural structures dedi-
cated to the execution of action” (Fuster, 1993). Neurons in PFC reflect a wide range
of processes, including decisions in the context of sensory discrimination (Kim and
Shadlen, 1999), working memory (Miller et al., 1996; Zaksas and Pasternak, 2006),
and signals related to eye movements, attention and executive control (reviewed by
Miller, 2000). The majority of research in the PFC has focused on relating activ-
ity from individual neurons to behavior. However, much less is known about the
dynamic processes driving this neural activity. To learn more about the processes
underlying visually-guided decision-making and movement preparation, we need to
better understand how activity of simultaneously recorded PFC populations relates
to task properties and behavior. We draw on the body of literature describing these
relationships for individual neurons.
Kim and Shadlen (1999) recorded from individual neurons in the PFC (focusing on
several areas, including prearcuate cortex) while their monkeys performed a random
dots discrimination task. They found that most neurons responded to target onset
in their response field, and firing rates reflected the direction and strength of the dot
movement, as well as the direction of the upcoming saccade. While monkeys were
30
31
trying to assess the direction of motion of the dots, neurons reflected the strength of
the motion. By the end of the dots epoch, many neurons reflected only the monkeys’
target choice. This work demonstrated that PFC neurons encode the outcome of
sensory processing, but also reflect the process of converting these signals into a
binary decision variable.
We would like to look at these processes of action selection with finer resolution,
inspecting how trial-to-trial differences in the state of the system relate to the mon-
key’s behavior on a given trial. Spiking noise makes it difficult to infer single-trial
estimates of the system’s state from individual neurons. However, combining simul-
taneous signals from many neurons on a given trial gives a better estimate of the
system’s state. Our ultimate goal is to use these single-trial estimates to learn about
the processes underlying decision-making and movement preparation. Before relat-
ing single-trial population activity to behavior, a useful first step is to use the same
methods to simply characterize the average dynamics of the population in the context
of the monkey’s task. This intermediate approach was recently taken by Machens et
al. (2010) to demonstrate the population encoding of task variables in PFC.
In this chapter our goal is to characterize response properties of a collection of PFC
neurons in the prearcuate region, from array recordings. To characterize responses
from ensembles of neurons in prearcuate cortex, we ask: (1) What are individual
cells’ responses, and how do they vary for different parts of the trial and in different
contexts? How similar are these cells to those in other oculomotor areas like LIP
or FEF? (2) Combining information from individual cells, what can we learn about
the dynamics of the population response? (3) What can we learn from population
recordings that was not possible with single-unit recordings?
In our recordings, we find neurons that respond to visual, decision-related, and
motoric aspects of the task, as well as many that represent past choices or reward
history. To what extent can we gain insight into neural dynamics by drawing together
signals from this diverse population? While a monkey performs a direction discrimi-
nation task or plans an eye movement, how are the dynamics of his decision process
reflected in the signals we record? We begin by describing some of the basic response
properties of neurons in the prearcuate area between the principal sulcus and FEF,
building on previous work described in Chapter 1.
In each of the following sections, we will describe both single-unit responses from
32
our array recordings, as well as characterizing the population as a whole. We highlight
some of the diversity of response types from individual units by showing peri-stimulus
time histograms (PSTHs), which relate the average responses of a neuron to differ-
ent parts of the task. In many studies, the population activity is summarized by
averaging together PSTHs from individual units. However, without a certain level
of homogeneity, averaging PSTHs cancels out rather than emphasizes the activity in
which we are interested. In this chapter we will use dimensionality reduction methods
to describe the average dynamics of the population, akin to a PSTH, but in multiple
dimensions. The advantage of this approach is that it allows us to visualize how the
population activity evolves over the course of the trial, without requiring units to
have similar response properties.
2.2 Methods
2.2.1 Subjects
Three adult male rhesus monkeys, C, T and V, were trained to perform a two-
alternative, forced-choice motion discrimination task, in which saccades served as
the operant response. Daily access to fluids was controlled during training and exper-
imental periods to promote behavioral motivation. Before training, the monkeys were
prepared surgically with a head-holding device (Evarts, 1966) and a scleral search coil
for monitoring eye position (Judge et al., 1980). After initial training, we implanted
a 96-channel silicon electrode array (Blackrock Microsystems, Salt Lake City, UT),
as described in (Churchland et al., 2006; Hatsopoulos et al., 2004). Training, experi-
ments and data collection were performed by John Reppas, Jamie Sanders, and Stacy
Rosenbaum, in the lab of Dr. Bill Newsome.
All surgical, behavioral, and animal care procedures complied with National Insti-
tutes of Health guidelines and were approved by the Stanford University Institutional
Animal Care and Use Committee.
2.2.2 Task design and training
Monkeys performed a delayed saccade task and a direction discrimination task. In
the delayed saccade task, monkeys made saccadic eye movements to targets presented
33
Fixate Targets On Delay Period Go Cue Saccade
Fixate Targets On Motion On Delay Period Go Cue Saccade
a
b
Hold Period Reward
Hold Period Reward
Figure 2.1: Behavioral tasks. (a) Monkeys were trained to perform delayed saccadesto targets on a radial grid. On each trial, a single target was presented one of thesethese grid positions. (b) In the direction discrimination task, monkeys were trained tomake a perceptual discrimination about the direction of motion in a patch of randomlymoving dots, and to saccade to the target in this direction. The presentation of thedots stimulus was followed by a delay period (∼1.5 sec) before the go cue was given.
within a polar grid of potential spatial locations, as shown in Figure 2.1a. Monkeys
were rewarded for shifting their gaze inside the target window after a variable delay
period (∼500-1500 ms). In some experiments, the fixation point was placed in the cor-
ner of the monitor to allow saccade eccentricities up to 24◦ to be measured. Neurons
in prearcuate cortex can have response fields for saccades greater than 24◦ (Suzuki
and Azuma, 1983; see Fig. 1.6h). However, physical limitations of our monitor limited
measurement of saccade eccentricities greater than 24◦.
Figure 2.1a illustrates the sequence of events comprising a typical trial of the
delayed saccade task. From left to right, trials began with the onset of a small dot
that the monkey was required to fixate for 150 ms. Next, one saccade target in the
radial grid (hollow gray circle) appeared for ∼500-1500 ms, in one of the locations
drawn in Fig. 2.1a. Following this delay, the fixation point disappeared, cueing the
monkey to saccade to the target. After completing the saccade, the monkey was
required to hold fixation at the target during a second delay period after which his
reward was delivered.
Figure 2.1b illustrates the sequence of events comprising a typical trial of the
motion discrimination task. On each behavioral trial the monkeys observed a noisy
random-dot motion stimulus and reported which of two possible directions of motion
were present by making a saccadic eye movement to one of two targets. We refer
34
to these targets as ‘T1’ and ‘T2’, which were placed at different locations across
experimental sessions. From left to right in Figure 2.1b, trials began with the onset
of a small dot that the monkey was required to fixate for 150 ms. Next, two saccade
targets (hollow gray circles) appeared for 250 ms. The two targets were 8-16 degrees
eccentric from the visual fixation point and either placed on opposite sides of the
fixation point, forming a line centered on the fixation point (contra-ipsi configuration),
or separated by 90 degrees in the contralateral hemifield, with the fixation point
forming the vertex of a right triangle (contra-contra configuration). The targets were
positioned in-line with the axis of motion being discriminated.
The targets were visible for 250 ms before onset of the visual motion stimulus,
which appeared for 800 ms, centered on the fixation point. The motion stimulus was
composed of dynamic random dots, viewed through a circular aperture on a dark
computer screen. On each trial a variable proportion of the dots moved coherently
in one of two directions while the remaining dots were flashed transiently at random
locations and times (for a detailed description see Britten et al. 1992, 1993; Bair et
al. 2001). The difficulty of the discrimination was varied parametrically from trial-to-
trial by adjusting the percentage of dots in coherent motion: the task was easy if most
of the dots moved coherently (e.g., 50% or 100% coherence), but became progressively
more difficult as the coherence decreased. Typically, the animals viewed a range of
signed coherences spanning their psychophysical threshold. The animals were always
rewarded for indicating the correct direction of motion, except at 0% coherence where
they were rewarded randomly (50% probability) irrespective of their choice.
Following offset of the motion stimulus, the monkey was required to maintain fix-
ation for a variable delay period (300-900 ms for monkey V, 500-1400 ms for monkey
T), after which the fixation point disappeared, cueing the monkey to report his deci-
sion with a saccade to the target corresponding to the perceived direction of motion.
If the monkey chose the correct direction of motion, he received a reward after holding
his gaze at the target location during the second delay period.
In both tasks, fixation was enforced throughout the trial by requiring the monkey
to maintain his eye position within an electronic window (1.5◦ radius) centered on the
fixation point. Inappropriate breaks of fixation were punished by aborting the trial
and enforcing a time-out period before onset of the following trial. Psychophysical
decisions were identified by detecting the time of arrival of the monkey’s eye in one of
35
two electronic windows (3-5◦ radius, scaled linearly with target eccentricity) centered
on the two choice targets (T1 and T2). All trials were presented pseudo-randomly in
block-randomized order.
2.2.3 Procedures
During both training and experimental sessions monkeys sat in a primate chair at
a viewing distance of 57 cm from a CRT (cathode ray tube) color monitor. Visual
stimuli were presented on the monitor under computer control. The monkeys’ heads
were positioned stably using the headholding device, and eye position was monitored
throughout all experimental sessions by means of a magnetic search coil apparatus
(0.1◦ resolution; CNC Engineering, Seattle, WA). Juice was delivered on correct trials
via a tube attached to the monkey’s chair.
Behavioral control and data acquisition were managed by a PC-compatible com-
puter running the REX software environment (Hays et al., 1982) and QNX Software
System’s (Ottawa, Canada) real-time operating system. Visual stimuli were gen-
erated using a VSG graphics card (Cambridge Graphics, UK) and presented on a
CRT display. Neural signals were amplified and collected along with digitized task
events and eye position traces using the Cerebus system (Blackrock Microsystems,
Salt Lake City, UT) operating in conjunction with REX. Thresholds for spiking ac-
tivity were set as 3.75 times the root mean square of the signal amplitude of each
channel, and were recalculated at regular intervals throughout an experimental ses-
sion. Snippets of waveforms were saved at the time of each threshold-crossing, from
0.3 ms before the threshold-crossing until 1.3 ms after and digitized at 30 kHz. All
data were subsequently analyzed offline with custom scripts written in the MATLAB
(The MathWorks, Inc., Natick, Massachusetts) programming language.
2.2.4 Array locations and neural recordings
Arrays were implanted into the surface of cortex, in the arcuate concavity between the
caudal aspect of the principal sulcus and the anterior bank of the arcuate sulcus, as
estimated visually from local anatomical landmarks (Fig. 2.2). Figure 2.3 shows the
placement of arrays with respect to the arcuate and principal sulci for all 3 monkeys.
36
PMd
anterior bank of arcuate
(prearcuate cortex)
as
ps
Figure 2.2: Array location. Neural activity was recorded from an array of 96 electrodesimplanted in the anterior bank of the arcuate, or in dorsal premotor cortex of monkeys(for PMd comparison figures). PMd array is from monkey H; prearcuate array isfrom monkey V. AS, arcuate sulcus; CS, central sulcus; PCD, pre-central dimple; PS,principal sulcus.
Monkey V
Left Hemisphere
Monkey T
Left Hemisphere
Monkey C
Right Hemisphere
a b c
PS
AS
PS
AS
PS
AS
A P A P P A
Figure 2.3: Array placement, with respect to arcuate and principal sulci. Anterior(A) and posterior (P) directions are indicated with labels and gray arrows, pointinganterior. Exact placement varied across the three monkeys with arrays implanted inthis region. Differences in location of the array likely influenced which subpopulationsof cells were recorded, which may explain some of the variation in results seen acrossmonkeys. (a) Monkey V. Units on this array were tended to be more motoric andless driven during the direction discrimination part of the task. (b) Monkey T. Unitswere more generally modulated by visual stimuli and the decision task than MonkeyV. (c) Monkey C. Responses on this array were overall much weaker and also lessstable than those from the other monkeys.
37
Neural recordings were collected using silicon microelectrode arrays, with 96 1.5-
mm electrodes (in prearcuate arrays; 1.0 mm electrodes in PMd), and read out using
the Cerebus recording system. Spikes from each electrode were sorted and clustered
offline, based on a principal component analysis of the resulting voltage waveforms
using the Plexon Offline Sorter (Plexon Inc., Dallas, Texas). This process initially
overestimated the number of clusters; custom Matlab scripts were used to automat-
ically remove recording artifacts (transient events evoking rapid changes in firing
rate across the array), post-process and group clusters based on waveform shape,
inter-spike interval, and firing rate. Daily recordings yielded ∼100-200 single and
mulit-units (not distinguished here, but see Reppas et al., 2010), collectively referred
to as ‘units’. As in Yu et al. (2009), our analyses do not depend on having single
units, so we refer collectively to the single- and multi-units on our array as simply
‘units’.
Units were active to varying degrees during the target onset period (visually-
modulated units), the delay period (movement preparation units), and/or after the
saccade/reach had been executed (post-choice units) (see Reppas et al., 2010). Though
target locations were constant during an experiment, response fields of the neurons
were spatially scattered. Thus, targets were in the center of some units’ RFs, but
only in the periphery for others. Therefore, we developed a criterion for minimal
tuning that we employed to select units for analysis. To study motor preparation, we
included units with firing rates that were significantly modulated relative to baseline
for at least one target (p <0.05; t-test) during a 300 ms window preceding saccade
initiation, and that had an average firing rate of at least 1 spike/second higher for
one of the two target locations during this window. Units with tuned activity in this
epoch comprised roughly a third of all units on the array. Units’ preferred directions
were estimated as the target location which elicited the greatest activity during this
window. This weak test of tuning was used in order to increase the size of the pop-
ulation we sampled. However, the exact threshold used did not change the nature
of our results. We did not typically explore response fields systematically; hence,
our operational assignment of preferred direction may be off considerably from the
real preferred direction. Units with no modulation during this epoch were excluded
from this analysis. Number of units included for each dataset were as follows: 34-113,
38
SNR: 11.28 SNR: 6.46
SNR: 3.92 SNR: 1.85
1
0
-1
1
0
-1
1
0
-1
1
0
-1
A.U
.A
.U.
1.6 ms
Figure 2.4: Diversity of waveform signal-to-noise ratios. The signal-to-noise ratio(SNR) described by Suner et al., 2005 gives a quantitative measure of reliability ofthe waveforms. The four examples shown here are representative of neurons includedin our population. Arbitrary units on the vertical axis are determined by the RMSnoise across channels and are the same across all panels.
mean: 69 (monkey T) and 41-124, mean: 81 (monkey V). Of those, the following num-
bers were tuned for a given target location: 9-96, mean: 35 (monkey T) and 6-107,
mean: 40 (monkey V).
Example waveforms are illustrated in Figure 2.4 along with a signal-to-noise ratio
(SNR), used to measure across-trial reliability. This SNR is defined in Suner et al.
(2005) as: SNR = A2 ∗ SDnoise
, where A, the amplitude, is the peak-to-peak voltage
of the mean waveform, and SDnoise is the standard deviation of the noise. Neurons
with SNRs ≤ 1.75 were excluded from our analysis.
Inter-spike intervals (ISIs) for these waveforms are shown in Figure 2.5. Waveform
SNRs and interspike interval statistics tend to be correlated; units with high waveform
SNRs also tend to have ISIs with less evidence of contamination (spikes within the
refractory period) (Suner et al., 2005).
39
0
100
200
300
0
100
200
300
0 50 1000
50
100
150
200
Interval (ms)
0 50 1000
500
1000
1500
Interval (ms)
Co
un
tC
ou
nt
Figure 2.5: Diversity of inter-spike interval statistics. The inter-spike interval (ISI)characterizes response statistics, and was calculated for each unit as part of the spike-sorting process. ISIs were used as a signature of unit identity, and used to differentiatebetween different units on the same electrode.
2.2.5 Dimensionality reduction of neural data
From a dynamic systems point of view, our goal is to extract low-dimensional tra-
jectories that describe the evolution of this neural system over time during single
trials. These trajectories provide a compact summary of the activity of the system
during behavior, and may offer insight into the underlying biology of the dynamical
system. In order to achieve this low-dimensional representation, we apply Sensible
Principal Component Analysis (SPCA), (Roweis, 1998) and Gaussian Process Factor
Analysis (GPFA), (Yu et al., 2009) dimensionality reduction techniques to data from
our multielectrode recordings.
GPFA aims to uncover latent dimensions that provide a compact description of the
variance in a higher dimensional space, distinguishing between variance independent
to individual neurons and variance shared across the population. This shared variance
represents the fluctuations common to many neurons in the population, which should
give us information about the state of the underlying neural process driving the
population.
40
Using dimensionality reduction techniques to find neural trajectories through this
state space allows us to estimate the time evolution of this underlying neural process,
which we can visualize on single trials, or averaged together across trials (in the same
spirit as a population PSTH). Reducing the dimensionality of the population data
also allows us to combine data from many cells, regardless of individual response
profiles, and to reduce noise by emphasizing the most salient features of ensemble
response properties.
GPFA is different than PCA-like methods in a few important ways. First, rather
than filtering all data with a filter with one time constant, instead GPFA finds the
time constants appropriate for each dimension separately. Second, it distinguishes
between sources of noise particular to individual cells (private variance) and cross-
trial variation that is correlated across many cells (shared variance).
Though SPCA and GPFA generally yielded roughly similar results, SPCA was
more robust to some types of noise in the data (e.g., situations like the Heywood case
in Factor Analysis, Kolenikov et al., 2006). Therefore, SPCA was used for preliminary
assessment of datasets, and was preferable in some cases for rapid visualization. How-
ever, in GPFA time constants for each dimension are determined separately, which
allowed us to achieve finer resolution of trajectories than with SPCA.
Dimensionality reduction was performed on data collected during the time win-
dow corresponding to the epoch of interest. For SPCA, data were preprocessed by
convolving each cell’s spike train with a Gaussian (acausal, not time-shifted) kernel
with width of 30 ms to produce a vector of firing rate varying continuously in time
(the parameters used in preprocessing do not have a significant influence on results—
see Afshar, 2008 for detailed controls). These vectors were then downsampled by a
factor of 10, representing firing rates every 10 ms, and placed in a T x N array, where
T = number of timesteps and N = number of neurons. SPCA was then performed
on this array.
To apply GPFA to our data, we organized our data into an N x M array of spike
times in 1 msec bins, where N is the number of simultaneous units which met our
criteria for inclusion, and M is the total number of timesteps in the epoch of interest
multiplied by the number of trials. The square root transform was applied to binned
spike counts, which served to stabilize spiking noise variance. Parameters of the GPFA
model were fit using the Expectation-Maximization (EM) algorithm as described in
41
Yu et al. (2009). Neural trajectories were then extracted from this model. These
trajectories were orthonormalized to order the dimensions in terms of the covariance
explained.
Since the covariance of the population varies across different epochs of the trial,
the choice of time window upon which GPFA is run will affect the neural trajectories
generated. The smaller the time window, the more closely the parameters of GPFA
will fit covariance in the epoch of interest. Therefore, different epochs of time can be
chosen to emphasize different aspects of the data. For instance, performing GPFA on
the entire trial is useful for visualizing how the neural state evolves through different
parts of the task, but will yield worse estimates of covariance during the perisaccadic
epoch than if GPFA had been run on that epoch alone.
2.2.6 Computing the ROC predictive index
To compute the cells’ receiver operating characteristic (ROC) curves, we followed the
methods described in Shadlen and Newsome (2001). We advanced a 200 ms window
by 10 ms intervals across each neuron’s firing rate during the dots epoch (300 ms
before dots onset, 1000 ms after dots offset) and saccade epoch (600 ms before and
after the saccade onset) to measure whether a neuron’s firing rate was greater than
a criterion κ for trials of a given choice and coherence level.
We used 42 of our best-tuned prearcuate neurons within one dataset to compute
predictive indices based on these ROC curves. Since some neurons were tuned for
T1 saccades and others were tuned for T2 saccades, the neurons tuned for T1 had
positive predictive indices and T2 neurons had negative predictive indices. To average
across T1 and T2-preferring neurons, we flipped the sign of the ROC predictive index
for T2-preferring neurons to create a population ROC curve based on neurons tuned
for either target location.
2.3 Results
2.3.1 Tuning properties of neurons / cell classification
FEF contains cells with visual and movement-related activity (e.g., Bruce and Gold-
berg, 1985; reviewed by Schall, 1997). In Bruce and Goldberg’s recordings, 40% of
42
the neurons showed visual, but no movement activity, 20% responded before cued
saccades, but showed little or no visual response, and the remaining 40% had visual
and motoric response properties. For a subset of these neurons, they calculated in-
dices for comparing movement and visual activity. The neural activity used for this
calculation was found by taking the 100 ms interval of the task with the highest firing
rate and subtracting the background rate. The movement activity index was defined
as the ratio of a cell’s activity in a learned-saccade task, in which the monkey did
not receive a visual target cue, to the cell’s activity in a visually-guided saccade task.
Similarly, the visual activity index was the ratio of a cell’s activity in a purely visual
task (no saccade) to the cell’s activity in the visually-guided saccade task. Overall,
there is a continuum along these visual and movement axes, indicating that there are
not two distinct classes of neurons represented. However, a slight correlation (r =
-0.26) shows a weak tendency for visuo-movement neurons to be more visual or more
motoric. We compared these response properties with data we recorded in prearcuate
cortex.
Across hundreds of units in prearcuate cortex (from 3 monkeys performing saccade
tasks), we found subsets of units that were modulated by various combinations of
visual stimuli, the dots task, during the perimovement period, and during the hold
period before the monkey received his reward. As in Bruce and Goldberg (1985),
our data also shows a diversity of responses with no clear clustering based on visual,
motoric, or decision-related signals (data not shown). Detailed description of neural
responses in prearcuate can be found in Reppas et al. (2010).
Figure 2.6 shows PSTHs for an example neuron from one of our arrays during
the delayed saccade task. The PSTHs are spatially arranged to correspond to the
delayed saccade target locations. Each PSTH shows this neuron’s response at the
particular target location, aligned in time to target onset and movement onset (left
and right panels, respectively). The polar tuning curve in the center summarizes
tuning strength for the visual response to target onset. The direction and strength
of tuning are computed for each neuron, during the target onset epoch, but also
during the delay period before the go cue and in the epoch before the saccade is
executed. Figure 2.7 shows tuning across the array for these three epochs, for one
example dataset for each monkey. For all three monkeys, units tended to be tuned
for contraversive saccades (Fig. 2.7), and tuning was weakest during the delay period
43
Monkey V
Tuning direction (θ): 82
S/N: 7.6
1 sec
Figure 2.6: Visual responses to the outer-most ring of targets from Figure 2.1, at12 degrees eccentric. Each PSTH is the average response across trials for the targetat that location, aligned in time to target onset and movement onset (left and rightpanels, respectively). The polar tuning curve is calculated from the time windowshown in red, just after targets appear. Mean and SEM are shown by the datapoints and error bars. We fit a circular Gaussian function to this data, shown in thecontinuous, elliptical plot overlaid on the data points. From this, the direction oftuning (theta) is estimated as the peak of the fitted Gaussian. For this neuron, thetawas 82 degrees, very near vertical. A signal-to-noise (S/N) ratio for tuning strengthis also computed, which is the peak response over the baseline or pre-target response.An S/N ratio of almost 8, shown here, is very strong tuning.
of the trial, and strongest at target onset and right before a saccade.
44
Target Onset Delay Period Pre-Saccade
a
b
c
log S/N ratio
Figure 2.7: Tuning during delayed saccade task. All units with target-onset tuningcurve fits with a S/N ratio > 2 (i.e. peak is at least twice baseline) are shown herefor one example experiment for each monkey. Plots show the preferred direction oftuning (theta) versus log-S/N ratio (radius). Each point is data from one neuron, andpanels show tuning during three epochs of the task: around target onset (50-350 msafter target onset), during the delay period (on average 650-950 ms after target onset),and before the saccade (250 ms before saccade onset). (a) - (c) show responses fromMonkeys T, V, and C, respectively. Arrays in Monkeys T and V were implanted inthe left hemisphere; Monkey C’s array was in his right hemisphere (Fig. 2.3). Acrossall three task epochs and all three monkeys, tuning is biased towards the contralateraldirection. Fewest neurons are tuned in the delay period, and when they are, theirS/N ratio tends to be lower.
45
2.3.2 Diversity of PSTHs
Once chronically implanted, non-moveable arrays are implanted, experimenters have
no control over the cells they sample. Therefore, recording from even well-characterized
cortical areas like FEF, array recordings could well yield a very different subpopula-
tion of neurons than recording with moveable electrodes. Recording with moveable
electrodes, experiments have the ability to select neurons that are responsive during
a particular task. Though the geometry of the arrays introduces a laminar bias—
neurons recorded likely come from the same layer or two, but there is no selection
bias as in single-unit recording. Not filtering for particular response properties, we
see diverse responses from the units on our arrays. These units are diverse in their
response properties and in their RF position with respect to saccade targets (see
Chapter 3 for a more extensive discussion of response diversity).
2.3.3 Visualizing population activity
In cortical areas where cells’ responses are fairly stereotyped, averaging PSTHs of
many cells reduces noise and gives an estimate of the population response. How-
ever, if neurons are tuned for different locations or have different timecourses of their
response, averaging together PSTHs will cancel signals from individual cells. Dimen-
sionality reduction offers us a better way to combine diverse responses. Reducing the
dimensionality of our data, we find the dimensions corresponding to shared fluctua-
tions of the population, onto which we can project the population activity. Though
we ultimately want to leverage this method to explore the population response on in-
dividual trials, we can also use this method to gain insight into the average dynamics
of the population. Here, we simply use this method to combine signals from di-
verse sets of neurons to make use of the information represented in the heterogeneous
population, rather than excluding it by averaging.
Figure 2.8 shows the average path the neural population follows through a low-
dimensional state space during single trials of the motion discrimination task. The
blue trace represents trials when the monkey ultimately made a saccade to the target
in his contralateral hemifield (T1) (red traces: same, for ipsilateral (T2) saccades).
As activity of the population settles following fixation, this is reflected in the trajec-
tories’ movement through this state space (Fig. 2.8a). Approximately 100 ms after
46
the targets turn on, the trajectories bend, corresponding to the visual transient seen
in many individual neurons. When the dots turn on (Fig. 2.8b), activity of the
population turns again, and follows the same path for T1 and T2 choices for ap-
proximately 200 ms, after which it moves in the direction of the upcoming saccade.
These low-dimensional representations of average population activity offer a simple
way to visualize activity of heterogeneous populations of neurons, compared with
other methods of combining the diverse responses of single neurons in prearcuate.
These low-dimensional visualizations can be used for rapid assessment of data and
for forming and testing hypotheses.
Fixate
Targets On
Motion On
x2
x1
Motion On
Saccade to T1
Saccade to T2
x2
x1
a b
Figure 2.8: Low-dimensional representation of population data. Applying SPCA(Roweis, 1998) to population data from individual trials, trajectories through thislow-dimensional space were computed and averaged with other trials correspondingto the same eye movement. Blue/red trajectories correspond to trials when the mon-key ultimately made a saccade to T1/T2. Dots on trajectories are spaced at 50 msintervals. (a) Beginning of trial, from fixation through dots on. Movement of thetrajectories during this part of the trial is likely driven by cells such as the ones inFig. 3.2b and Fig. 3.2c. (b) Middle of the trial, from dots on through the time of sac-cade initiation. These trajectories show the average path along which the populationevolves as the monkey responds during different task epochs.
2.3.4 Decision-related activity
As in Kim and Shadlen (1999), we find individual neurons in this area that respond
differentially during the presentation of the dots stimulus and represent planning to
an upcoming saccade target. PSTHs in Figure 2.9 highlight the diversity of some of
the decision-related responses in prearcuate. These neurons passed criteria for tuning
47
0 0.5 10
45
Dots On
−0.5 0 0.5
Saccade
0
45
0
15
Fir
ing
Ra
te (
sp
ike
s/s
)
0 0.5 10
80
Fir
ing
Ra
te (
sp
ike
s/s
)
Dots On
−0.5 0 0.5
Saccade
0
14
0
30
Fir
ing
Ra
te (
sp
ike
s/s
)40%20%10%5%0%
Unit 305
Unit 272Unit 280
Unit 278
Unit 285Unit 217
Figure 2.9: Peri-stimulus time histograms (PSTHs) for prearcuate units simultane-ously recorded during the dots task. Solid (dashed) lines correspond to eventual T1(T2) choices. The strength of the motion signal is represented by line color, as definedin the legend. Responses are aligned to time of dots onset (left panel) and saccadeonset (right panel). Individual units exhibit a wide range of response dynamics duringthe presentation of the dots stimulus, and leading up to the saccade.
in the dots epoch of the task, though not necessarily for having responses that distin-
guished between different strengths of motion signals or coherences. Some neurons
look very similar to LIP neurons, distinguishing between different coherences around
200 ms. Others, like Unit 272, are more unusual, showing a coherence-dependent
response at the same time but which disappears by the time the dots turn off. This
48
0.4
0.5
0.6
0.7
0.8
0.9
1
Pre
dic
tiv
e In
de
x
0 0.5 1
0.4
0.5
0.6
0.7
0.8
0.9
1
Pre
dic
tiv
e In
de
x
0
0.4
0.5
0.6
0.7
0.8
0.9
1
Pre
dic
tiv
e In
de
x
0 0.5 1 0
Dots On SaccadeDots On Saccade
Unit 305
Unit 272Unit 280
Unit 278
Unit 285Unit 217 40%20%10%5%0%
Figure 2.10: ROC predictive index for prearcuate units simultaneously recorded dur-ing the dots task. The strength of the motion signal is represented by line color, asshown in the legend. Individual units exhibit a wide range of ROC predictive indexvalues, which match coherence effects in PSTHs shown in Fig. 2.9. Note the oddreversal of coherence sign during the delay period in Unit 305.
type of neuron seems to reflect sensory discrimination, but not to be storing variables
related to the monkey’s decision. Figure 2.10 shows predictive indices computed from
cells’ receiver operating characteristic (ROC) curves, following the methods described
above and in Shadlen and Newsome (2001). The timecourse of the ROC predictive
indices roughly matches that of the cells’ PSTHs. For Unit 272, the predictive index
peaks around 300 ms, and then tapers off, consistent with that cell’s PSTH.
49
For comparison with responses to the same task in LIP, Figure 2.11 shows LIP
results from Shadlen and Newsome (2001) next to the average ROC predictive index
from one day of our experiments in prearcuate. Note the difference in vertical scales—
the magnitude of ROC predictive indices is greater in LIP. The dynamics in the two
areas are similar, though the absolute values of the predictive index are smaller for
prearcuate. Also note that the shape is slightly different, which is likely influenced by
some of the idiosyncratic ROC predictive indices from prearcuate neurons. Kim and
Shadlen looked at the ROC predictive index in a PFC population, including neurons
from prearcuate cortex (see Kim and Shadlen, 1999, Figure 5). Their results were
similar to both the LIP and prearcuate predictive indices in terms of dynamics of
predictability, with respect to coherence. The magnitude of their predictive index
was closer to that in LIP, perhaps due to stronger selection criteria about which cells
to include in their population. Taking note of the diversity of the individual ROC
predictive indices in Figure 2.10 emphasizes that for our population, averaging is
likely not the best way to combine these signals. Therefore, we sought a better way
to assess how much predictive information is captured by the population.
How can we combine the predictive information in the neural population without
averaging? We appeal to the low-dimensional visualizations to gain intuition about
how the strength of motion signals is represented in the population activity.
2.3.4.1 Visualizing the effect of coherence on population activity
Figure 2.12 shows a low-dimensional representation of the population activity during
the motion discrimination epoch of the task. Trajectories from individual trials are
averaged together based on eventual saccade direction (solid lines = T1, dotted lines
= T2) and strength of dot motion (indicated by color). Dots on the trajectory
are placed at 50 ms intervals. When the dots turn on, the trajectories start from
approximately the same place. As the monkey integrates the motion information in
the dots stimulus, the trajectories deviate toward the region of space representing a
movement plan for T1 or T2. Rotating this plot in three dimensions, it is apparent
that the mean trajectories toward these T1 and T2 plan regions separate earlier
on high-coherence trials. This is consistent with the dynamics seen in the ROC
analysis seen in prearcuate and in LIP in Figure 2.11, that decision-predicting neural
activity is evident sooner on high coherence trials. This is notable because it allows
50
LIP population
Dots on Saccade Dots on Saccade
Prearcuate population
500 ms
0.45
0.5
0.55
0.6
0.65
0.7
0.75
Pre
dic
tiv
e In
de
x (
me
an
)a b
40%20%10%5%0%
0.5
0.6
0.7
0.8
0.9
500 ms
51.2%25.6%12.8%6.4%0%
Pre
dic
tiv
e In
de
x (
me
an
)Figure 2.11: ROC predictive index comparison between prearcuate and LIP (fromShadlen and Newsome, 2001, with permission). (a) Average ROC indices acrossprearcuate neurons recorded. Note that averaging ROC curves with diverse time-courses like Fig. 2.10, Unit 278 and Fig. 2.10, Unit 272 will cause destructive in-terference, decreasing the overall amplitude of the predictive index. (b) AverageROC predictive index around the time of dots presentation and saccade in LIP (fromShadlen and Newsome, 2001). Though the absolute value of the predictive index islower in prearcuate, it should be noted that the stimuli were not optimized for theneurons recorded on the array and targets were not necessarily within neurons’ RFs.Nevertheless, the dynamics of the responses and the coherence effects were similar inboth areas.
us to visualize dynamics of decision-making within a more diverse population. This
coherence-based separation of the T1 and T2 trajectories in the state space can be
quantified by measuring the distance between mean trajectories, as a function of
coherence, as illustrated in Figure 2.13.
The analysis in Figure 2.13 can be performed in this low-dimensional space used
for visualization (as shown), but also in the high-dimensional space of firing rates
from each neuron in the population. Figure 2.13b shows that before the dots turn on,
the distance between the mean T1 and T2 trajectories is similar across coherences,
as would be expected since the monkey has not yet been given any information about
his eventual saccade target. After the dots turn on, the distance between the mean
T1 and T2 trajectories begins to increase, reflecting that the neurons are integrating
51
40%20%10%5%0%
dots on -> dots off, all coherences
t1
t2
x1
x2
x3
dots off
dots off
dots on
time
Figure 2.12: Low-dimensional representation of population activity from dots onsetto when the dots were extinguished 800 ms later. Using GPFA to project populationactivity into 3 dimensions, the average trajectories to each target are shown as afunction of coherence (see legend for colors). Solid (dotted) lines represent averagetrajectories on trials resulting in a T1 (T2) saccade. Velocity can be estimated bylooking at distance between dots on trajectories, which are drawn at 50 ms intervals.
the motion information to allow the monkey to make a decision about which way to
look. The distance between the high-coherence T1 and T2 trajectories increases faster
than that between the low-coherence T1 and T2 trajectories, presumably reflecting
differences in the timecourse of deciding which way to saccade, as seen in the ROC
curves. By the time the dots turn off, the distance between the mean trajectories is
nearly the same for all coherences, suggesting that neurons in prearcuate reflect the
decision-making process, but that this process is completed by the time the dots turn
off.
Figure 2.13a illustrates how the distance between mean trajectories is measured,
and Figure 2.13b presents results of applying the method in (a) to one dataset. This
trend is seen across multiple datasets in one monkey (monkey T). The neural popu-
lation recorded in monkey V was not as responsive during the dots epoch, though a
weak effect appears to be present in this monkey as well.
We repeated this analysis within the high-dimensional space (42 dimensions in
52
this dataset) and found similar results. Data shown in Figure 2.13 was prepared by
reducing the dimensionality to 8 dimensions using GPFA. Reducing the dimension-
ality to 3, 5, or 16 dimensions using either GPFA and SPCA also produced similar
results. Finding similar dynamics of the coherence-based separation of mean T1 and
T2 trajectories across different dimensions and using different methods demonstrates
that this result is not sensitive to the exact parameters of our analysis method.
40%20%10%5%0%
t1
t2
t i+1
x1
x2
x3
dots on -> dots off, all coherences
distance at t i
Euclidean distance between mean T1 and T2 trajectories
0 0.5 1
0
0.2
0.4
0.6
0.8
1
Dots on
Time (s)
% M
axim
um
Sep
arat
ion in 4
2-D
Spac
e
a b
Dots off
Figure 2.13: Measuring distance between mean trajectories as a function of coherenceand time. (a) To quantify the effect of coherence on neural dynamics during motiondiscrimination, we calculated the Euclidean distance between the average T1 and T2trajectories (in low or high-dimensional space) at each time point, as a function ofcoherence. (b) We normalized these distances by the maximum distance, and plottedthe normalized distance between T1 and T2 trajectories as a function of time. Thisplot shows very similar dynamics to the ROC predictive index curves, but unlikethe ROC curves, does not require averaging or selective processing based on units’tuning. Rather than the strict pre-selection of neurons that are typically used in ROCanalyses, this distance metric implicitly includes responses from all neurons.
2.3.4.2 Dimensionality of data
The low-dimensional representation of the population trajectories is compelling, but
how well does it capture coordinated activity of the population? Using GPFA, we can
distinguish between private variance, which is the variance particular to individual
neurons (e.g., from spiking noise), and shared variance, which describes trial-to-trial
covariation across the population. Of the total variance accounted for across the whole
trial the majority was private variance, with shared variance accounting for 13.9%
53
of the total variance in the population (Fig. 2.14, left panel). After this separation,
we can look at how much of the shared variance in the population is captured by
each of the latent dimensions. Figure 2.14 breaks down the shared variance in three
different epochs of the trial by how much variance is accounted for by each of 15
latent dimensions. During the dots epoch (800 ms, from dots onset to dots offset), 7
dimensions are needed to explain 75% of the shared variance. However, during the
saccade (150 ms before/after saccade) and reward epochs (the second hold period),
only 5 dimensions are needed. Visualizing neural trajectories in three dimensions
will give us a more accurate depiction of underlying dynamics when the first three
three dimensions capture a larger proportion of the shared variance. Knowing the
underlying dimensionality of the neural activity during particular epochs can guide us
in choosing how many dimensions to use in our analyses, such as the one described in
Figure 2.13. Future work may elucidate behavioral or task-related variables to which
these dimensions correspond.
Whole trial Dots epoch Saccade epoch Reward epoch
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
x12
x13
x14
x15
private
shared
source of
variance
variance
accounted for
by dimension x
Figure 2.14: Relative proportion of shared variance explained varies across epochs.Left panel: Shared variance accounts for 13.9% of the total variance of the population.Right panels: Relative contribution of dimensions to shared variance during dots,saccade and reward epochs of the task.
54
2.3.5 Effects of choice and reward history: this trial
After monkeys completed the direction discrimination part of the task and made their
saccade to the appropriate target location, they were required to hold their gaze at
this target for a second delay period, until the reward was given. This epoch of the
task, the ‘hold period’, is highlighted in Figure 2.15. Responses during this part of
the task were typically stronger than the visual, saccadic, or decision-related signals,
as described in Reppas et al. (2010). This suggests that neurons in our populations
are more tightly connected to circuits involved in evaluating the outcome of previous
actions than to circuits involved in taking in sensory input to select and generate
upcoming behavior.
Fixate Targets On Motion On Delay Period Go Cue Saccade Hold Period Reward
Figure 2.15: Behavioral task. As in Figure 2.1b, monkeys were trained to makea perceptual discrimination about the direction of motion in a patch of randomlymoving dots, and to saccade to the target in this direction. After the monkey’ssaccade, he was required to hold his gaze at the target during a second variable delayperiod before the reward was given.
Hold-period responses were robust both in the delayed saccade and in the dots
tasks. What do these responses reflect? Many cells that responded vigorously during
the hold period did not modulate their firing rates nearly as much during the other
task epochs (target onset, dots presentation, etc.). This suggests that the hold-period
response is not likely explained by low-level visual signals. We also wondered whether
the hold-period responses were being driven by eye position. To address this question,
we performed a control in which targets were placed at two positions along a horizontal
line. After collecting several hundred trials of the dots task in this configuration, the
targets were displaced horizontally such that the target that had been on the right
side stayed in place (becoming the left target), and the other target moved further to
the right (becoming the right target). In this configuration, the eyes were held at the
same position for saccades to the right target in the first block as the left target in the
second block. If neurons’ hold-period responses reflected eye position, the responses
55
a
b
Figure 2.16: Hold period activity represents previous action in some neurons (s) andprevious choice in others (b). Blue (red) lines correspond to T1 (T2) saccades on thecurrent trial.
to this target should be the same in both blocks. However, we found that neurons’
hold-period responses reflected the direction of eye movement in both blocks, despite
the differences in absolute eye position. Thus, the hold-period responses represent
the direction of the just-completed eye movement.
Some neurons responded similarly during the hold period in both the delayed-
saccade and dots tasks, suggesting that hold period activity reflected the previous
saccade, independently of task context (e.g., Fig. 2.16a). However, other neurons’
hold period activity was selective during the dots task but not after a delayed sac-
cade, indicating that some cells reflected the previous saccade only when it was the
consequence of a choice, rather than an instruction (Fig. 2.16b).
56
Saccade
x1
x3
x2
Saccade
Saccade
+ 700 ms
a b
x1
x3
x2
Saccade
+ 700 ms
Figure 2.17: Hold period effect in population activity. (a) Average hold periodtrajectories for T1 and T2. Dots on trajectories are placed at 50 ms intervals. (b)Individual and average hold trajectories for T1 and T2. 20 of ∼500 trials are shownfor each target (for ease of viewing). Shading on mean trajectories represents SEM.
In the dots task, we show two state-space views of the population activity during
the hold period. Figure 2.17a shows the average trajectories corresponding to T1 and
T2 choices, from the time of saccade initiation to 700 ms after, the minimum length
of the delay period. Figure 2.17b also shows the mean trajectories, with the standard
error of the mean indicated with shading. 20 single trial trajectories are also shown,
which show trial-to-trial variability. Presumably, this variation is due to a number
of factors which may include coherence of the dots (correlated with confidence that
a reward will be delivered), choice and reward history, among other factors such as
confidence a reward will be delivered, fatigue, the monkey’s degree of satiety and
overall motivation level.
2.3.6 Effects of choice and reward history: subsequent trials
The effects seen during the hold period often extend beyond the end of the trial, and
continue to influence neurons’ firing rates into subsequent trials. Figure 2.18b shows
this effect in the PSTH from a single unit. Responses in this PSTH are split into 4
groups based on the monkey’s saccade direction on the current and previous trials.
Both shades of blue (red) correspond to T1 (T2) choices on the current trial. Light
blue (red) lines correspond to trials in which the monkey’s choice on the current trial
was the same as the previous trial (e.g., light blue (red): T1 (T2) current / T1 (T2)
57
previous). Dark blue (red) lines correspond to trials in which the monkey’s choice on
the current trial was the opposite of the previous trial (e.g., dark blue (red): T1 (T2)
current / T2 (T1) previous).
Before the monkey fixates, this neuron’s responses reflect only the monkey’s choice
on the previous trial. At this point in the trial the monkey has no information
about his upcoming movement, so neurons’ responses should not reflect a movement
plan to either target. However, somewhat surprisingly, neurons such as the one in
Figure 2.18b reflect the outcome of the previous trial fairly robustly. This is notable
because the dots task does not require, or provide any advantage to track previous
trials. This effect persists up until the time the dots turn on, shortly after which the
neuron’s response reflects both the monkey’s previous saccade as well as his upcoming
choice. This effect of choice history persists throughout the trial, though to different
degrees in across the population.
Figure 2.18 shows this effect on a population level. For clarity, only T1 choices
are included in this figure, though the same effects are present for both targets. The
trajectories begin 1000 ms before the fixation point turns on, and represent choices to
T1 (light blue) or T2 (dark blue) targets on the previous trial. When the monkey’s
gaze is focused on the fixation point, the eye position signals should in theory be the
same, regardless of trial history. However, we see that the ellipses which represent
the SEM of the trajectories at the time of fixation are clearly separated, indicating
that saccade history in fact does influence the activity in this population.
Similar effects are seen when trials are sorted by whether the monkey was rewarded
or not on the previous trial. These effects of choice and reward history suggest that
prearcuate cortex is involved not only in choosing and generating eye movements, but
also in tracking previous actions in order to inform upcoming behavior.
58
Fixate
Fixation - 1000 ms(T1 chosen on prev. trial)
PC 1
Fixation - 1000 ms(T2 chosen on prev. trial)
Targets On
Motion On
x2
x1
T1 / T1 choices
Target choice on
current / previous trial
T1 / T2 choices
T2 / T2 choices
T2 / T1 choices
a
b
Figure 2.18: History effects. (a) PSTH for one unit. Responses are broken downby target choice on current and previous trials. Blue/red represent T1/T2 choiceson current trial. Bright color indicates choice on previous trial was the same ascurrent; dark color indicates previous trial choice was to the opposite target. (b)Low-dimensional representation (using SPCA) of the population for T1 choices onthe current trial, where the previous choice was T1 (light blue) or T2 (dark blue).Trajectories begin 1000 ms before fixation was acquired (which sometimes extendsinto hold period of previous trial) and continue until the dots are presented. Ellipsesrepresent SEM at trial events. Trajectories are significantly influenced by targetchoice on previous trial up to the time the dots are presented and sometimes throughthe saccade and hold period of the trial (not shown in population, but visible inPSTH).
59
2.4 Discussion
2.4.1 Bias in population measures of choice predictivity
When recording from single electrodes, experimenters implicitly choose cells based
on their response characteristics. When recording from arrays, this is not an option.
Fixed in place, the electrodes on the arrays pick up signals from subpopulations of
neurons close to the array. These multielectrode recordings yield many cells for which
the target is not in their response field, for which responses to stimuli are weak or
absent, or cells with unusual patterns of response which would typically be excluded
from single-electrode recordings. However, in some contexts it is more important to
have signals from many neurons at once than to have a smaller number of high-quality
single-units (for instance, in neural prosthetics). In this chapter, we show that we can
see many of the same characteristics other groups have demonstrated in single units
in our population, despite the fact that the stimuli are not optimized for the neurons
recorded.
When comparing response properties from multielectrode recordings during the
dots task with the single-electrode recordings described in the literature, should we
screen the neurons, and only include ones with strong predictive indices or which
distinguish between dots of different coherences? This approach would give us the
best circumstances for comparing our neurons with those of LIP (e.g. Shadlen and
Newsome, 2001). In Figure 2.11, this is what we did.
Using the method described in Section 2.2.6, the population ROC predictive
index shows similar dynamics to that of Shadlen and Newsome (2001) in LIP: in
both prearcuate and LIP, decisions are made more rapidly when the motion signal
is stronger. However, Figure 2.13 includes all neurons that differentially modulate
their firing rates during the dots period. In LIP, Shadlen and Newsome recorded
cells for which the target was placed in their response field. Here, the target was not
necessarily in cells’ preferred or null direction, nor did we adjust our analysis based
on individual cells’ response properties or tuning. However, looking at the ROC pre-
dictive index curves in Figure 2.10, it is apparent that averaging might not be the
best way to combine the signals from these neurons. Not only are the signs of these
curves different for T1- and T2-tuned neurons, but the timecourses are different as
well. In other cortical areas where this analysis has been applied (such as LIP), the
60
dynamics of individual neurons’ responses are more similar to each other, and aver-
aging the ROC curves should provide a reasonable view of the average dynamics of
the predictive signal. However, if the goal is to know how well the decision can be
predicted based on population data, averaging ROC curves is not the best approach;
the data should be combined in an intelligent way before finding the ROC curve.
Using a state-space analysis provides a straightforward way to combine the choice
predictive signals from the population, as described in Section 2.3.4. Though our goal
was not to directly compare the numbers from this analysis with the predictive indices
in Figure 2.11, the dynamics look appealingly similar. In order to understand how
these predictive indices relate to distances in this firing rate space, we would like to
see LIP data worked up in the same way. Since this analysis does not require single-
trial data, this should be possible, and would provide a useful basis for comparison
with our state-space methods.
Like Kim and Shadlen (1999), many of the neurons we recorded showed decision-
related activity, that is, their responses during the dots epoch were predictive of
the monkey’s upcoming choice. Our responses were similar to those found in LIP,
which makes sense since the two areas are interconnected (Medalla and Barbas, 2006).
We showed here that we can take a population of all-comers and, with very little
preprocessing or biased selection, we can reproduce some of the same dynamics seen
using conventional methods such as ROC analysis in other areas as well as in this
area. This result shows that our tools allow us to reproduce previous findings using
different techniques, even though our cells are much less stereotyped and were not
selected based on their response properties. This opens the door for other population
studies. The more we develop generalizable analysis methods that do not require
individually-selected neurons, the more intuition we can gain about natural ensembles
of neurons.
The next steps for this experiment would be to (1) focus on single-trial analyses,
to investigate the dynamics of decision-making on single trials, building upon the
state space analyses that we performed with the average trajectory in Section 2.3.4.1,
(2) to look at data from other better characterized areas in this way (e.g., Shadlen
and Newsome, 2001), and in future experiments, (3) to stimulate neuron(s) while
the monkey is making a decision about the direction of motion of the dots, to test
hypotheses about how decision-making evolves through neural state space on single
61
trials.
2.4.2 Dimensionality vs. epoch
During the direction-discrimination task populations of neurons in prearcuate cortex
represent different aspects of the task, during which the underlying dynamics may
be described by different numbers of latent dimensions (e.g., Yu et al., 2009, Figure
2). We wanted to compare how many dimensions were needed to describe responses
during different epochs of the task. Given different firing rates and response proper-
ties during different epochs of the task, the axes of this space describing the latent
dimensions corresponding to the shared variability will also be different. Therefore,
to compare the dimensionality of the data across different epochs we need to measure
the shared variability during different epochs within the same space. To do this, we
ran GPFA on the window of the trial from target onset to reward, which included
the epochs of interest, and used the resulting dimensions to generate the pie charts
in Figure 2.14.
Figure 2.14 shows that the dimensionality/shared variance is different during dif-
ferent epochs of the trial. This is likely is related to distinct patterns of firing rate
variation over different epochs of the trial. We need more dimensions to explain the
same amount of the variance in the hold and saccade epochs of the task than during
the dots period.
Across the prearcuate recordings, we see very small amounts of shared variability.
Why is this? Prearcuate appears to have both lower dimensionality and less shared
variability than PMd. This might be due to more cognitive influences, lower firing
rates, or more heterogeneity in what cells in the prearcuate population represent. Our
assessment of the underlying variability in prearcuate is consistent with the results of
Machens et al. (2010), who, in the context of a different task (and serial recordings),
found that shared variability in PFC is captured by about 6 dimensions.
One assumption we have made in employing dimensionality reduction to investi-
gate the underlying dynamics of decision-making and movement preparation is that
the neurons we record are all participating in the same process. It could be the case
that we are sampling from two distinct populations of neurons involved in movement
preparation, and by combining them we obscure our ability to learn about the pro-
cess driving either subpopulation. An important future analysis will be to repeat the
62
host of analyses described in this chapter, after splitting up prearcuate neurons into
subpopulations, based on anatomy and response properties. In the context of a tac-
tile frequency-discrimination task, Machens et al. (2010) found that while individual
neurons response properties did not cluster into distinct groups, the dimensions of
variance in the population corresponded to two task parameters, elapsed time and
frequency of the first stimulus. In the dots task in prearcuate, do different dimensions
also represent different task variables (e.g., eye movements or choice predictivity)?
At this point, our results in prearcuate are intriguing, but still introduce more
questions than they answer. Future work will help us interpret the significance of the
different dimensionalities across tasks, actions, and systems.
2.4.3 History effects: rethinking the idea of ‘baseline’
Many studies show evidence for an effect by comparing cells’ firing rates during their
epoch of interest with firing rates during the ‘baseline’ period, after fixation but
before target onset. However, in Section 2.3.6, we provided evidence that activity
throughout the trial can carry strong influences of previous choices (or rewards, data
not shown). Our task does not require the monkey to remember what he has done
in the past, nor would this information help him with the task. Nevertheless, the
population carries a strong influence of what happened in the past. Similarly, Dorris
et al. (2000) demonstrated that neural activity of superior colliculus neurons can be
significantly influenced by previous saccades. In PFC, Genovesio et al. (2006) found
that many neurons in PFC encode elapsed time in the context of a delayed saccade
task. It is plausible that neurons that track time within a trial may also reflect
the passage of time on longer timescales. Our results are also consistent with those
of Hasegawa et al. (2000), who showed that activity of prefrontal neurons reflected
past or future performance much more than performance on the current trial of their
oculomotor task.
Our data showed that choice and reward history can have a significant influence
on the next trial. Likewise, assessments of baseline activity in this area are likely to be
biased. In this experiment, subjects have no reason to keep track of what happened
in the past, yet neural activity and behavior show influences of previous actions and
rewards. This is consistent with this region of cortex being involved in short-term or
working memory, as described in Fecteau and Munoz (2003).
63
One challenge we faced was that our experiment included many different condi-
tions (e.g., two possible target locations, 5-7 different coherence levels), which made
it difficult to see significant effects after sorting trials by each of these conditions.
Future experiments could use a smaller number of task conditions to more fully ex-
plore effects of history. However, some of these effects might only come about in more
cognitively engaging contexts. Further work is needed.
2.5 Summary
At the beginning of this chapter, we set out to characterize response properties from
an ensemble of prearcuate neurons, both on the single neuron and population levels,
and averaged across trials as well as on single trials.
We demonstrated that neurons in prearcuate have diverse responses, represent-
ing visual input, decision-related signals, movement preparation, traces of where the
monkey directed his saccade both in the same trial and on the previous trial, and
also whether or not he received a reward on the trial before. These history effects are
present at the population level for both the delayed saccade and the decision-making
task. We compared the choice predictivity of neurons in prearcuate with Shadlen
and Newsome’s measurements of choice predictivity in LIP (Shadlen and Newsome,
2001). Although individual neurons in prearcuate showed a wider range of response
profiles in their PSTHs and ROC predictive index timecourses, the dynamics of choice
predictivity in the populations as a whole looked encouragingly similar.
Our second goal was to combine information from individual cells to investigate
the dynamics of the population response. We used dimensionality reduction to vi-
sualize the average neural trajectories in a state space constructed from single-trial
population responses. During the dots presentation, we found that the average trajec-
tories corresponding to eventual T1/T2 choices separated sooner for easier decisions,
consistent with what we saw in the ROC analysis. We also compared the dimensional-
ity of different epochs of the task, showing that different numbers of latent dimensions
are needed to capture the majority of the shared variance of the population. With
respect to task history, we demonstrated that the single-neuron effects we see are very
strongly represented within the population.
Finally, we wanted to learn what we could discover about neural dynamics that
64
was not possible with single-unit recording. The methods we used in this chapter
confirm that state-space approaches allow us to visualize neural dynamics, on average.
In Chapter 3, we look at neural dynamics on single trials during the perimovement
epoch of the task.
Chapter 3
Population activity in prearcuate
cortex accounts for variance in
saccadic latencies
3.1 Chapter Intro
In Chapter 2, we used dimensionality reduction to visualize dynamics and to examine
properties of these dynamics on single trials. We found that various metrics of these
trajectories differed depending on stimulus properties, or how hard the monkey’s task
was. In this chapter, we investigate the relationship between these trajectories and
behavior. The specific behavior we have examined is saccadic reaction time, and our
goal is to determine whether movement preparation activity in the prearcuate gyrus
predicts reaction times. The relationship between neural activity and reaction times
has been examined extensively in FEF, where activity of individual cells has been
shown to correlate with velocity, reaction time, and other saccade metrics. Whether
activity in prearcuate is also tightly tied to these parameters is not known. Within
the rich dataset described in Chapter 2, this chapter will focus on the preparatory
neural activity around the time of the go cue.
65
66
3.2 Background & Motivation
In order to make an eye (or arm) movement, neural circuits first prepare and then
initiate the movement. In an instructed delay task, the preparatory activity can be
studied in isolation between the times the target appears and the presentation of a
go cue. After the go cue, the preparatory activity changes so as to initiate the move-
ment. When no instructed delay period is present, it is believed that neural circuits
still briefly prepare the movement and then initiate the movement (e.g., Crammond
and Kalaska, 2000). A target for an eye (or arm) movement is simply presented,
and the movement can begin as soon as the subject is ready. This type of reaction
time task has been used in the oculomotor system by Hanes and Schall (1996) and
others to study the neural correlates of movement initiation. In this chapter, we use
an instructed delay task to focus on movement preparation, the period just before
movement initiation.
Movement preparation allows the rapid and accurate execution of voluntary move-
ments, and can be influenced by factors that may change from moment to moment,
such as attention and differences in stimulus properties. Consequently, movement
preparation unfolds differently across many different repetitions of the same move-
ment. By looking at individual trials, our goal is to investigate the effect of prepara-
tory neural activity on saccade execution.
Simultaneous recording from populations of neurons allows dynamics of movement
preparation to be estimated on single trials. During behavior, the brain presumably
combines input from populations of cells with wide ranges of response field positions
relative to the upcoming saccade. However, in the oculomotor system this relationship
has only been studied in individual cells with saccade targets in their response field.
To study the dynamics of movement preparation, we asked: how does variation in
reaction times relate to trial-to-trial variation in ensemble activity, from neurons with
heterogeneous response fields?
Prearcuate cortex is the cortical area we investigated to address this question.
We define prearcuate cortex as the cortical surface on the rostral bank of the arcuate
sulcus, between the frontal eye field (FEF) and the principal sulcus (Fig. 2.2). This
may include any part of FEF on the surface of the cortex, as well as area 8Ar (called
area 8Ar by Schall 1997; Preuss and Goldman-Rakic 1991; 8r or r8 by Gerbella et al.
2007; Medalla and Barbas 2006) and possibly area 46. This area lies between FEF, a
67
patently oculomotor area, and the principal sulcus, which contains areas involved in
visuomotor memory (Levy and Goldman-Rakic, 2000; Romanski, 2004), and cognitive
processing (Fuster, 1973; Goldman-Rakic, 1987; Funahashi et al., 1990; Zaksas and
Pasternak, 2006). Prearcuate cortex has fewer projections to brainstem nuclei than
FEF (Gerbella et al., 2007; Schall, 1995); however, neural responses in this area are
clearly involved in movement preparation (Seidemann et al., 2002; Moschovakis et al.,
2004).
We chose prearcuate cortex for (1) its saccade-related activity and the ability to
evoke eye movements with stimulating current (Seidemann et al., 2002), (2) its role
in cognitive processing and visuomotor memory, and tight correlation with behavior
(Kim and Shadlen, 1999), (3) its anatomical connections to LIP and other visual
areas (V2, V3, V4, MT, TEO) (Schall, 1995), and (4) its location on the surface of
the brain, which allows us to use multielectrode arrays to record from many cells at
once.
However, despite the wealth of signal types in prearcuate, the eye-movement re-
lated activity itself has not been well-studied. Is the state of movement preparation
in prearcuate cortex, as reflected by ensemble activity, tightly linked to parameters
of eye movements?
To ask this, we recorded from a heterogeneous population of prearcuate neurons
while monkeys performed a task in which saccades served as the operant response.
We analyzed these recordings in the context of a state-space analysis, where ensemble
activity was represented as a trajectory evolving through a high-dimensional space,
corresponding to cells’ firing rates. Finding a trial-by-trial relationship between vari-
ance in behavior and ensemble activity within this state-space demonstrates how such
analyses can be used to gain leverage on understanding the dynamics of movement
preparation.
Our results demonstrate that movement preparatory neural activity from a di-
verse population can fruitfully predict reaction time. We find that the trial-by-trial
evolution of the system’s state is related to the monkey’s saccadic reaction time, and
that this relationship is stronger when making use of the simultaneity in diverse pop-
ulation of neurons than by looking at correlation between individual cells’ activity
and behavior. These results are consistent with recent findings in PMd (Afshar et
al., 2010), which show similar correlation coefficients between population activity and
68
reach reaction times.
Parts of this work have been published previously in abstract form (Kalmar et al.,
2010).
3.3 Methods
3.3.1 Subjects
Two adult male rhesus monkeys, T and V, were trained to perform a delayed-saccade
task and monkeys T and V also performed a two-alternative, forced-choice motion
discrimination task, in which saccades served as the operant response. After initial
training, we implanted a 96-channel silicon electrode array (Blackrock Microsystems,
Salt Lake City, UT), as described in Churchland et al., 2006; Hatsopoulos et al., 2004.
More details can be found in Section 2.2.1.
3.3.2 Task design and training
Monkeys were trained on the direction discrimination task described in Section 2.2.2,
but the discrimination per se will not be important for the analyses described in
this chapter. We used these trials as a source of eye movement behavioral data and
neural data related to eye movement planning. This task and the epoch of interest
are illustrated in Figure 3.1.
For the data reported in this chapter, we focused on the window of time around
the go cue and saccade. The full data set analyzed in this chapter consists of 45
experiment sessions from monkey T and 35 sessions from monkey V.
3.3.3 Procedures
Monkeys’ behavior was closely monitored and enforced by computer control. Behav-
ioral and neural signals were acquired by REX and Cerebus systems. Spikes were
sorted and analyzed offline (see Section 2.2.3 for details).
69
Fixate Targets On Motion On Delay Period Go Cue Saccade
Figure 3.1: Behavioral task. Monkeys were trained to make a perceptual discrimina-tion about the direction of motion in a patch of randomly moving dots, and to saccadeto the target in this direction. The presentation of the dots stimulus was followedby a delay period (∼1.5 sec) before the go cue was given. We analyzed perisaccadicactivity 500 ms before and after the go cue. This epoch included the monkeys’ sac-cades, as shown by the highlighted region. Saccade latencies were typically 150 - 300ms.
3.3.4 Array locations and neural recordings
Arrays were implanted into the surface of cortex, in the arcuate concavity between
the caudal aspect of the principal sulcus and the anterior bank of the arcuate sulcus,
as estimated visually from local anatomical landmarks (Fig. 2.2). Spike-sorting and
tuning criteria are described in Section 2.2.4.
3.3.5 Datasets
We analyzed 90/70 datasets from 2 monkeys (subjects T and V), from 45/35 days of
recording (monkey T/V). Comparison figures from PMd (Afshar et al., 2010) were
drawn from 5 days of recording in monkey G and 2 days of recording in monkey H.
Data were kindly supplied by Afshar et al. (2010). Our recordings sampled similar
subsets of units day-to-day; however, populations of units were not assumed to be
independent nor identical across days (Chestek et al., 2007; Batista et al., 2007).
Use of different task parameters on consecutive days of recording reduced day-to-day
redundancy of signals from similar populations of units. Data here are sampled from
datasets which represent experimental sessions with the highest trial counts and best
signals from the array.
70
3.3.6 Behavior
We measured monkeys’ latency to saccade, or reaction time (RT) as the time between
the delivery of the go cue and saccade initiation. Saccade start and end times were
calculated as the time the derivative of the analog eye position first exceeded a velocity
threshold of 15◦/sec and decreased below 10◦/sec. Trials on which saccade velocity
profiles were not Gaussian within a reasonable goodness of fit were excluded.
Monkeys’ RTs were brisk, typically between 150 and 300ms, and varied with the
length of the delay period This is consistent with what would be expected if the
subjects were planning during the delay period (e.g. Figure 1.1, and Churchland et
al. 2006). Trials with RTs greater than two standard deviations from the mean were
excluded. This comprised a very small percentage of trials. Since we are using brisk
RTs as an indicator that the monkey was planning and our objective was to study
neural activity related to this process, we did not want to include trials where there
was any behavioral evidence that the monkey was not planning.
3.3.7 Dimensionality reduction of neural data
GPFA methods are described briefly in Section 2.2.5 and in detail in Yu et al. (2009).
Figure 3.3 shows neural trajectories in two dimensions, focusing on the time between
when the monkey is given the go cue (green dots) and when the monkey initiates (blue
dots) and executes his saccades. To focus in on the epoch most related to movement
preparation, neural data were aligned to the time of the go cue and the trajectories
plotted as a function of time in each of the orthonormalized dimensions (Fig. 3.11).
3.3.8 Analysis and presentation
Unless otherwise noted, all data visualization was done by reducing the dimensionality
of the data, and all calculations were done in the full-dimensional space.
71
3.4 Results
3.4.1 Diverse neural responses in prearcuate
Our units’ response profiles were diverse in two ways: (1) location of response fields
with respect to the target location, and (2) timecourses and patterns of response,
compared to typical eye-movement responsive cells in FEF or LIP.
Location of RFs. Because we we could not move our electrodes, we had access
to only the population of units under our array. The majority of units’ response
fields were in the contralateral hemifield, but we could not choose target locations to
match all units’ response fields. Though we only included in this analysis units with
stronger responses for one of the two targets present, the target is not likely in the
center of most of the units’ response fields. From day to day, we varied our target
locations to sample different subpopulations of units. However, on any given day,
we recorded from a population of units with varied response field locations, relative
to the target. Units’ preferred directions tended to be in the contralateral hemifield,
though target and saccade-related responses were usually seen for ipsiversive saccades
as well (though often weaker), as illustrated in the previous chapter (Figure 2.7). A
minority of units preferred the ipsilateral hemifield (data not shown). However, given
that we did not optimize target locations based on individual units’ responses, these
mappings of preferred direction are very broad.
Description of responses. In Figure 3.2 we show an example of the diversity of
the average responses collected simultaneously on the array, during one recording
session. We see units with various combinations of activity that is modulated during
target onset (visually-modulated), the dots presentation and subsequent delay period
(decision-related), around the time of the eye movement (saccade-related), and during
the hold period before the delivery of the reward (choice/action-related). For example,
the top right panel in Figure 3.2 shows a unit with a brisk response to the target onset,
but modulates its responses much less until right after the monkey completes saccades
to either target. In contrast, the unit in the upper left panel is more modulated during
the dots epoch of the trial, and its perisaccadic response is much more gradual. This
unit appears to be more involved with the decision about the direction of dot motion
than the visual or saccadic response per se. Many units in this region respond most
vigorously in the epoch of the trial after the saccade to the target and before receiving
72
the reward, like the unit in the center panel of Fig. 3.2 (Reppas et al., 2010).
Diversity of PSTHs. In addition to recording from units whose RFs were not
lined up with the target location, we also recorded from units with a variety of
target-RF alignments, with corresponding diversity in PSTHs. Some units increased
their firing rates briskly directly preceding a saccade, similarly to motoric cells in FEF
(e.g. top right panel, Fig. 3.2). Other units increased their activity before a saccade
more gradually, like the top two left-hand panels in Fig. 3.2. Still another group of
units decreased their activity before a saccade, but differentially for different target
locations, as in the center and right panels in the bottom row of Fig. 3.2. Many trends
were seen during the perisaccadic epoch. We included all units that were tuned in
the 300 ms epoch before a saccade (as defined in Methods), which included units of
each type mentioned here (units in Fig. 3.2 were chosen to illustrate diversity of cells
recorded on the array, but were not necessarily included in perimovement analyses).
Each of these units show different relationships with behavior, despite the fact that
they were recorded simultaneously. How can we describe the relationship between
this heterogeneous population and behavior?
3.4.2 Neural trajectories evolve along a stereotyped path
Reducing the dimensionality of the data using GPFA, we can visualize how the en-
semble activity evolves over the course of individual trials. Figure 3.3 shows neural
trajectories during the perimovement epoch of the trial, in 2 dimensions. Figure 3.3a
shows data from many single trials to the same target in prearcuate, from a monkey
performing the saccade task; Figure 3.3b shows data from PMd in a different monkey,
performing a delayed reach task (e.g., Fig. 4.10). In this figure, each gray line is a
neural trajectory for a different trial, all representing saccades/reaches to the same
target. These trajectories illustrate the evolution of the population state from 100 ms
before the go cue (green dots) until 100 ms after the end of the saccade (Fig. 3.3a)
or reach (Fig. 3.3b) (movement onset shown with blue dots). While individual trials
evolve differently, they tend to follow a stereotyped path as the subject prepares and
executes his movement. The stereotyped path has less trial-to-trial variability in PMd
than in prearcuate. Possible reasons for this will be discussed later. However, the
primary effect is similar across systems: for each target location, neural trajectories
follow a stereotyped path.
73
2
22
10
40
5
25
5
45
0
14
Fir
ing
Ra
te (
sp
ike
s/s
)
−0.5 0 0.5 1
5
45
−0.5 0 −0.5 0 0.5 1 −0.5 0 −0.5 0 0.5 1
6
22
−0.5 0
dots dots dotssaccade saccade saccade
targets
on Time (s)
T1 choices
T2 choices
5
30
6
24
Figure 3.2: Peri-stimulus time histograms (PSTHs) for prearcuate neurons simulta-neously recorded during the dots task. Examples are representative of diversity ofneural responses recorded on array. Activity is shown during target onset and dotsepoch to allow comparison with PSTHs from other oculomotor areas. Visuomotor andmotoric neurons show response profiles similar to those of similar neurons in FEF.Note that some neurons increase their firing rates leading up to a saccade; othersdecrease their firing rates leading up to a saccade.
Visualizing movement preparation trajectories allows us to gain intuition about
the dynamics of the population on single trials. Figure 3.3 suggests that there is a
common path along which the neural state moves in order to initiate and execute
movements. In addition to being a convenient visualization, do these trajectories
actually reflect dynamics in the underlying system relevant to movement preparation?
The best way we have of answering this question is to try to relate the dynamics of
these trajectories to behavior. On single trials, does the state of the population tell
us about how quickly the animal will execute its movement? One might consider the
stereotyped path in Figure 3.3 as being like a race track, with the neural trajectories
as runners trying to get from the starting block (go cue) to the finish line (movement
74
Go cueMovement onset
x1
x2
x1
x2
a b
Figure 3.3: Neural trajectories for a single saccade/reach target in (a) prearcuatecortex and (b) PMd evolve along a stereotyped path. For this visualization, only theperimovement epoch of the trial is shown (100 ms before the go cue, to 100 ms afterthe end of movement).
onset). In this analogy, trying to predict subjects’ RT based on features of the
population trajectories would be akin to predicting runners’ race times based on
their starting position relative to the starting block. We want to understand how a
runner’s race time corresponds to how, when, and where he moves on the race track.
To what extent does trial-by-trial variability in race paths or trajectories correspond
to trial-by-trial differences in behavior?
3.4.3 Trial-by-trial relationship between neural dynamics and
RT
Afshar et al. (2010) studied the relationship between neural trajectories within a
high-dimensional state-space and RT, in PMd. They found that the further along the
mean trajectory the ensemble firing rate was on a given trial, the faster the subject’s
reaction time. In the foot race example, this would be akin to runners’ race times
being faster on races when their starting point was closer to where they needed to
run. Presumably there is a limit to how far along this path the neural state can be
without triggering a movement, just as a runner wants to get as close as possible
to the starting line without crossing it. We hypothesize that there is a single-trial
relationship between responses from the neural population in prearcuate and saccadic
initiation times, as has been shown in PMd. Using the same approach in prearcuate
cortex, we investigated the relationship between RT and dynamics of the population
response.
75
x1
x2
x1
x2mean trajectoryacross trials
reference(go cue + t)
mean at
ktrial at reference
projection
go vector
x1
x2
a b cgo cue
go cue
vector
G
M
Figure 3.4: Measuring distance along the mean trajectory. Schematic diagrams showthe mean trajectory in blue/green; green dots represent the state of the system atthe time of the go cue on individual trials. (a) Trajectory for trial i shown in gray.The black arrow represents the difference between the average state of the system atthe go cue and on trial i. On this trial, the state is less far along the mean path thanaverage. (b) Example trajectory for trial j. At the time of the go cue on this trial,the system has progressed further along the mean path than on the average trial,illustrated by the black arrow. (c) Schematic of projection computation. The vectorG from mean go cue to trial k go cue is projected onto the mean path, estimated bya reference vector between the mean go cue and a reference point, vector M. Notethat this schematic is drawn in two dimensions, but is easily extended (though less-easily visualized) to an arbitrary number of dimensions. Unless otherwise noted, theanalyses done in this chapter were done on the full dimensionality of the data.
Figure 3.4 illustrates how we measure the evolution of movement preparation in
the population response. Though this schematic is depicted in two dimensions, the
actual neural trajectories reflect the ensemble activity of 100 units, within a 100-
dimensional space. In Figure 3.3 and in the schematic for Figure 3.4, employing
dimensionality reduction allows us to visualize these high-dimensional neural trajec-
tories within a low-dimensional space. However, the following analysis was done in
the full N-dimensional space, where N is the number of tuned units recorded on a
given day. Unless otherwise noted, all data visualization was done by reducing the
dimensionality of the data, and all calculations were done in the full-dimensional
space.
To evaluate the progress of movement preparation at the time of the go cue,
we assumed that the stereotyped shape of the trajectories in Figure 3.3 represents
the path along which movement preparation progresses. In the same way that a
focused runner will be as close as possible to his ideal starting position when the
starting gun is fired, on trials in which movement preparation is more complete the
state of the system should be further along this path when the go cue is delivered. We
76
approximated this path by computing the mean trajectory across trials with the same
saccade direction. Figure 3.4 describes the process by which we measured distance
along this path. We chose a reference point on the mean trajectory 140 ms after the
go cue, and defined the reference vector as connecting the time of the go cue and the
reference time on the mean trajectory. In Figure 3.4, the hollow arrow represents the
mean trajectory, where the hollow green circle represents the mean population state
at the time of the go cue. The green part of this trajectory is before the go cue,
the blue part is after. We hypothesized that the state of more complete movement
preparation corresponds to being further along the reference vector at the time of the
go cue. For each trial, we then measured the progress of movement preparation by
finding the projection of the trajectory at the time of the go cue onto this reference
vector. Figure 3.4a illustrates a trial where the neural trajectory is behind the mean at
the time of the go cue, corresponding to a long RT. Figure 3.4b shows a trial ahead of
the mean at the time of the go cue, corresponding to a fast RT. Figure 3.4c illustrates
how we measured this. If this hypothesis is supported by our data, we expect to see
that trials with greater projections are also the trials with faster reaction times.
Figure 3.5 supports this hypothesis, demonstrating a systematic relationship be-
tween projection along the mean trajectory and reaction time. The top row of Fig. 3.5
shows these results for one example dataset for each of 4 monkeys. Each panel shows
RT as a function of projection, with the best fit line (using linear regression). The cor-
relation coefficient (inset) and p-value summarize the relationship for each dataset.
The bottom row of panels in Fig. 3.5 shows histograms of these correlation coef-
ficients across datasets. Each point corresponds to one target location during one
experimental session. Dark bars represent correlation coefficients significant with p
<0.05. There is variation across days, some of which we account for in the next
section. Despite this variation, the mean correlation coefficient (monkey T: -1.05e-1
± 9.67e-3, monkey V: -8.41e-2 ± 1.42e-2) is significantly smaller than zero for each
monkey (t-test, p <0.001). Having a large number of datasets was important for
uncovering this effect, as individual datasets showed a wide range of correlation co-
efficients. Our result is consistent across both prearcuate and both PMd monkeys
(data from Afshar et al., 2010).
As a control, we asked how much of the relationship between our distance metric
and RT was due to chance. We shuffled distance along the mean trajectory and
77
150
200
250
300
150
200
250
300
−40 0 40150
200
250
300Reaction time (ms)
−100 0 100150
200
250
300
Projection onto mean path (A.U.)
a b c d
e
r = -0.360
p < 0.001
r = -0.365
p < 0.001
−100 0 100 200−50 0 50
r = -0.471
p < 0.001
r = -0.413
p < 0.001
2
4
6
0
4
8
12
−0.5 0 0.50
−0.5 0 0.5−0.5 0 0.50
4
8
12
Correlation coefficient (r)
Number of datasets
−0.5 0 0.50
4
8
12
16
f g h
*
** *
Figure 3.5: Projection along the mean path is correlated with reaction time. (a-d)Reaction times are faster on trials when the state of the system at the go cue isfurther along the mean trajectory. (a, b) Example prearcuate datasets, monkey T(r = -0.360, p <0.001), monkey V (r = -0.365, p <0.001). (c, d) Example PMddatasets, monkey H (r = -0.471, p <0.001), monkey G (r = -0.413, p <0.001). (e-h)Histograms of the correlation coefficient across multiple datasets and target locations.Dark bars represent correlation coefficients significantly different than zero (p <0.05).The means of these histograms are less than zero (p <0.001, all datasets). Arrowsindicate population mean, asteriks correspond to examples in (a-d). (e)monkey T (N= 90 datasets/conditions), (f) monkey V (N = 70 datasets/conditions), (g) monkeyG (N = 78 datasets/conditions), (h) monkey H (N = 24 datasets/conditions)
reaction times with respect to one another, and recalculated the correlation coefficient.
This was repeated 100 times for each target location. Mean correlation coefficients
were 6.30e-4 ± 5.07e-4 (monkey T) and 5.65e-4 ± 8.80e-4 (monkey V), which are
significantly smaller than the average non-shuffled correlation coefficients (t-test, p
<0.001 for both monkeys). This shows that the neural data explains more reaction
time variance than chance.
These results were calculated using a reference point 140 ms after the go cue,
though a wide range of reference points gave similar results. Figure 3.6 shows cor-
relation coefficients calculated from offsets from 250 ms before the go cue to 250 ms
after. Thin gray lines show results for individual datasets and saccade direction of
the correlation coefficient for each choice of reference point offset. The average and
78
−200 −100 0 100 200−0.4
−0.2
0
0.2
0.4
Reference point offset from go cue (ms)
Co
rre
lati
on
co
eff
icie
nt
(r)
−200 −100 0 100 200
a b
Figure 3.6: Correlation coefficients as a function of reference offsets. Thin graylines correspond to datasets/target locations; thick gray lines show the average andstandard error of the mean. Purple lines show the average and standard error ofcorrelation coefficients significant with p <0.05. (a, b) Monkeys T, V. Analyses inthis chapter were performed with a reference point of 140 ms after the go cue. Resultswere similar within reference points chosen across a range.
standard error of the mean across all saccade directions and experiments is shown
by the thick gray line. The purple line shows the same, for correlation coefficients
that were statistically significant with p <0.05. Arrows mark the reference point
used in Figure 3.5. For offsets around the time of the go cue, the reference vector
captures the direction of the mean trajectory poorly, resulting in a noisier estimate of
distance along the mean path and a smaller correlation coefficient. Note that positive
correlation coefficients for negative reference points show the same effect as negative
correlation coefficients for positive reference points. When the slope of the RT vs.
distance plot is positive, this supports the interpretation that further the neural tra-
jectory has traveled away from the direction of movement initiation, the longer the
RT should be. As in Figure 3.5, the results in Figure 3.6 show that there is a lot of
variability across datasets. However, the relationship between RT and distance along
the mean path is fairly robust to the choice of reference point offset.
We wondered to what extent our effect was driven by increases in firing rate.
Does the mean path simply correspond to each unit increasing its firing rate in its
preferred direction? We performed the following control to address this question. In
the analyses represented in Figure 3.5, the reference vector approximated the direction
in which the neural state needed to move in order to initiate a saccade. The reference
79
vector might correspond to a simultaneous increase in each unit’s firing rate, or to
half the units modulating their firing rates up and the other half modulating them
down. In this analysis, we defined the “rise” vector to correspond to an increase in
the firing rate of each unit tuned for the upcoming saccade. By projecting each trial’s
neural trajectory at the time of the go cue onto this rise vector, we could similarly
measure whether projection in this direction was related to movement preparation.
Following the same steps outlined above, we found correlation coefficients for each
dataset, summarizing how much variance in RT was explained by projection onto the
rise vector. These correlation coefficients supported a significant relationship between
the projection onto the rise vector and RT (t-test, p <0.001 for both monkeys; mean
correlation coefficients, monkey T: -5.48e-2 ± 1.15e-2, monkey V: -5.78e-2 ± 1.35e-2).
The magnitude of this effect was significantly smaller than that of projection onto the
reference vector for monkey T (t-test: p <0.005). For monkey V, this effect was also
smaller than for projections onto the reference vector, but not significantly so (t-test,
p = 0.18). However, projecting neural activity onto the mean path rather than the
rise vector yielded better correlation coefficients in both monkeys. This suggests that
though increasing firing rates do explain part of this effect, focusing on this increase
in firing rates alone would leave out a significant part of the story.
3.4.4 RT correlation varies with a number of factors
What is the source of the variation in the RT correlation coefficients in Figure 3.5?
Across experimental sessions, various factors changed, some under the monkey’s con-
trol (e.g., number of trials), some under our control (e.g., location of the targets),
and some due to sources of variance beyond our control (e.g., signal quality from
the electrode array). Even if we could hold all variables constant, some amount of
variation would still be expected due to intrinsic noisiness in the system. Figure 3.7
breaks down how much of the variation in the correlation coefficients in Figure 3.5
can be accounted for by some of these factors.
On the array, the subpopulation we recorded fluctuated from day-to-day. Since
we could not choose target locations to match response fields of each unit within a
recording session, we varied the location of the target positions across recording ses-
sions. As we moved the targets, this influenced the number of units on the array that
were tuned for the given target. Consequently, the number of units that were tuned
80
and included in this analysis varied from day to day. Fig. 3.7a shows how the correla-
tion coefficient varied when saccade targets were placed at different locations, with 0
degrees representing the ipsilateral hemifield and 180 representing the contralateral.
Unsurprisingly, correlation coefficients tend to be stronger when the target is in the
contralateral hemifield. Fig. 3.7b illustrates that for datasets with greater numbers
of tuned units, the correlation coefficient also tends to be greater in magnitude. The
number of trials performed on a given day also has a weak influence on the correlation
coefficient (Fig. 3.7c), but likely this matters most in having enough data points to
achieve statistical significance for the RT vs. projection plot.
The precise shape of the neural trajectories varies from trial to trial, as shown
in Fig. 3.3. Many factors can influence this shape, including the number of units in
the population, their firing rates and how they are modulated through time, and the
location of the target positions. When the shape of the mean trajectory becomes more
complex (less like a line), evaluating the progress of movement preparation using the
projection method described above becomes more difficult. This is due to the way we
are evaluating progress along the mean path; approximating the mean path with a
vector will be less accurate when the mean path is less linear. To assess how this shape
influenced our ability to estimate progress of movement preparation and its relation
to RT, we calculated the curvature of the mean trajectory. We defined curvature
as the distance between the mean go cue and the reference point along the mean
path, divided by this distance in Euclidean space (Fig. 3.8). Figure 3.7d supports
this interpretation, that correlation coefficients are weaker for mean trajectories that
are more curved.
3.4.5 Does simultaneity matter? Comparing results for single
cells and populations
How important is the simultaneity of the data? We compare the correlation coeffi-
cients for the population trajectories (e.g., Fig. 3.9a) with those of the population
of individual units. For each individual unit, we found its projection along its own
mean firing rate, and plotted RT as a function of those (Fig. 3.9b). As with the
population trajectory, we summarized this relationship by finding the correlation co-
efficient for the units’ projections and the monkey’s RTs. The histogram in Fig. 3.9c
81
0 50 100 150
−0.3
0
0.3
Target location (theta)
Correlation coefficient (r)
0 1 2 3 4
Curvature of mean path
0 1000 2000
Number of trials
0 40 80 120
Number of units
r = 0.317
p < 0.001
r = -0.283
p < 0.001
r = -0.070
p = 0.378
r = 0.215
p < 0.01
a b c d
Figure 3.7: Correlation coefficients of datasets depend on multiple factors: (a) Targetlocation (r = 0.317, p <0.001), (b) Number of tuned units recorded (r = -0.283, p<0.001), (c) Number of trials recorded (r = -0.070, p = 0.378) and (d) Curvature ofmean trajectory (r = 0.215, p <0.01). Trends are similar in both monkeys, combinedhere for greater statistical power.
x1
x2
x1
x2
a b
x1
x2
c
Low curvature Medium curvature High curvature
Figure 3.8: Measuring curvature. (a) Example of a mean trajectory with low cur-vature. (b) Example of a mean trajectory with high curvature. (c) Curvature wascalculated by dividing the distance along the path (gray) by the Euclidean distancebetween the mean trajectory at the time of the go cue and at the reference time (reddashed line).
shows the distribution of correlation coefficients for the population of individual units.
The correlation coefficient of the simultaneous population projection from Fig. 3.9a
is indicated with the dashed line. In this dataset, using the population trajectory
explained more of the RT variance (r = -0.365) than the mean of the individual units
(r = -0.038), and more than the best unit (r = -0.312). Repeating this analysis using
simultaneous responses from the 33 best individual units, the correlation coefficient
was -0.201, which is better than the mean, but still not as good as the full population,
or two of the best units individually. This indicates that including responses of many
82
−40 0 40 80150
200
250
300
−100 0 100150
200
250
300
−0.2 0 0.20
5
10
15
20
25
30
Re
ac
tio
n t
ime
(m
s)
Nu
mb
er
of
un
its
Correlation coefficient (r) Projection (A.U.)
−50 0 50 100150
200
250
300
a b
c d
r = -0.365
p < 0.001r = -0.098
p < 0.05
r = -0.201
p < 0.001
All Units Unit 236
Best Units
Projection (A.U.)Projection (A.U.)
Re
ac
tio
n t
ime
(m
s)
Figure 3.9: Does simultaneity matter? Trials with larger projections along the meantrajectory have faster RTs, for the population and for individual cells. Figure showsexample data from one dataset from monkey V. (a) Projection vs. RT, using all cellssimultaneously to estimate state of system and mean trajectory. (b) Projection vs.RT, using one example cell’s firing rate to estimate the state of the system and themean trajectory. Correlation coefficients were collected for each cell individually. Ahistogram of these values is shown in (c). Blue bars show correlation coefficients sig-nificantly different than zero (p <0.05). Dashed line: correlation coefficient from (a).(d) The analysis was repeated for the subset of neurons with correlation coefficientsin shaded blue region, taken simultaneously. The relationship between neural activityand reaction time was strongest when using all cells simultaneously.
individual units is better than just looking at a subset of the best ones.
For each dataset and target location, we calculated the relative number of individ-
ual units with correlation coefficients greater than the population. For most datasets
in both monkeys, the correlation coefficient from the population was better than most
of the units. Figure 3.10 shows the percent of individual units in each dataset with
correlation coefficients better than the population (for populations with significant
correlation coefficients). In Subject T, at most 12% of individual units did better
than the population. In Subject V, no more than 5% of units did better than the
population, with two exceptions. In these two datasets, the mean trajectories have a
83
0 10 200
4
8
12
16
% of units/dataset that outperform the population
Nu
mb
er
of
da
tas
ets
0 100
5
10
15
90 100/ /
a b
Figure 3.10: Simultaneity improves correlation with RT. Histograms show per-centage of neurons with stronger correlation coefficient than population fordatasets/conditions with significant correlation coefficients (p <0.05). (a) monkeyT (N = 49 datasets/conditions), (b) monkey V (N = 26 datasets/conditions).
high amount of curvature (see Fig. 3.7d and Fig. 3.8) when a reference offset of 140
ms is used. When reference offsets are optimized for the shape of the mean trajectory
in each dataset, such artifacts due to curvature can more often be avoided.
3.4.6 Dimensionality of neural data
In the previous section, we described how the correlation between population activity
and RT compares to that of individual units, and how many units contribute to this
result. Another way to look at what is driving this relationship is to look at the
contributions of factors, rather than individual units. In Figure 3.3, we visualized
how activity of the population formed a neural trajectory as it evolved through 3
dimensions of state space. To gain intuition about how activity of the population
evolves in each of the latent dimensions found using GPFA, we can plot population
activity projected into each latent dimension individually. In Figure 3.11, each panel
shows the timecourse of GPFA projections in a different latent dimension, for T1
choices. As in Figure 3.3, green dots correspond to the time the go cue was given,
and blue dots represent the state of the system at the time of saccade initiation.
(For illustrative purposes, only 10 trials are shown. These trials span the recording
session.) In contrast to (Yu et al., 2009), the structure in the timecourses of these
factors is visible for only the first 5 dimensions, unlike the 10 dimensions in Yu et al.
We used the same raw data as used in Yu et al. and repeated our analyses using the
84
same amount of data as in prearcuate (500 ms before/after the go cue) and found the
same results. Thus, differences in the dimensionality of PMd and prearcuate cannot
be explained by differences in the amount of data.
The trajectories in Figure 3.11 represent only part of the variability captured by
GPFA. GPFA distinguishes between private variability, which is unique to individual
units and includes factors such as spiking noise, and shared variability, which describes
the covariations of the population. The goal of GPFA is to detect shared variance
in the population activity and describe factors that capture the shared variance.
The power of GPFA is limited by how much shared variance actually exists in the
population we examine. The shared variability is the aspect that reflects the state of
the neural system. Fig. 3.12a shows, for each example dataset and across all target
conditions, the proportion of the total variance in the population captured by private
and shared variance. Fig. 3.12b shows what proportion of the shared variance shown
in a is accounted for by each factor. As shown in Fig. 3.11, the first 2 dimensions
account for the majority of the shared variance. This supports the observation made
in Figure 3.11, that the dimensionality of movement preparation is smaller than in
PMd.
3.5 Discussion
Individual neurons in different areas of the brain reflect movement preparation in
different ways, but little is known about how these diverse responses fit together as
part of a larger dynamical system. In the rise to threshold hypothesis of Hanes and
Schall (1996), the rate at which neural activity increases during movement initiation
(after the go cue in a reaction time task) predicts how quickly subjects initiate a
saccade. In instructed delay tasks, movement preparation and movement initiation
are separated by a go cue, allowing these processes to be studied independently. A
natural extension of the rise to threshold hypothesis for movement preparation would
be that the higher a neuron’s firing rate is at the time of the go cue, the faster a subject
initiates his movement. In the context of arm movement tasks, Riehle and Requin
(1993) and Bastian et al. (2003) found evidence that neurons in M1 and premotor
cortex support this hypothesis (but see Crammond and Kalaska, 2000; Churchland
et al., 2006).
85
x 2, :
x 7, :
x 12 , :
x 3, :
x 8, :
x 13 , :
x 4, :
x 9, :
x 14 , :
x 5, :
x 10, :
x 15 , :
500 ms
−4
0
4x 1, :
−4
0
4x 6, :
−4
0
4x 11 , :
La
ten
t d
ime
ns
ion
Prearcuate (Monkey T)
Figure 3.11: Time-course of GPFA projections for T1 choices, ordered by relativecontribution. In contrast to Yu et al., 2009, only the first 4 dimensions are needed toexplain 75% of the variance.
Figure 3.13a illustrates this extended version of the rise to threshold hypothesis,
where the axes correspond to firing rates of individual neurons (3 axes shown; actual
dimensionality would be higher, corresponding to all relevant neurons). In this figure,
firing rates for each neuron increase during the delay period, before the go cue (green
part of the trajectory; green circle is the go cue). After the go cue, firing rates continue
increasing until the movement is initiated (blue part of trajectory; blue circle is the
average time of movement initiation). In this extended rise to threshold hypothesis,
trials with higher population firing rates correspond to short RTs as illustrated in
Figure 3.13a.
Also using an instructed delay reaching task, Churchland et al. (2006) examined
the relationship between firing rates in PMd and monkeys’ RTs. They did not find
a consistent relationship between higher firing rates and shorter RTs. Rather, they
found that RTs were shortest when firing rates were close to their mean, falling
within an ‘optimal subspace’ (as illustrated in Figure 3.13b). In this figure, the mean
86
a
b
Prearcuate PMd
Monkey T Monkey V Monkey G Monkey H
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
x12
x13
x14
x15
private
shared
source of
variance
variance
accounted for
by dimension x
Figure 3.12: (a) Relative proportions of private variance (gray) vs. shared variance(black) captured by GPFA. The average percent shared variance captured by GPFAfor each monkey was (from left to right): 12.3, 13.2, 37.9 and 36.2%. (b) Relativeproportions of shared variance accounted for by each dimension. One example datasetis shown for each monkey, which includes data at all target locations.
trajectory is arbitrarily depicted as curving downward, but any shape of the mean
trajectory would be compatible with this hypothesis. The important distinction from
the extended rise to threshold hypothesis is that the optimal subspace hypothesis
predicts that trials in which firing rates are closest to their mean, rather than trials
with high firing rates, have shorter RTs.
Building on the optimal subspace hypothesis of Churchland et al. (2006), Afshar et
al. (2010) also described a relationship between firing rates and RT in which elevated
firing rates do not necessarily predict a short RT. Afshar et al. (2010) demonstrated
that while RTs are fastest on trials when PMd population activity is close to its
mean, that there is also structure within this optimal subspace. In Churchland et al.’s
optimal subspace hypothesis, the only factor that affects RT on a given trial is how
close the neural trajectory is to the heart of the optimal subspace. In contrast, in the
extended rise-to-threshold hypothesis, the only relevant factor is how quickly units’
firing rates increase before an upcoming saccade. Afshar et al.’s initial condition
hypothesis (Fig. 3.13c) draws on both of these concepts, and demonstrate that in
general RTs are faster on trials when the population activity is close to its cross-trial
87
a
Rise to threshold
hypothesis (extended)
b c
Optimal subspace
hypothesis
Initial condition
hypothesis
Long RTs
Short RTs
Long RTs
Short RTsLong RTs
Short RTs
x1
x2
x3
x1
x2
x3
x1
x2
x3
Figure 3.13: Hypotheses relating neural preparatory activity to reaction time. Thethree axes represent firing rates of three (of many) neurons, and the a schematicof the average neural trajectory is depicted by the green/blue arrow. Green (blue)circles represent the average neural trajectory at the time of the go cue (averagetime of movement initiation), and the arrow indicates the direction corresponding tomovement execution. Green (red) dots represent the state of the neural trajectoryat the time of the go cue on trials with short (long) RTs. (a) The rise to thresholdhypothesis, extended to preparatory activity in population activity. This hypothesispredicts that trials in which neural trajectories are further along the mean path (in thedirection of increasing firing rates) at the time of the go cue will have correspondinglyshort RTs. (b) Optimal subspace hypothesis, from Churchland et al. (2006). Thishypothesis predicts that trials in which neural trajectories are close to their averageat the time of the go cue will correspond to trials with the shortest RTs. In thishypothesis, the path of the mean trajectory need not correspond to increasing firingrates. (c) Initial condition hypothesis, from Afshar et al. (2010). This hypothesispredicts that trials in which neural trajectories are furthest along the mean path atthe time of the go cue will have correspondingly short RTs. This hypothesis is similarto the extended version of the rise to threshold hypothesis shown here, except thedirection of the mean trajectory need not correspond to increasing firing rates.
mean. However, within the optimal subspace, short RTs occur on trials in which
firing rates are closest to the region of the state space corresponding to movement
initiation. As in Figure 3.13b, the path of the mean trajectory within this space is
unimportant. The relevant factor is how far in the direction of movement preparation
neural activity has progressed, at the time of the go cue.
Here, we bring the initial condition hypothesis to the oculomotor system. How
does neural activity in prearcuate cortex during saccades fit into this framework?
Our results suggest that, despite differences between the reach system and data from
PMd, that the optimal-subspace model and Afshar et al.’s extension of it also explain
88
the dynamics of movement preparation in prearcuate cortex. Similar to Afshar et al.,
we were able to explain a significant amount of the across-trial variance in subjects’
RTs with trial-to-trial differences in ensemble activity. However, here we also: (1)
recorded from an oculomotor area with heterogeneous activity with respect to task
variables neurons encode, including cognitive factors, (2) characterized an area that
was much less explored, (3) identified several factors explaining variance in corre-
lation coefficients, (4) showed how population results compared to individual units,
(5) studied the dimensionality of response, and (6) compared results and visualized
trajectories across areas.
It is enticing, and perhaps unexpected, that the initial condition hypothesis ex-
plains dynamics of movement preparation in prearcuate cortex. However, caution
should be used in interpreting these results in the context of other models and stud-
ies in this area, especially in its implications for the validity of the rise-to-threshold
hypothesis. In confirming that the initial condition hypothesis describes our data,
this does not exclude the possibility that the rise-to-threshold hypothesis also ex-
plains movement preparation in prearcuate. The primary distinction between the
rise-to-threshold and initial condition hypotheses is that in the former, short RTs
always coincide with trials in which neurons have elevated firing rates at the time
of the go cue. In the initial condition hypothesis, short RTs correspond to trials in
which neurons’ firing rates are also extreme with respect to their averages, but the
sign may be positive or negative, depending on whether the cell increases or decreases
its firing rate before a saccade (e.g., units in Fig. 3.2).
On a neuron-by-neuron level, we can also inspect our units with respect to saccade
initiation, as in Hanes and Schall’s version of the rise-to-threshold hypothesis (with
the caveat that task structure is different). Doing this, in fact we do see many
individual units in this area that look very similar to those reported in (Hanes and
Schall, 1996) (e.g., Fig. 3.14). Are these units, as a population, better described by
the extended rise-to-threshold or the initial condition hypothesis?
We compared whether the extended rise-to-threshold or initial condition hypoth-
esis better explained the relationship between RT and trial-to-trial activity in our
population. We found that when we projected ensemble activity at the time of the go
cue onto the rise vector (described in Section 3.4.3), there was a significant correlation
with RT. However, these correlation coefficients were weaker than when projecting
89
160 200 240 280
0.2
0.3
0.4
0.5
Reaction time (ms)
Ra
te o
f g
row
th (
sp
ike
s/s
)
Fir
ing
Ra
te (
sp
ike
s/s
)
Time from saccade onset (ms)
10
35
−300 −200 −100 0 100
2
a b
c d
Figure 3.14: (a) Example FEF neuron. (from Hanes and Schall, 1996. Reprintedwith permission from AAAS.) (b) Slope of firing rate increase vs. RT. Slope of fit toactivation functions: 0.6 - 1.8 spikes/sec2 (r2 = 0.87 - 0.99). Mean slope = 1.25 ±
0.1 spikes/sec2. Rate of growth of the activation function decreased significantly withincreasing reaction time. 22/25 cells showed the same trend. (c) Example prearcuateneuron (V2007-08-08, unit 202). (d) Slope of fit to activation functions: 0.20 - 0.48spikes/sec2 (r2 = 0.72 - 0.98). (slope = -0.26 - 1.01 spikes/sec2 on individual trials).21/27 cells with significant trends were negative.
this activity onto the reference vector. This control demonstrated that though our
data could be explained by the extended rise-to-threshold hypothesis, they were better
explained by the initial condition hypothesis.
3.5.1 Saccadic preparation in prearcuate
A number of studies have explored the cortical surface between the principal sulcus
and the arcuate sulcus (reviewed by Schall, 1997), but little physiology has been
done to systematically characterize the eye-movement responses in the area of the
arcuate concavity rostral to the frontal eye field (FEF). The higher density of layer V
pyramidal neurons (Stanton et al., 1989) and lower current needed to evoke saccadic
eye movements from FEF (Bruce et al., 1985) have motivated researchers to focus on
the caudal aspect of the arcuate concavity. However, recent studies by Moschovakis
et al. (2004) and Seidemann et al. (2002) suggest that the oculomotor region of the
90
arcuate concavity extends beyond the region functionally defined as FEF, including
the area defined by Preuss and Goldman-Rakic (1991) as 8Ar.
Our objective was to investigate how the state of a system involved in eye move-
ments relates to behavior, on single trials. Approximating the system’s state on single
trials requires simultaneous recording from many units. In theory, this simultaneous
recording could be done in any oculomotor area, including FEF. In practice, technical
limitations currently restrict this kind of recording to the cortical surface. By target-
ing the anterior lip of the arcuate sulcus and the adjacent area on the arcuate gyrus,
we were able to record from a population of units in the oculomotor region described
by Moschovakis et al. (2004) and Seidemann et al. (2002), straddling the border of
FEF and 8Ar. Due to the nature of our electrode arrays, we were not able to pass
stimulating current to elicit eye movements. Based on comparison with anatomical
and microstimulation studies, we estimate that our arrays were on the boundary of
FEF and 8Ar.
Our recordings from prearcuate yielded impressive heterogeneity in response types,
as illustrated in Fig. 3.2. Given the nature of array recordings, stimuli and behav-
ioral tasks were not tailored for individual neurons, but rather chosen so as to yield
reasonable responses from as many neurons as possible. Rather than catalogue and
characterize the units individually, we chose to analyze the responses at a popula-
tion level. We wanted to see what the system was doing, rather than each of the
components individually. What we found was that we can make better predictions
about behavior by using the simultaneity of the neural data. Drawing from a pop-
ulation of units that individually are quite variable in their ability to predict RT
better accounts for RT variance than including only units which individually predict
RT variance well. These results support a role for this area in saccade preparation.
Also, these results suggest that unlike areas such as superior colliculus, where indi-
vidual neurons often account for RT better than the population, in prearcuate cortex,
movement preparation is best accounted for by ensemble activity.
3.5.2 Comparison between dynamics in prearcuate and PMd
Reducing the dimensionality of our data allows us to look at single-trial evolution
of population activity. While individual trials evolve differently, they tend to follow
a stereotyped path. This path has more trial-to-trial variability in prearcuate than
91
in PMd. Why are trajectories noisier in the prearcuate data? Several things could
account for this. First, the prearcuate units we record are less tuned during the peri-
movement period than the PMd units (differences in mean firing rate for the most and
least responsive target location in prearcuate were at least 1 spike/second, whereas in
PMd they were at least 4 spikes/second). Second, the shared variance in prearcuate
is lower than in PMd (Fig. 3.12), which may be due to greater heterogeneity of re-
sponse type in prearcuate. The dimensionality of perimovement signals is also lower
(Fig. 3.11), possibly corresponding to a lower degree of complexity in controlling the
effector. Third, firing rates are generally lower in prearcuate than in PMd. With
lower firing rates, the prearcuate data are more sensitive to noise, and may indicate
that our prearcuate population is not being as strongly driven by eye movements as
PMd is by arm movements. Finally, the saccade tasks did not pressure the monkey
to consistently prepare saccades during the delay. In the saccade tasks used, delay
periods were much longer and monkeys were not penalized for extremely long RTs.
Therefore, the monkey could wait until later in the delay period, or even after the go
cue, to begin preparing his eye movement. If the monkey prepared eye movements
on some trials but not others, this might lead to the type of trial-to-trial variation in
trajectories that we see.
Despite all of these differences in the prearcuate and PMd data, our ability to
relate the underlying dynamics of the system is roughly comparable. This suggests
that, despite potential differences in mechanism, some aspects of the neural strategy
underlying movement preparation may be common to both the oculomotor and reach
systems.
3.5.3 Summary
In this chapter, we studied how single-trial dynamics of populations of units in
prearcuate reflect subjects’ level of preparation for an upcoming movement, and com-
pared this to the relationship between single-trial dynamics and movement prepara-
tion in PMd. Using a heterogeneous population of units, we were able to explain
a significant amount of the across-trial variance in subjects’ RTs with trial-to-trial
differences in ensemble activity. This is noteworthy because it (1) extends the class
of reaction-time prediction models to a population of units with different RF loca-
tions and different response properties, (2) validates that the dynamics of movement
92
preparation that we visualize as low-dimensional trajectories do in fact reflect behav-
ior, (3) provides physiological evidence that this area of prearcuate cortex is tightly
linked to eye movements, and (4) demonstrates that despite differences in the oculo-
motor and reach systems, movement preparation can be described in the same way in
both systems. This framework for analyzing neural population activity and dynamics
should permit further comparisons of movement preparation within and across motor
systems.
Chapter 4
PMd experiments
In Chapters 2 and 3, our goal was to connect the neural state of the oculomotor system
to decision and movement preparation processes. One of the aims of our research is to
understand how these processes compare for eye movements and for arm movements.
This chapter outlines experiments related to movement preparation and decisions in
the reach system, connecting plan dynamics in PMd to arm movements.
In a natural environment, we must form plans even if we have incomplete infor-
mation about where to move. To make a movement quickly and accurately, one must
start preparing a movement, but be able to change plans according to incoming sen-
sory input. In this chapter, we describe two sets of experiments that address forming
and switching plans, and one that addresses the relationship between plan signals
and muscle activation. The first set of experiments probes how quickly plans can
be formed and changed. Second, we address how plans are formed in situations of
uncertainty. The final set of experiments explores how planning dynamics relate to
muscle activity.
4.1 How quickly can movement plans be switched?
4.1.1 The prosthetic cursor task
Neural prosthetic systems, capable of translating cortical activity into control signals
for guiding prostheses, must operate quickly and accurately in order to be clinically
viable. In communication prostheses, it is important to be able to directly estimate
desired cursor positions from the delay (plan) activity present before, or even without,
93
94
Chain Position 1
Chain Position 3
Chain Position 2
360o��
0o��
Pre
f D
ir
200 ms
H
E
Touch Hold Trial #1 Trial #3Trial #2 Trial #4 'GO' Cue Real Reach
Figure 4.1: The chain task. (adapted from Santhanam et al. (2006) by permissionfrom Macmillan Publishers Ltd: Nature.) In this task the monkey was trained toperform two types of trials: real reach trials, where the monkey planned and executeda movement, and plan-only trials, where the monkey planned a movement, but wasnot given the go signal to execute the reach. For the plan-only trials, the monkey’smovement plan was read out and used to select the planned reach location. If thelocation read out correctly matched the target location, another plan-only trial waspresented. The plan-only trials were presented either until the decode algorithmselected the wrong target location or until a chain, or sequence, of N plan-only trialswas completed. (N = 3 for Monkey H but was arbitrarily long for Monkey G.) Theend of each chain was followed by a real reach trial.
arm movements. Our lab has reported that such systems can achieve high perfor-
mance (Santhanam et al., 2006). However, achieving this high performance in new or
different paradigms requires understanding how neural activity changes in these con-
texts? Here we asked how premotor cortex (PMd) neurons respond during planning
to a rapid succession of targets.
We are able to plan movements on very fast timescales. Behavioral studies have
revealed temporal limits of planning, demonstrating that following movement instruc-
tion it takes at least ∼200 ms to execute a reaching movement (Rosenbaum, 1980;
Riehle and Requin, 1989; Crammond and Kalaska, 2000). However, less is known
about the neural basis of this lower limit.
The study by Santhanam et al. (2006) demonstrated that not only are monkeys
capable of planning reaching movements at very fast speeds (reach targets were pre-
sented at ∼3 Hz), but that this plan activity in neurons in PMd could be decoded
95
by a computer algorithm and used to control a prosthetic device. Monkeys were en-
couraged to plan reaches to targets as soon as they appeared by withholding rewards
if reaction times were too slow. To read out a monkey’s planned reach, Santhanam
used a maximum likelihood (ML) decoder to determine to which target location the
monkey was planning to reach. While their algorithm was able to decode a sequence
of planned reach locations in their prosthetic cursor task, the accuracy of the decoder
decreased for targets presented later in a sequence. That is, plan activity to a given
reach goal changed depending on whether the plan was generated first in a sequence
of plans, or later on in this sequence. In this section, we ask: how do movement
plans change as a function of sequence position? To address this question, we inves-
tigated how changes in the location of a reach goal influence plan activity in PMd,
and how this depends on the speed with which a monkey must switch his plan within
a sequence.
In the prosthetic cursor experiment of Santhanam et al. described above, we hy-
pothesized that the drop-off in decoder performance for movements planned in rapid
succession would be due either to a change in tuning of the neurons recorded, or to
a change in the firing rate of these neurons (Kalmar et al., 2005). To test this hy-
pothesis, we analyzed the neural data from the prosthetic cursor task of Santhanam
et al. (2006). In the prosthetic cursor task, a monkey was given a sequence of poten-
tial reach targets, but only instructed to execute a reach to the last target position
(Fig. 4.1). Since the go cue could be given at any point in the sequence, the monkey
needed to plan reaches to each of the potential targets, to ensure a short enough RT
once the go cue was given. During delay periods between presentation of a target and
either the go cue or the next target, neural activity was recorded and a ML decoder
was used to determine the monkey’s plan. At the end of each delay period, either the
target icon flashed to indicate that the decoder had read out a plan to the correct
target location, or the fixation and touch points were extinguished, which was the
monkey’s cue to reach to the target on the screen. Sequences of ‘plan-only’ target pre-
sentations were interleaved with cued reaches. Successful decoding of planned reach
direction allowed us to confirm that the monkey was planning on each trial. However,
the accuracy of the decoder decreased for plans deeper into a sequence (Gilja et al.,
2005). This suggested that something about movement plans changed as a function
of sequence position.
96
4.1.1.1 Tuning curves are modulated by chain position
Chain Position
1 2 3
30
70
110
150
190
230
310
350
Ta
rge
t D
ire
cti
on
(d
eg
)
10
30
70
110
150
190
230
310
350
25
2
15F
iring
rate
(sp
ike
s/s
)
1 2 3
0
1
0
1
Not normalized Normalized
Figure 4.2: Tuning does not depend on position in chain. Left: Average firing ratesfor each target direction and chain position are shown in grayscale, for two exampleneurons. Right: Same as figures on left, except within the tuning grid each columnis normalized by the peak firing rate for that chain position.
To visualize changes in tuning or firing rate across chain positions, we plotted 2D
tuning curves for both neurons (Fig. 4.2, left column). In these tuning curves, each
column shows relative levels of neural activity for each target direction—at a given
chain position, and each row shows the relative change in firing rate for a given target
location—across chain positions. The target direction eliciting the highest firing rate
from a cell was considered that neuron’s preferred plan direction. The majority of
neurons maintained the same preferred plan direction across chain positions; only
these neurons were included in the rest of this analysis. (The neurons that did not
maintain the same preferred plan direction across chain positions generally did not
show clear trends. Many of these neurons had low firing rates, and changes in their
preferred direction were not statistically significant.) The consistency in tuning is
97
Monkey H Monkey G
Chain Position
1 2 3 1 2 3 1 2 3
Chain Position
1 2 3 1 2 3 1 2 3
30
70
110
150
190
230
310
350
30
70
110
150
190
230
310
350
30
70
110
150
190
230
310
350
30
70
110
150
190
230
310
350
30
70
110
150
190
230
310
350
30
70
110
150
190
230
310
350
Ta
rge
t D
ire
cti
on
(d
eg
)
Firin
g ra
te (s
pik
es
/s)
16
2
11
4
14
4
14
4
10
6
25
5
8
1
11
5
25
10
45
10
35
10
45
10
28
10
16
2
34
18
14
2
26
6
35
10
Figure 4.3: For each neuron, data were grouped by target location and position inthe chain. Tuning curves were often modulated up or down for sequential chain po-sitions, though few changes in tuning direction were observed. Interestingly, MonkeyG tended to have positive gains whereas Monkey H tended to have more negativegain cells.
easier to see when each column of the 2D tuning curve is scaled by its maximum
value, as in Figure 4.2 (right column). Using these 2D tuning curves, we found that
though cells’ tuning remained relatively constant, the tuning curves were often scaled
up or down for sequential chain positions. Figure 4.3 illustrates a representative
sample of neurons’ tuning at different chain positions. Many neurons recorded on our
arrays had similar preferred directions.
We summarized the modulation of tuning curves by finding the gain for popula-
tions of neurons in two monkeys. The gain was computed by regressing a line to the
preferred direction’s average firing rate across chain positions. Interestingly, Monkey
G tended to have positive gains (Fig. 4.4, right) whereas Monkey H tended to have
more negative gain cells (Fig. 4.4, left), though this could be due to slight differences
in task structure. If gains were only negative, this would be consistent with the idea
that the dynamics driving movement planning cannot change fast enough for cells to
98
100
80
60
40
20
0 -12 -10 -8 -6 -4 -2 0 2 4
60
50
40
30
20
10
0 -8 -6 -4 -2 0 2 4 6 8
25th percentile: -0.6676
50th percentile: -0.1143
75th percentile: 0.2961
p<0.05
25th percentile: -0.1932
50th percentile: 0.6655
75th percentile: 1.7227
p<0.05
GainGain
Nu
mb
er
of
ne
uro
ns
Nu
mb
er
of
ne
uro
ns
Monkey H Monkey G
Figure 4.4: Gain histograms. For each neuron, gain was calculated by regressing aline of average firing rates calculated during the delay period (∼300-450 ms) vs. chainposition, treating position in the sequence as a continuous variable. Histograms showtuning curve gains for neurons combined across multiple experiments. The histogramof tuning curve gain is shifted to the left for Monkey H, but slightly to the right forMonkey G.
maintain the same firing rates for tuning curves deeper in the chain. Sequential chain
positions would then have increasingly scaled down versions of the tuning curve at
the first chain position. Since we saw cells with both positive and negative gains in
the same recordings, this suggests that the data cannot be explained by an inability
to produce enough spikes repeatedly. What is causing this modulation of firing rates?
We did not see evidence of this modulation in the standard delayed reach task, which
was used to train the decoding algorithm. This suggests that modulation of overall
firing rates is likely related to the rapid switching of planned movements, shorter
delay periods in the prosthetic cursor task, or possibly to the level of mental effort
exerted by the monkey.
4.1.1.2 Inter-monkey differences
Why are the gain effects different for the two monkeys? One possibility is that even
though both arrays were positioned similarly in PMd, slight differences in placement
or electrode depth might have caused subsampling of different populations of neurons.
If, as in Andersen et al. (1985a), an external factor is modulating tuning of these
neurons, this factor might be acting differently on the two arrays. Figure 4.5 shows
placement of the arrays with respect to sulcal landmarks for the two monkeys. Though
the arrays are very nearly in the same location in both monkeys, slight differences in
the populations of neurons recorded may account for some of the differences in gain.
99
Monkey H
Monkey G
Figure 4.5: Array differences: Placement of electrode arrays. Arrays were placed indorsal aspect of premotor cortex (PMd) in both monkeys. Though array positionswere quite similar, small differences in placement could have caused a different sub-population of neurons to be recorded. Intraoperative photographs of arrays with sulciindicated. Sulcal landmarks were drawn by hand on magnified, full-color, full-contrastversions of these photographs, then pasted in register on these images. Ce.S., centralsulcus; S. Pc. D, superior precentral dimple; Sp. A. S., spur of the arcuate sulcus; A.S., arcuate sulcus. Figure modified from Batista et al. (2007), with permission.
Both monkeys performed the reaching task with the hand contralateral to the array.
Another possibility is that differences in trial structure and reward schedule (Fig. 4.6)
caused the monkeys to adopt different strategies, which translated into differences in
neural representation of the task. Monkey H was rewarded for real reaches only, and
chains of prosthetic cursor trials were limited to 3-target sequences. For monkey G,
chains continued until the decoding algorithm failed, after which a real reach was
always cued. These chains were up to 12 trials long, but for this monkey each trial
was rewarded. It is unclear to what extent differences in neurons’ tuning gain re-
late to these task differences. The experiment in Section 4.1.2 describes an improved
experiment designed to address the neural limits of fast planning.
100
Monkey Hreach
planonly
no reward reward
Monkey G... reachplan
only
reward
Figure 4.6: Trial structure and reward schedule. For Monkey H, chains of prostheticcursor trials were up to three trials long and the monkey was rewarded on the realreach trial at the end of each chain. For Monkey G, chains of prosthetic cursor trialswere variable in length and the monkey was rewarded on each trial.
4.1.1.3 Gain field analogy
Figures 4.7 and 4.8 illustrate two compatible hypotheses about this effect. Figure 4.7
shows Andersen et al.’s model of gain effects, applied to a posterior parietal cortex
neuron. In their model, individual cells’ retinotopic tuning is modulated by the posi-
tion of the eyes. Using this approach to describe our data, we notice that something is
happening differently at different chain positions that modulates the neurons’ tuning
curves up or down. What is modulating neural activity in the prosthetic cursor task?
The source of the modulation in firing rate across chain positions is not understood,
but may be due to attention, reward expectation or intrinsic cortical dynamics.
Retinotopic postition (degrees)
Re
sp
on
se
(s
pik
es
/se
co
nd
)
Gain modulation in PPC (from ref. 2)
Eye postition * Retinotopic tuning
Figure 4.7: The change in firing rate and lack of tuning shift in PMd responses resem-ble gain fields seen in other cortical areas, such as posterior parietal cortex (Andersenet al., 1985b), where an animal’s eye position modulates the gain of retinotopic tuningcurves.
101
4.1.1.4 State-space model for investigating cortical differences
Figure 4.8 shows a way to think about this task in neural state space (e.g., Yu et al.,
2009, Figure 1). As a monkey plans to three sequential targets, population activity
moves through the appropriate regions of state space—that is, given enough time
to plan (blue trajectory). However, as the plan instructions get faster, the intrinsic
dynamics of cortical processing are not fast enough to bring ensemble activity all
the way to the appropriate regions (red trajectory), and the population trajectory
falls short of the optimal region within this state space. Since this state space is
comprised of many neurons tuned for different directions, reaching the appropriate
region within this space does not necessarily correspond to an increase in firing rate.
Similarly, falling short of this subregion need not necessarily correspond to a decrease
in population firing rates. To bring the system into the appropriate state, individual
cells may modulate their firing rates up or down. These results were reported in
Kalmar et al. (2005). To test how the gain effect seen in individual neurons relates
to this ‘base-cutting’ picture, we need to be able to systematically vary the speed
of the target presentation. The next section describes an experiment we designed to
address this explicitly.
neuron 1
(firing rate)
neuron 2
neuron 3
Reach
target 1
Reach
target 2
Reach
target 3
One “slow” target
presentation chain
One “fast” target
presentation chain
Figure 4.8: The gain effect seen during rapid movement preparation could be dueto a variety of influences, including attention, reward expectation (e.g., Roesch andOlson, 2003) or intrinsic cortical dynamics.
102
4.1.2 Future directions
One limitation of the prosthetic cursor task is that the speed of planning and rapid
plan-switching were not varied systematically. Future work can build on the obser-
vations from Section 4.1.1 to explicitly investigate the effects of planning speed and
sequence position on neural activity, and to identify the neural basis for the speed
limits imposed on plan-switching. Figure 4.9 describes a behavioral task that could
be used to measure how quickly neural plan activity in PMd can signal a change in
the direction of a reach goal, and how this relates to a monkey’s reaction time in a
task requiring rapid switching of reaching plans. This experiment would give us an
estimate of how quickly movement plans can be modified in a task where the monkey
is instructed where to reach.
p = 0.75 for another plan cue
p = 0.25 for the go cue (left target)
p = 0.25 for the go cue (right target)
Figure 4.9: Schematic of a proposed task for investigating neural dynamics of rapidplan-switching. As in the delayed reach task, each trial of the plan-switching taskwould begin with hand and eye fixation. Next, two isoluminant squares, one blue andone yellow, would be presented. Either color square may appear in either of the twolocations. The squares are presented for a variable delay period of 30-430 ms. Thisdelay period would be used for all target presentations in a chain, and a new delayperiod drawn after the monkey executed a reach, which marks the end of each trial.At the end of the delay period, either the blue and the yellow square switch places(with probability = 0.75), or the go cue would be given (p = 0.25). The monkey wouldbe trained to always ignore the blue square (the distractor) and to reach to the yellowsquare (the target) when the go cue is given. He would be rewarded for brisk andaccurate movements, according to the same criteria described for the delayed reachtask. Typically, 1000 - 1500 trials of this task would be recorded, each consisting ofchains 1-8 targets long.
103
4.2 Planning with uncertainty
Providing the monkey with explicit cues about where to reach is a straightforward way
to study movement preparation. Though we can learn many basic principles about
the motor system using simple tasks, some properties of movement preparation may
only be revealed in more ethologically-relevant contexts. In the real world, there is
often uncertainty about which actions will best achieve our goals. In this section, we
will explore how reach plans are represented when there is uncertainty about where
to reach.
We need to be able to anticipate events in the environment in order to respond
quickly. Often, though, the appropriate response is not immediately apparent. In
order to act rapidly, our motor system needs to be able to accommodate uncertainty
about the environment, and incorporate these statistics into movement plans. A num-
ber of studies have demonstrated a strong behavioral correlation between subjects’
reaction times and the degree of uncertainty in a directed motor response task (Pos-
ner et al., 1980; Rosenbaum, 1980; Favilla et al., 1989; Pellizzer and Hedges, 2003).
To explore the neural basis of this relationship, several studies have used similar
tasks while recording patterns of neural activity in various motor-related brain areas.
Basso and Wurtz (1998) found that superior colliculus neurons modulate their activ-
ity in relation to the number of possible saccade targets (irrespective of number of
visual stimuli), thus encoding uncertainty about upcoming eye movements. Similarly,
Bastian et al. (1998) demonstrated that the amount of prior information about the
direction of an upcoming reach is reflected in the activity of single neurons in motor
cortex. Expanding on this, Cisek and Kalaska (2002) proposed that when presented
with two potential reach targets, individual neurons in premotor cortex simultane-
ously represented movement plans to both target locations. However, Horwitz and
Newsome (2001), using a different instructed-delay task, found suggestive evidence
that superior colliculus neurons alternated their movement plan between two targets
on individual trials, rather than representing both plans simultaneously. Plan ac-
tivity from individual neurons on single trials is noisy, making it is difficult, if not
impossible, to disambiguate the plan strategies proposed by Cisek and Kalaska, and
Horwitz and Newsome with existing data.
To study the dynamics of movement planning at a fine temporal scale, we need to
examine how the underlying signals of the neuronal population evolve from moment
104
to moment. Difficulty in interpreting single-cell responses is often solved by averaging
neural activity across many trials; however, this would possibly obscure the dynamics
of planning we want to study. A solution to this problem is to record from many
cells simultaneously, as described throughout this thesis. This single-trial approach
is necessary for prostheses, which cannot afford to average across ‘trials’ to determine
an upcoming movement, and will allow us to study dynamics of movement planning
on a behaviorally-relevant timescale. In this section, we use multi-electrode recording
techniques to visualize single-trial movement plans of a population of PMd neurons,
and to characterize how plan dynamics are influenced by uncertainty in our task.
Recent evidence by Cisek and Kalaska (2002) suggests that neurons in movement
preparatory areas are capable of simultaneously preparing reaches to two spatially
distinct target locations when there is ambiguity about which target will be cued.
Pellizzer and Hedges (2003) found behavioral evidence supporting this finding. A
pilot study in our lab attempted to reproduce these experiments, that is, to determine
whether we found evidence of co-coding—simultaneous representation of mutually
exclusive movement plans—on single trials. If the monkey instead changed his plan
back and forth between the two potential target locations on each trial (flipping), but
did so with different timing across trials, the across-trial average activity during the
plan period would look as if the monkey was planning to both target locations on each
trial. With access to multiple neurons recorded simultaneously, we were able to look
at single-trial plan activity in greater detail. The results in this section came from a
collaboration with Zuley Rivera Alvidrez, which was reported in Rivera Alvidrez et
al. (2007) and Rivera Alvidrez et al. (2008).
To disambiguate the two hypotheses (simultaneous plans to competing targets vs.
rapidly alternating plans to single targets), we used a color cue task. This task is
a variation of the standard delayed reach task described above. After fixation, blue
cues were presented simultaneously at two of 7 possible target locations (Fig. 4.10,
bottom panel). These potential reach targets were illuminated for a 400 ms delay
period. At the end of this delay one of the possible targets turned yellow, indicating
that this was the reach target. The other possible reach target, the distractor, did not
change color but remained on the screen. At the end of a second (generally shorter)
delay (30-800 ms), the monkey was given the go cue and rewarded for touching the
yellow target within a fixed time window.
105
Acquire touch
Acquire fix
Possible targets
Delay Period
Target
disambiguation
Movement period
Target acquired
Distractor task with ambigious period
Go cue
Distractor task
Acquire touch
Acquire fix
Target cue
Delay period
Movement period
Target acquiredGo cue
Delayed reach
Acquire touch
Acquire fix
Target cue
Delay period
Movement period
Target acquiredGo cue
Figure 4.10: Delayed reach and distractor tasks. Monkeys were trained to reach totargets on a screen when the go cue appeared. The target (yellow) could appear atthe beginning of a delay period by itself, together with a distractor (blue), or at theend of an ambiguous delay period. These tasks allow controlled analysis of neuralactivity during the planning and execution of reaching movements, during periodswhere the reach direction is completely or incompletely specified.
To assess how ambiguity about the upcoming reach cue affects the monkey’s reach
plan, we measured how plan activity in PMd evolves during the first delay period,
on single trials. We analyzed our results within a state space, where movement plans
are represented by the firing rates of a population of neurons (Fig. 1.11) (Churchland
et al., 2006). This state space analysis has been discussed extensively in Chapters 2
and 3.
Recent work by other members of the Shenoy lab has focused on different methods
of dimensionality reduction for state-space analyses. While Yu et al. (2006, 2009)
describe a low-dimensional nonlinear dynamical systems model by which dynamics of
106
F3
F2
F1 F1F2
F3
a b
Figure 4.11: (a) Trajectories during the delay period to two different targets(Fig. 4.10, delayed-reach task), and (b) with a distractor presented simultaneouslyat the opposing location (Fig. 4.10, distractor task).
plan activity can be studied, simpler linear approaches, such as Principal Components
Analysis (PCA), and Factor Analysis (FA) have also been explored. The analyses in
this section were performed before GPFA was developed. Fig. 4.11 shows a state-
space view of plan trajectories to two different targets’ optimal subspaces (green/blue
dots), using factor analysis to reduce the responses of ∼250 neurons to their three
largest dimensions of variability. To visualize plans to different targets (with different
optimal subspaces) within the same state space, we perform dimensionality reduction
on data collected from all reach directions. We define the optimal subspaces by fitting
a Gaussian to points at the end of (sufficiently long) delay periods in the standard
delayed reach task (or in other instructed tasks).
In the distractor task (Fig. 4.10, middle row), we used factor analysis (FA) to
visualize the neural population’s plan activity. If PMd were co-coding plans to both
targets, we would expect to see the plan trajectory fall between the subspaces for
the two targets (Fig. 4.12a). If PMd flipped plans between the two possible targets,
we expected to see the plan trajectory move between the two subspaces (Fig. 4.12b).
Fig. 4.13 illustrates several example plan trajectories during the delay period. Though
there is evidence for flipping on some trials, there is a considerable amount of variation
between different trajectories. Some of this variation is likely due to the presence of
a second delay period before the go cue, which gave the monkey less incentive to
begin planning during the first delay. To demonstrate that the plan trajectories were
actually related to movement preparation, a control was performed in which data from
107
Leftreachtarget
Rightreachtarget
x1
x2
x3
Both targets(non-executable)
Leftreachtarget
Rightreachtarget
x1
x2
x3 Co-coding Flipping
a b
Figure 4.12: (a) Co-coding hypothesis, as described in Cisek and Kalaska, 2002. (b)Plan-flipping hypothesis, similar to proposed mechanism from Horwitz and Newsome,2001.
each neuron were shuffled across trials. For trials of the ambiguous distractor task
(Fig. 4.10, bottom row), firing rates from each neuron during the ambiguous delay
period were randomly permuted across trials, as illustrated in Figure 4.14a. On each
shuffled trial, firing rates of each neuron were drawn randomly from that neuron’s set
of firing rates from trials in which the ambiguous distractor was presented.
Breaking simultaneity of recording with this shuffle means that any structure in
these trajectories would likely be due to chance (Fig. 4.14c). Though it is clear
that there is more structure in the plan period in Fig. 4.14c than the control in
Fig. 4.14b, without access to other behavioral or task variables, few conclusions can
be drawn from this result. A supporting piece of evidence that this structure relates
to planning comes from the monkey’s behavioral bias. On some days, the monkey’s
default plan during the ambiguous period was heavily biased toward one target (e.g.
Fig. 4.15). Such biases in planning are reflected in longer RTs for the target to which
the monkey was not biased, in comparison with RTs for the standard delayed reach
task. Figure 4.15a shows neural plan trajectories during the ambiguous period, for
trials when the monkey’s bias was the same as the target to which he was ultimately
cued (in this figure, the left target). Figure 4.15b illustrates trials in which the
monkey’s bias is obvious. On these trials, the monkey planned to the left target
during the ambiguous delay period, and then switched his plans to the right target
once the cue was given for that target. This further supports the hypothesis that
neural activity in the plan period of the ambiguous distractor task corresponds to
108
F1
F2
F3
F1
F2F3
a b
c d
Figure 4.13: Sample trajectories from the ambiguous period of the distractor task.Neural trajectory endpoints to two targets (blue and green dots) are from delayedreach trials (no distractor). Neural trajectories from the ambiguous distractor taskare projected into this space. Red dots mark starting points of the trajectories atthe time of target presentation. Crosses represent the mean of clusters, and trajecto-ries show neural activity during the whole ambiguous delay period. (a) Example ofactivity during the ambiguous period that stays near the baseline state, presumablycorresponding to a trial in which the monkey was not planning during the delay pe-riod. (b) shows a trajectory with evidence of flipping—it moves toward the subspacefor the blue target before moving to the green. (c) and (d) show more examples offlipping. The 3-dimensional space in which the data was projected was found usingfactor analysis on the delay period of the delay-reach trials and the ambiguous periodof the distractor trials for the same two opposing targets.
the monkey’s upcoming plan. However, to better understand what drives changes
in planning during this ambiguous period we may gain leverage by systematically
varying parameters of this task.
In this pilot study, we asked how uncertainty influences plan activity when the
monkey had no information about to which of the two targets he should plan. Future
experiments could clarify interpretation of these data by (1) enforcing planning by
excluding the second delay period, thus pressuring the monkey to begin planning
109
0.015 0.045
T1 T2N1
T1 T2 TnN2
Tn T39 T17N1
T8 T72 T27N2
T2
0
180
a
b c
Figure 4.14: Occupancy rate and shuffling test. In the ambiguous distractor task delayperiod, this test measures what percentage of the time the trajectories occupy a targetregion of the state space corresponding to planning, as opposed to the baseline region,which does not. If these trajectories represent planning, firing rates of all neurons willbe driven by the same underlying system. Breaking the simultaneity of the neural databy shuffling firing rates of individual neurons across trials should eliminate structurein the trajectories related to planning. Neural activity was shuffled across trials as acontrol as shown in (a). (b) shows shows unshuffled neural trajectories during theambiguous distractor delay period, where purple dots mark the endpoint of the plantrajectory in the non-ambiguous distractor task. The black lines separating targetclusters from baseline were found using regression. The portion of the trajectoriescrossing these lines into the target region of the state space are shown in magenta.The occupancy rate for the trajectories in this target region for this set of targetswas 13.1%. (c) is same as (b), except the shuffled data were used. The occupancyrate computed for the shuffled data was 2.6%. The figure inset shows a histogramof the occupancy rates obtained after 1000 random shuffles of the neural data. Themean of the distribution was 2.58% with 0.47% standard deviation. These analysessupport our interpretation that flipping trajectories, during the ambiguous distractordelay period represent not noise unrelated to planning, but changing plan states.
during the ambiguous period, and (2) systematically increasing the likelihood the
monkey will be cued to reach to a given target location. This would allow us to
quantify how movement plans change as a function of how much information about
the upcoming reach direction is available during the delay period, and to what extent
110
a b
Figure 4.15: Illustration of strong bias towards left targets during planning in theambiguous distractor task. (a) shows trajectories during the ambiguous period fortwo opposing targets, with highlighted trajectories showing planning towards the lefttarget (blue dots). Note that there is no bias in terms of the ultimate saccades made(i.e. roughly equal numbers of green and blue dots), but that the bias is apparentduring the ambiguous period of the trial, in which the monkey seems to form a defaultplan to the blue target. (b) is the same as (a), except both the ambiguous periodand the delay period are shown for the trials in which the right target was ultimatelycued. As in (a), the neural trajectories during the ambiguous delay period movetoward the region corresponding with planning reaches to the blue target. However,during the second delay period when the correct target is cued, these default plansare then switched to the correct location (indicated by the purple dots). Highlightedtrajectories show apparent flipping in the motor plan during the delay period.
we see evidence for co-coding or flipping.
Our pilot study suggested that on some trials monkeys alternated planning to the
two possible reach goals, planned reaches to only one of the targets, or formed plans
not directly corresponding to any of the targets. To our knowledge this is the first
evidence that in a task with two target options, a variety of strategies are used on
different trials. Future experiments will be necessary to determine how the degree of
uncertainty in a task influences which strategies are employed, ultimately allowing us
to understand the dynamics of planning in ethologically-relevant contexts.
111
4.3 Neural dynamics and muscle activity
In the previous sections, we endeavored to understand how features of external stim-
uli influence how populations of neurons represent movement plans. This section
describes a collaboration with Zuley Rivera Alvidrez, in which we investigated how
neural representations of movement planning relate to muscle signals. More details
about this experiment can be found in Rivera Alvidrez et al. (2010).
How does activity in the motor cortex relate to muscle activation? Understanding
how ensembles of PMd and M1 neurons correspond to muscle activity is of both
fundamental scientific interest, as well as important for designing neural prostheses.
The state-space models described throughout this thesis can provide us with a
way to denoise, visualize (if 3-D or less), and think of the evolution of the neural
population in a few meaningful dimensions (illustrated in Fig. 4.16a). In Chapters 2
and 3 we used this state-space to help us gain intuition about the neural population,
and to relate the neural dynamics within this space to sensory encoding, decision-
making, and trial-by-trial differences in behavior. However, our understanding of
the underlying system would be further enhanced by relating these orthogonal axes,
which explain features of the neural population in state-space (S1, S2 and S3 in
Fig. 4.16a), to task variables. These state-space axes are likely to be a combination
of the behavioral signals explained by the neural data, rather than clearly relating to
any one behavioral signal in particular. Nevertheless, these axes may be recombined to
allow a potentially more useful and intuitive representation of the data (e.g. Machens
et al., 2010).
A body of literature relates activity in individual neurons in PMd and M1 to sub-
jects’ spatial goals, hand motion, joint motion, force output, and electromyographic
(EMG) activity (Scott, 2003). However, characterizing the relationship between in-
dividual neurons and various task variables has not added up to a comprehensive
understanding of how these movement areas translate neural activity into motor out-
put. We hypothesize that there is a systematic representation of motor output within
ensemble activity in PMd/M1. Further, we propose that we can rederive neural axes
such that axes in the state space correspond to muscle activity (see schematic in
Fig. 4.16b). Within this space, we predict that a neural trajectory moving along
an axis, say muscle1, will activate or deactivate muscle1, wheareas one moving along
muscle2 will change the activity of muscle2. Moreover, in this idealized view the axis
112
Figure 4.16: Low-dimensional neural state-space. (a) shows the hypothetical evolu-tion of the neural population activity (neural trajectory) in the statespace definedby the axes S1, S2, S3. t1 and t2 are two different time steps corresponding todifferent points in the neural trajectory. (b) shows an alternative space defined bybehaviorally-relevant axes. Figure from Rivera Alvidrez et al., 2010.
corresponding to a particular muscle would reflect the underlying relationship of the
neural data to that muscle. If this space accurately captures the relationship between
activity of the neural population and muscle activation, we expect that it should gen-
eralize across all tasks and behavioral contexts. To find this relationship, we asked:
how do average neural trajectories for one reach target relate to the corresponding
mean EMG?
We recorded simultaneous neural activity from motor cortices (M1/PMd) of rhesus
monkeys performing an arm-reaching task while recording EMG from arm muscles.
Reducing the dimensionality of the neural data, we asked whether any aspects of
the low-dimensional neural activity strongly correlated with the EMG signals, as
schematized in Figure 4.16b. We explored how a host of parameters of the low-
dimensional projections related to EMG signals. Of these, we found that the curvature
of the low-dimensional trajectories (as in Fig. 3.8) was tightly related to the timecourse
of muscle activity. To determine this, we found neural axes based on reaches to
one reach target (<5% of the data). Figure 4.17 shows how we defined axes of
neural curvature, and the corresponding curvature shown as a function of time. By
finding the curvature of the neural trajectory as a function of time, we produced 1-D
timeseries that could be compared with the mean EMG signals for each target.
In Figure 4.18 (right column), we show these curvature timeseries for four differ-
ent muscles. In the delayed-reach task used in this experiment, monkeys reached to
a large number of different target locations. Neural curvature axis (NCA) projec-
tion timeseries corresponding to reaches to different target positions are shown using
113
Figure 4.17: Illustration of neural curvature axis computation. (a) 3-D neural trajec-tories (circles correspond to turns). We defined the neural curvature axis (NCA) as anaxis perpendicular to the neural trajectory at time t (perpendicular to the black linesegment in (a, inset)) and lies in the same plane as the neural trajectory segmentdefined by [t-10ms, t+10ms]. Picking an axis perpendicular to the neural trajectoryat the reference time point ensured that the projection of the neural trajectory ontothe axis would have a derivative equal to zero at that point, capturing the turn orpoint of high curvature in the neural trajectory. Picking an axis that lies in the samelocal plane as the neural data ensured that the axis would be relevant (explained localneural variance) to the trajectory locally. This plane was found by computing the toptwo principal components of the neural trajectory segment. The neural trajectorieswere projected onto the this axis to obtain 1-D timeseries for each target (b). Figurefrom Rivera Alvidrez et al., 2010.
different line colors and widths. The left column of Figure 4.18 shows mean EMG
signals for each target, with reaches to different targets represented using the same
colors as in the right column.
A significant correlation across the right (neural curvature) and left (EMG) columns
support the hypothesis that the neural curvature metric is a good indicator of muscle
activity. On average, these neural axes could explain EMG for reaches across multiple
targets (average R2 = 0.65).
These results suggest that there are fundamental directions in the neural space
along which the neural population moves to activate particular muscles. Further, our
neural population included a mixture of single neurons and multi-units of unidentified
type. That our results were found using a heterogeneous neural population suggests
that the tight relationship between ensembles of cortical neurons and muscle activity
114
is not limited to corticomotorneuronal cells, and suggests that any transformation of
the cortical signals made by the spinal cord circuits is likely to be minor. Overall, our
ability to relate neural activity to muscle activity with few parameters and using very
little training data makes this a promising framework in which to study motor control
both from a scientific perspective and for the development of prosthetic systems with
highly generalizable performance.
Figure 4.18: NCA projections predict EMG features. Right column shows the NCAprojections (see text for details) for all targets (superimposed and shown in differentcolors) associated with wrist extensor (a), wrist flexor (b), trapezius (c), and biceps(d). The correlations R2 between the NCA projections and the mean EMG actvityfor the same muscles (left column) were 0.71 (extensor), 0.6 (flexor), 0.73 (trapezius),and 0.55 (biceps). Figure from Rivera Alvidrez et al., 2010.
115
4.4 Summary
This chapter focused on the reach system, describing the relationship between neural
dynamics in PMd and behavior in three different contexts. First, we investigated
how neural dynamics reflect rapid, sequential planning. In the context of a rapid
sequential-planning task, we showed that the firing rates of PMd neurons were mod-
ulated not only by the direction of a planned reach, but also by the order in which
a reach target was cued. This effect of sequence position was observed only in when
plans were cued rapidly (several target presentations per second), suggesting that
there are limits to how quickly neural plan dynamics can change. Second, we vi-
sualized neural trajectories during an ambiguous planning period, preceding final
specification of a reach target. While subjects waited for one of two possible reach
targets to be cued, neural trajectories reflected evidence of plan-switching more often
than predicted by chance. Finally, we demonstrated that population activity in PMd
reflects signatures of muscle activity, and that these neural signatures generalize ro-
bustly across muscles and reach directions. Taken together, these results extend our
understanding of how neural dynamics of movement planning change when the system
is pushed to plan quickly, how plan dynamics unfold in the context of uncertainty,
and how these underlying neural dynamics relate to motor output.
Chapter 5
Conclusion
5.1 Summary & Contributions
As we take in sensory input to decide where and when to move, neural circuits
in the brain evaluate and choose between multiple motor plans. On millisecond
timescales, neurons within and across circuits must coordinate their activity to allow
us to produce fast, accurate movements. Understanding the dynamics of this process
will provide insight into the neural basis of behavior and may facilitate development
of neurally-controlled prosthetic devices.
Studies of movement preparation in the arm-movement system have demonstrated
that the longer subjects have to prepare, the faster they can initiate arm movements.
Across many cortical and subcortical areas, researchers have demonstrated a clear
neural correlate for this effect (discussed in Snyder et al., 2006; Churchland et al.,
2006). Recent work has begun to describe the neural dynamics underlying movement
preparation and to describe relationships between these dynamics and behavior (e.g.,
Churchland et al., 2006; Afshar, 2008; Afshar et al., 2010).
In the oculomotor system, a large body of literature has described the relationship
between subjects’ reaction times and neural activity reflecting movement initiation
in eye-movement areas such as FEF, SC, and LIP (e.g., Wurtz and Goldberg, 1972;
Hanes and Schall, 1996). However, considerably less research in the oculomotor sys-
tem has focused on movement preparation—activity involved in planning a movement
before an instructed go cue. Recording from prearcuate cortex, an oculomotor region
on the surface of the frontal lobe, we used a rich behavioral paradigm that allowed us
116
117
to study responses related to movement preparation and to decision-making. Though
responses in prearcuate cortex have been characterized in the context of its broader
cortical neighborhood (Section 1.5.1), few studies have explicitly focused on this area
of the cortical surface between the arcuate and principal sulci. Utilizing multielectrode
recording techniques in prearcuate cortex, we were well positioned to ask (1) how the
population responses in this area reflect movement preparation and decision-making,
(2) how these responses compare to those recorded in other oculomotor areas, (3)
how dynamics of movement preparation unfold, and (4) how these dynamics compare
to those described in the arm-movement system.
In Chapter 2, we characterized response properties of a heterogeneous popula-
tion of prearcuate neurons, using both traditional metrics and novel state-space ap-
proaches. Dimensionality reduction methods allowed us to visualize average dynamics
of the population response, which would be obscured by averaging across the diverse
PSTHs of individual neurons. During the presentation of the dots stimulus, we found
neurons that reflected choice predictivity, similarly to those described by Kim and
Shadlen (1999) in this region of prefrontal cortex and to those of Shadlen and New-
some (2001) in LIP. Applying state-space methods to either the full dimensionality
of the data or to the dimensionality-reduced responses, we characterized dynamics of
choice-predictive activity in the population. These dynamics reflected a similar time-
course of dot motion integration to those we obtained using ROC analysis. During
the perimovement epoch of the task, our recordings contained neurons that responded
before, during and after saccades, some of which exhibited similar characteristics to
FEF neurons (e.g., Schall, 1997).
In Chapter 3, we drew upon visualizations of the neural trajectories during the
perimovement epoch to test hypotheses about movement preparation within the state
space. Including all neurons with saccade-related responses, we were able to explain
a significant amount of variability in monkeys’ reaction times based on the position
of neural trajectories at the time of the go cue on individual trials. This is the first
time, to our knowledge, that the dynamics of movement planning derived from the
activity of multiple, simultaneously recorded neurons in the oculomotor system have
been related to reaction time on a trial-by-trial basis.
We also used dimensionality reduction methods to determine how many dimen-
sions were necessary to describe the shared variance of the population responses in
118
prearcuate and in PMd. Overall, less of the variability in prearcuate responses was
shared across the population. This may be due to individual neurons in the prearcuate
population receiving more disparate cognitive influences. Though the shared variabil-
ity in prearcuate was lower, fewer dimensions were needed to describe the variability
that was shared across the population. Thus, of the dimensions that did capture
covariation, these dimensions explained a large amount of variance.
Why is the dimensionality of PMd greater than that of prearcuate? A body of
work in the arm-movement literature describes complex movements that are elicited
when passing current into primary motor cortex (e.g., Graziano et al., 2002). How-
ever, passing even large amounts of current into oculomotor areas elicits only saccades
(Robinson and Fuchs, 1969). Eye and arm movements differ in many ways (e.g., ef-
fector speed, weight, number of muscles required), so one might expect oculomotor
and reach systems to utilize different strategies for motor preparation. Arm move-
ment has more degrees of freedom than eye movement, and involves more muscles
as well. As far as physical constraints, eye and arm movement differ as well, both
in terms of forces (e.g., gravity), and in terms of obstacle avoidance during move-
ment. Given these differences in biomechanical aspects of the arm and the eye, it
would be unsurprising if the strategies required to control arm movement were more
complex or higher-dimensional. Continued application of dimensionality reduction
approaches across different movement systems will further our understanding of how
the underlying processes driving movement preparation compare across systems.
In Chapter 4, we found that neural plan activity in PMd was influenced by prior
target cues within a rapid sequential-planning task, in which multiple targets were
presented each second. On short timescales, the amplitude of neurons’ responses was
modulated not only by the location of a target cue, but also by the order in which
the target was presented. However, this effect of sequence position on neurons’ firing
rates was not seen when a sequence of target cues was presented slowly (> 1 second
between target presentations). One interpretation of this result is that the dynamics
of movement planning cannot change quickly enough to meet the demands of the
rapid plan-switching task. This inability to fully change plans on fast timescales may
be a consequence of limits of the neural dynamics of planning. Future experiments
will allow us to visualize dynamics in such tasks, which may aid in understanding the
source of constraints on rapid planning and plan-switching.
119
In the rapid plan-switching task, reach plans on previous trials influence neural
activity on the current trial. In prearcuate cortex we found an analogous effect on
longer timescales (several seconds per trial), as described in Chapter 2. In these
longer-timescale effects, the monkey’s prior action was reflected in neural activity,
both during the hold period after his saccade and often well into the next trial. Such
effects in PFC have been hypothesized to maintain a history trace of subjects’ actions
for the purpose of short-term working memory (reviewed by Fecteau and Munoz,
2003). The mechanisms underlying these shorter- and longer-timescale effects of
previous actions in PMd and prearcuate cortex may be similar. Insight into these
mechanisms could be gained by further characterization of the dynamical systems
driving both saccadic and reach planning.
Finally, the latter sections of Chapter 4 describe collaborative work with Zuley
Rivera Alvidrez in which we further characterize the neural dynamics of planning arm
movements. In Section 4.2, we used dimensionality reduction to visualize complex
plan dynamics in an instructed delay task with an ambiguous period. Section 4.3
also describes plan dynamics, but focuses on relating them to muscle output. Using
population recording in PMd/M1 and EMG recording, our results suggest that there
are fundamental directions in the neural state space along which the neural population
moves to activate particular muscles.
Our results in prearcuate cortex and in PMd bring us several steps closer to
understanding the neural dynamics of choosing and preparing movements, and more
generally, how the process of movement preparation is reflected in different systems
in the brain.
5.2 Array recording
5.2.1 Pros and cons of array recording
5.2.1.1 Advantages
The multielectrode recording arrays we used for our experiments offer both advan-
tages and disadvantages. Some of these advantages have been described throughout
this thesis. In particular, arrays enable simultaneous recording of populations of neu-
rons and the ability to relate activity of neural populations to behavior on single
120
trials. This simultaneous recording allows time-varying influences common to the
population (such as cognitive factors) to be either factored out or directly investi-
gated. Another advantage of using arrays is that the recording yield is higher. For
each behavioral experiment, many more cells can be recorded. Likewise, it is possible
to collect a higher number of trials for each neuron, assuming relative stability of
the array. The fixed position of the array also allows across-day comparisons of the
population response, which would be of particular use in studies of learning. Having
unmovable electrodes also prevents selection bias based on cells’ response properties
and implicitly creates variety in the relationship between cells’ response fields and
target locations.
5.2.1.2 Challenges of array placement
Though these advantages are substantial, there are also significant disadvantages of
chronic electrode array recording. First, determining correct placement of arrays
can be challenging. Our experience with misplaced arrays has underscored the im-
portance of confirming array locations early. We have learned that sulcal anatomy,
though a good general landmark, is not a reliable predictor of physiological responses.
Across and even within a subject (across hemispheres), the exact location of the arm
representation in PMd can vary by several millimeters.
Additionally, we learned that neural responses can be task-modulated (though
not well-tuned) even if the array is in the wrong place for the behavioral task. Ta-
bles 5.1 and 5.2 provide an overview of the recording utility of the arrays implanted in
prearcuate and PMd over the course of the experiments described in this thesis. The
problems with array placement noted in these tables have led us to explore various
techniques for targeting the appropriate cortical area.
To target the appropriate area for array placement, several groups have tried
recording or microstimulating before array implantation. Both of these approaches
carry different advantages and complications. Using single-electrode recording from
the cortical area of interest provides detailed spatial resolution about the response
properties in that region. However, the depth of the electrodes on the array may not
match the depth at which the neural signals of interest were recorded. Depending
on the density of the neurons of interest (e.g., Betz cells in M1), the population
recorded with single electrodes may not be well-sampled by the array. Additionally,
121
dural adhesion can occur in areas of single-electrode recording, which can complicate
array insertion surgeries (Romo, unpublished observation). Though dural adhesion
was initially a concern, members from our group have recorded from PMd and M1
prior to successful array insertions. However, we have not yet ruled out the possibility
that prior recording diminishes the longevity of high-quality neural signals obtainable
from the array.
Microstimulating the region of interest prior to array implantation can carry less of
a risk of dural adhesion, since it can be done intraoperatively or in a smaller number of
experimental sessions. If microstimulation is to be done in the context of behavior, an
extra surgery is necessary in order to access the tissue. If done interoperatively, effects
of anesthesia must be taken into consideration. Some anesthetics inhibit movement,
and the anesthesia must be kept very light in order to see effects of microstimulation.
We explored this possibility extensively, though ultimately determined that the risk to
the monkeys and potential problems with the approach were too significant. One such
problem of microstimulation, both intraoperative and in the context of behavioral
experiments, is that passing current into the region of interest may influence a broader
population of neurons than desired, which could lead to incorrect placement of the
array.
Microstimulating the region of interest after array implantation is a fairly reli-
able way to determine how neural responses correspond to limb representation. The
disadvantages of this method are that it requires a full array implantation surgery,
and that microstimulation through the array has previously been shown to damage
electrodes or the neural tissue (Shenoy lab, unpublished observation). However, array
technology has improved over the past few years, and passing current through the
Table 5.1: PMd arrays
Array SignalsGeorge good signalsHam I good signalsHam II array lost signals after 1 weekHam III array misplaced (leg representation)Larry I weak signalsLarry II array misplaced (leg representation)Larry III array misplaced (finger representation), no plan activityIsaac good signals
122
Table 5.2: Prearcuate arrays
Array Overall Dots Epoch Perimove Epoch Hold EpochVito I array misplaced
(M1)N/A N/A N/A
Vito II good signals fair signals good signals good signalsNorris no neural signals N/A N/A N/ACody array instability,
poor tuningpoor signals poor signals good signals
Tex good signals good signals good signals good signals
array to evoke movements now carries less of a risk of array or tissue damage.
Despite these challenges involved in appropriately positioning electrode arrays,
our experience has allowed us to improve our ability to place arrays correctly and to
more rapidly identify when an array is misplaced.
5.2.1.3 Other challenges of array recording
The second key disadvantage of using multielectrode arrays is that the signal quality
can be quite variable, and ultimately the ability to record single units disappears
entirely. This process of array ‘roll off’ may occur either gradually or abruptly, and
can begin as soon as 4 months (Shenoy Lab, Monkey L, array I) or as late as 3 years
(Shenoy Lab, Monkey D, array I) after array implantation. The average duration of
high-quality recordings from the arrays in the Shenoy and Newsome labs is approx-
imately 9 months. Given the inherent surgical risk of implanting arrays, the length
of time required to train a monkey, and the uncertainty in determining appropriate
array placement, having access to high-quality signals for as long as possible is obvi-
ously important. The limited duration of high-quality signals from array recordings
is one of their most fundamental drawbacks.
A third disadvantage of using chronically implanted arrays is that they do not
allow targeting of specific neural populations, or neurons with particular response
properties. Though this yields population recordings less sensitive to biases in se-
lecting neurons based on responses, our recordings are instead biased toward cells in
the same or nearby layers. If the cells sampled are responsive to the task, this may
not be a problem. However, if cells are not responsive to the task or have complex
123
response properties, this may complicate analysis. Also, the depth of tuning of a neu-
ron on the array may be worse than that of a single-electrode-recorded neuron, for
which target properties can be more readily optimized. Neurons’ firing rates in array
recordings may be lower due to this lack of target optimization. Firing rates may also
appear lower due to spike-sorting issues, which may be exacerbated by suboptimal
positioning of electrodes with respect to cell bodies.
Recording from collections of neurons with varied response properties may be
advantageous in some situations. However, a significant difficulty with including in
our analyses nearly every cell in a “population” is that we do not know whether
these cells actually belong together functionally. The cells we record from on the
array might belong to 3 or 4 different functional assemblies, and our state space
trajectories may therefore represent a mongrel assemblage of these systems that does
not actually reflect the neural state of any functional system in the brain.
A fourth challenge of using fixed electrode arrays is that signals from different units
recorded on the same electrode can be difficult to separate. Using movable electrodes,
experimenters can adjust the position of electrodes to achieve clean isolation of units’
activity. Without this ability, assigning spikes to neurons relies solely upon spike-
sorting methods, none of which achieve the same signal quality as possible with
single-unit recording. Therefore, for analyses that require a collection of well-isolated
neurons (e.g., determining differences in response properties between anatomical cell
types, such as putative interneurons and pyramidal cells), recording with moveable
electrodes is preferable to fixed multielectrode array recording. Our ability to reliably
determine neurons’ identity in array recordings is limited by spike-sorting methods,
each of which has a different set of limitations.
Neural signals from PMd data in this thesis were sorted using in-house methods
(“RRR”) described in Santhanam et al. (2004). Neural signals from prearcuate data
were sorted using the Plexon Offline Sorter. When applying the RRR method to
prearcuate data for the purpose of comparison, we found that RRR “underclustered”
the data, finding only one multi-unit cluster (Fig. 5.1, upper left panel). In contrast,
the Plexon Offline Sorter tended to “overcluster” the data, splitting signals from one
cell into multiple clusters (Fig. 5.1, upper right panel). The correct solution is likely
somewhere in between. Though automated scripts were written to iteratively modify
spike-sorting performance of the Plexon Offline Sorter, the quality of the automatic
124
Vo
lta
ge
(µ
V)
Time relative to trough (ms)
Vo
lta
ge
(µ
V)
Time relative to trough (ms)
RRR sort, Channel 96, 1 unit Plexon sort, Channel 96, 8 units
0
40
-40
0
50
-50
0-0.2 0.2 0.4 0.6 0-0.2 0.2 0.4 0.6 -10 -5 0 5
0
6
-6
PC1
PC
2
a b c
Figure 5.1: Comparison of waveforms found using (a) RRR and (b) Plexon OfflineSorter. (c) shows the first two principal components (PC1 and PC2) of raw datafor this recording channel, using RRR. Blue circles correspond to centers of clustersof events assigned to Unit 1 (blue waveform in (a)); the pink square corresponds tounassigned events (pink waveform in (a)).
spike sorts is still not comparable to hand-sorted spikes. For this reason, and for
the intractability of hand-sorting spikes from hundreds of neurons from the number
of data sets (∼100) surveyed in this thesis, we did not focus our analyses on any
questions that required single units. We hope that in the future, better automated
spike-sorting methods will allow better resolution of the neural populations recorded
on unmovable electrode arrays.
Finally, using fixed electrode arrays presents a challenge in determining the func-
tional area recorded. Areas such as FEF and 8Ar are distinguished by the microstim-
ulation current needed to evoke an eye movement. Stimulating through the array is
possible, but it is unlikely that the current thresholds would be directly comparable
with those reported in the literature (e.g., different materials are used for electrodes;
array may be encapsulated by glia). Some of these limitations of multielectrode
recording may be addressed with future improvements in recording technology.
5.2.2 What can and cannot be addressed by multielectrode
recording and state-space approaches?
Single-electrode recordings have shown that prearcuate cortex particpates in sensory-
guided decisions and movement planning (Boch and Goldberg, 1989; Kim and Shadlen,
1999; Zaksas and Pasternak, 2006). However, less is known about how the dynamics
125
of these computations play out across neural circuits or neural populations. Though a
stimulus may initiate a process that unfolds differently on different trials, traditional
trial-averaging methods will not be able to explain what happened on individual
trials. State-space approaches as described in this thesis and in Churchland et al.
(2007) and Yu et al. (2009) provide the ability to visualize influences that may not
be the same across many repetitions of a trial. These methods allow investigation
of dynamics beyond those driven directly by a stimulus, such as volitional changes
of mind or occasional, unpredictable lapses in attention. Recording from individual
neurons, spiking noise limits the viability of finding concrete signatures of this type
of cognitive processing. To investigate changes of mind that are linked to external
stimuli, single neurons’ responses can be aligned to these stimuli to look at average
responses. However, it is not possible to time-lock rasters to internal events like
changes of mind, whose exact times vary from trial to trial and are not externally
accessible. Without averaging responses across trials, the signals available in individ-
ual neurons on single trials are typically too noisy to interpret directly. Even if it
were it possible to interpret these signals, another problem is that individual neurons
might not reflect the underlying dynamics of the overall system. Therefore, simul-
taneous recording and state-space approaches are necessary to visualize or otherwise
characterize internally-driven cognitive dynamics.
We used multielectrode recording for the experiments in this thesis because, de-
spite the aforementioned limitations, this approach offers the best way to characterize
the dynamics of neural populations. In this thesis, array recording and state-space
approaches allowed us to describe relationships between subjects’ behavior and de-
cision variables and movement preparation on a trial-by-trial basis. Such methods
are important not only for studying the single-trial dynamics of movement prepara-
tion on behaviorally-relevant timescales, but are also necessary for neurally-controlled
prostheses, which have access to signals from only a very limited amount of time.
5.3 Open questions & Future directions
In Chapter 3 we discussed different hypotheses of movement preparation and how they
would be represented in populations of motoric neurons. The extent to which these
hypotheses describe neural data may be influenced by the criteria used in selecting
126
the neurons to include in the analysis.
Our characterization of cortical areas is heavily influenced by researchers’ choices
of which cells to record. How would our understanding of FEF or LIP differ if pre-
vious studies had relaxed their selection criteria to include recordings from all cells
encountered, as on the array? Alternatively, if our population had comprised of neu-
rons typical of core FEF (e.g., Bruce et al., 1985; Hanes and Schall, 1996), how would
our conclusions differ? Individual neurons in FEF described in the literature typically
have higher firing rates than the prearcuate neurons we recorded here. Higher firing
rates of the population would create larger excursions of neural trajectories through
state-space, perhaps creating more stereotyped trajectories (as in PMd). Though we
can speculate how some features of FEF trajectories may differ from our state-space
descriptions of prearcuate populations, future work is needed to determine the influ-
ence of individual neurons’ response properties on our description of neural dynamics.
Additionally, further exploration of cell selection biases on state-space representations
across cortical areas may provide a more complete understanding of neural dynamics
than would be possible using sets of neurons selected in single-electrode recording.
Another important avenue of future research is to investigate differential charac-
teristics and contributions of subpopulations of neurons. Determining which sets of
neurons to include in population analyses (as described in Section 5.2.1) may signif-
icantly influence our understanding of neural dynamics. Future studies could sepa-
rate populations of neurons based on functional properties. For instance, population
analyses of hypothetical FEF recordings may yield different outcomes depending on
whether visual, visuo-motoric, and motoric neurons are combined or separated. To
target the appropriate subpopulations, we need to understand where different neurons
project. In the future, optogenetic techniques in concert with array recording may
provide the ability to (1) selectively stimulate specified sub-populations of neurons,
(2) simultaneously stimulate and record from neural ensembles, and (3) stimulate
incoming projections from specific areas (e.g., projections to FEF or 8Ar from LIP).
In Chapter 4 we described a result in which the dynamics of movement planning
in PMd appeared not to be able to keep up with the pace of the behavioral task. After
determining the appropriate populations of neurons in which to investigate dynamics
of movement planning, we can return to the question: what are the fundamental
speed limits of changes in neural plan dynamics? How do plan dynamics differ when
127
planning is being driven by an external stimulus, rather than by internally-driven
changes of mind? Future experiments may address these questions, as well as how
speed limits of neural dynamics compare in the oculomotor and reach systems.
The trial-to-trial relationship between population activity and reaction time in
Chapter 3 provides evidence that neural trajectories reflect dynamics of the underlying
system that drive behavior. If we could stimulate (optically or electrically) neurons on
the array during decision-making, how would this be exhibited in neural trajectories
and the state space representation? We can test our understanding by stimulating
selected neural ensembles to understand how the activity of these neurons influences
the dynamical system represented by the larger population.
5.4 Conclusions
The central finding of this work is the demonstration that neural dynamics of hetero-
geneous populations of oculomotor neurons can be described using state-space meth-
ods, and that these dynamics relate to monkeys’ behavior on a trial-by-trial basis.
Recording from neural ensembles in prearcuate cortex, a relatively uncharacterized
oculomotor area, we demonstrated this effect in the context of movement preparation.
Our results also support findings from single-unit studies in this region of the frontal
lobe. Using both traditional and state-space approaches, we found that prearcuate
cortex reflects decision-making in a random dots discrimination task, similar to Kim
& Shadlen’s results (Kim and Shadlen, 1999). We also found that the responses of a
heterogeneous population of neurons in prearcuate cortex contain signatures of pre-
vious actions. Finally, state-space and multielectrode recording methods allowed us
to compare systems-level descriptions of movement preparation an oculomotor area
(prearcuate) and an arm-movement area (PMd). Despite differences across the two
systems and across effectors, we found that the same framework could be used to
describe the dynamics of movement preparation.
Taken together, the experiments described in this thesis provide a deeper under-
standing of planning activity in the motor system and how plan dynamics change with
different task constraints. Moreover, relating the dynamics of movement planning to
behavior may ultimately aid in the development of neurally-controlled prostheses.
Bibliography
Achtman N, Afshar A, Santhanam G, Yu BM, Ryu SI, Shenoy KV (2007) Free-paced
high-performance brain-computer interfaces. J Neural Eng 4:336–47.
Afshar A, Santhanam G, Yu B, Ryu S, Sahani M, Shenoy K (2010) Single-trial neural
correlates of arm movement preparation (submitted).
Afshar A (2008) Neural mechanisms of motor preparation and applications to pros-
theses. Ph.D. dissertation., Stanford University.
Andersen RA, Asanuma C, Cowan WM (1985a) Callosal and prefrontal associa-
tional projecting cell populations in area 7a of the macaque monkey: a study using
retrogradely transported fluorescent dyes. J Comp Neurol 232:443–55.
Andersen RA, Essick GK, Siegel RM (1985b) Encoding of spatial location by posterior
parietal neurons. Science 230:456–8.
Andersen RA, Hwang EJ, Mulliken GH (2010) Cognitive neural prosthetics. Annu
Rev Psychol 61:169–90, C1–3.
Bair W, Zohary E, Newsome WT (2001) Correlated firing in macaque visual area
MT: time scales and relationship to behavior. J Neurosci 21:1676–97.
Barbas H, Mesulam MM (1981) Organization of afferent input to subdivisions of area
8 in the rhesus monkey. J Comp Neurol 200:407–31.
Barraclough DJ, Conroy ML, Lee D (2004) Prefrontal cortex and decision making in
a mixed-strategy game. Nat Neurosci 7:404–10.
Basso MA, Wurtz RH (1998) Modulation of neuronal activity in superior colliculus
by changes in target probability. J Neurosci 18:7519–34.
128
129
Bastian A, Riehle A, Erlhagen W, Schoner G (1998) Prior information preshapes
the population representation of movement direction in motor cortex. Neurore-
port 9:315–9.
Bastian A, Schoner G, Riehle A (2003) Preshaping and continuous evolution
of motor cortical representations during movement preparation. Eur J Neu-
rosci 18:2047–2058.
Batista AP, Santhanam G, Yu BM, Ryu SI, Afshar A, Shenoy KV (2007) Refer-
ence frames for reach planning in macaque dorsal premotor cortex. J Neurophys-
iol 98:966–83.
Beevor C (1888) A further minute analysis by electric stimulation of the so-called
motor region of the cortex cerebri in the monkey (macacus sinicus). Philosophical
Transactions of the Royal Society of London. B 179:205–256.
Bizzi E (1967) Discharge of frontal eye field neurons during eye movements in unanes-
thetized monkeys. Science 157:1588–90.
Bizzi E (1968) Discharge of frontal eye field neurons during saccadic and following
eye movements in unanesthetized monkeys. Exp Brain Res 6:69–80.
Bizzi E, Schiller PH (1970) Single unit activity in the frontal eye fields of unanes-
thetized monkeys during eye and head movement. Exp Brain Res 10:150–8.
Boch RA, Goldberg ME (1989) Participation of prefrontal neurons in the preparation
of visually guided eye movements in the rhesus monkey. J Neurophysiol 61:1064–84.
Britten KH, Shadlen MN, Newsome WT, Movshon JA (1992) The analysis of vi-
sual motion: a comparison of neuronal and psychophysical performance. J Neu-
rosci 12:4745–65.
Britten KH, Shadlen MN, Newsome WT, Movshon JA (1993) Responses of neurons
in macaque MT to stochastic motion signals. Vis Neurosci 10:1157–69.
Bruce CJ, Goldberg ME (1985) Primate frontal eye fields. I. Single neurons discharg-
ing before saccades. J Neurophysiol 53:603–35.
130
Bruce CJ, Goldberg ME, Bushnell MC, Stanton GB (1985) Primate frontal eye fields.
II. Physiological and anatomical correlates of electrically evoked eye movements. J
Neurophysiol 54:714–34.
Chestek CA, Batista AP, Santhanam G, Yu BM, Afshar A, Cunningham JP, Gilja
V, Ryu SI, Churchland MM, Shenoy KV (2007) Single-neuron stability during
repeated reaching in macaque premotor cortex. J Neurosci 27:10742–50.
Churchland MM, Santhanam G, Shenoy KV (2006) Preparatory activity in pre-
motor and motor cortex reflects the speed of the upcoming reach. J Neurophys-
iol 96:3130–46.
Churchland MM, Shenoy KV (2007) Delay of movement caused by disruption of
cortical preparatory activity. J Neurophysiol 97:348–59.
Churchland MM, Yu BM, Ryu SI, Santhanam G, Shenoy KV (2006) Neural vari-
ability in premotor cortex provides a signature of motor preparation. J Neu-
rosci 26:3697–3712.
Churchland MM, Yu BM, Sahani M, Shenoy KV (2007) Techniques for extracting
single-trial activity patterns from large-scale neural recordings. Curr Opin Neuro-
biol 17:609–618.
Churchland M, Cunningham J, Kaufman M, Ryu S, Shenoy K (2010) Re-examining
the preferred direction during movement preparation. Neuron (submitted).
Cisek P, Kalaska JF (2002) Simultaneous encoding of multiple potential reach direc-
tions in dorsal premotor cortex. J Neurophysiol 87:1149–54.
Coe B, Tomihara K, Matsuzawa M, Hikosaka O (2002) Visual and anticipatory bias
in three cortical eye fields of the monkey during an adaptive decision-making task.
J Neurosci 22:5081–90.
Collin NG, Cowey A, Latto R, Marzi C (1982) The role of frontal eye-fields and
superior colliculi in visual search and non-visual search in rhesus monkeys. Behav
Brain Res 4:177–93.
131
Crammond DJ, Kalaska JF (1989) Neuronal activity in primate parietal cortex area
5 varies with intended movement direction during an instructed-delay period. Exp
Brain Res 76:458–62.
Crammond DJ, Kalaska JF (2000) Prior information in motor and premotor cortex:
activity during the delay period and effect on pre-movement activity. J Neurophys-
iol 84:986–1005.
Crosby E, Yoss R, Henderson J (1952) The mammalian midbrain and isthmus regions.
Part II. The fiber connections. D. The pattern for eye movements on the frontal
eye field and the discharge of specific portions of this field to and through midbrain
levels. The Journal of comparative neurology 97:357.
Donoghue JP (2002) Connecting cortex to machines: recent advances in brain inter-
faces. Nat Neurosci 5 Suppl:1085–8.
Dorris MC, Pare M, Munoz DP (1997) Neuronal activity in monkey superior colliculus
related to the initiation of saccadic eye movements. J Neurosci 17:8566–79.
Dorris MC, Pare M, Munoz DP (2000) Immediate neural plasticity shapes motor
performance. J Neurosci 20:RC52.
Dum RP, Strick PL (2002) Motor areas in the frontal lobe of the primate. Physiol
Behav 77:677–82.
Evarts EV (1966) Pyramidal tract activity associated with a conditioned hand move-
ment in the monkey. J Neurophysiol 29:1011–27.
Evarts EV (1968) Relation of pyramidal tract activity to force exerted during volun-
tary movement. J Neurophysiol 31:14–27.
Evarts EV, Tanji J (1974) Gating of motor cortex reflexes by prior instruction. Brain
Res 71:479–94.
Everling S, Munoz D (2000) Neuronal correlates for preparatory set associated with
pro-saccades and anti-saccades in the primate frontal eye field. J Neurosci 20:387.
132
Favilla M, Hening W, Ghez C (1989) Trajectory control in targeted force impulses.
VI. Independent specification of response amplitude and direction. Exp Brain
Res 75:280–94.
Fecteau JH, Munoz DP (2003) Exploring the consequences of the previous trial. Nat
Rev Neurosci 4:435–43.
Ferrier D (1874) On the localisation of the functions of the brain. British Medical
Journal 2:766.
Ferrier D (1875) Experiments on the brain of monkeys. No. I. Proceedings of the
Royal Society of London 23:409–430.
Funahashi S, Bruce CJ, Goldman-Rakic PS (1989) Mnemonic coding of visual space
in the monkey’s dorsolateral prefrontal cortex. J Neurophysiol 61:331–49.
Funahashi S, Bruce CJ, Goldman-Rakic PS (1990) Visuospatial coding in primate
prefrontal neurons revealed by oculomotor paradigms. J Neurophysiol 63:814–31.
Funahashi S, Bruce CJ, Goldman-Rakic PS (1991) Neuronal activity related to sac-
cadic eye movements in the monkey’s dorsolateral prefrontal cortex. J Neurophys-
iol 65:1464–83.
Fuster JM (1973) Unit activity in prefrontal cortex during delayed-response perfor-
mance: neuronal correlates of transient memory. J Neurophysiol 36:61–78.
Fuster JM (1993) Frontal lobes. Current Opinion in Neurobiology 3:160–165.
Gamlin PD, Yoon K (2000) An area for vergence eye movement in primate frontal
cortex. Nature 407:1003–7.
Genovesio A, Tsujimoto S, Wise SP (2006) Neuronal activity related to elapsed time
in prefrontal cortex. J Neurophysiol 95:3281–5.
Georgopoulos AP (1996) Arm movements in monkeys: behavior and neurophysiology.
J Comp Physiol A 179:603–12.
Georgopoulos AP, Kalaska JF, Caminiti R, Massey JT (1982) On the relations be-
tween the direction of two-dimensional arm movements and cell discharge in pri-
mate motor cortex. J Neurosci 2:1527–37.
133
Gerbella M, Belmalih A, Borra E, Rozzi S, Luppino G (2007) Multimodal archi-
tectonic subdivision of the caudal ventrolateral prefrontal cortex of the macaque
monkey. Brain Struct Funct 212:269–301.
Gerbella M, Belmalih A, Borra E, Rozzi S, Luppino G (2009) Cortical connections
of the macaque caudal ventrolateral prefrontal areas 45a and 45b. Cereb Cortex .
Gilja V, Kalmar R, Santhanam G, Ryu S, Yu B, Afshar A, Shenoy K (2005) Trial-by-
trial mean normalization improves plan period reach target decoding. In Abstract
Viewer/Itinerary Planner. Program No. 519.18., Washington, DC. Society for Neu-
roscience.
Godschalk M, Lemon RN, Kuypers HG, van der Steen J (1985) The involvement of
monkey premotor cortex neurones in preparation of visually cued arm movements.
Behav Brain Res 18:143–57.
Goldberg ME, Bushnell MC (1981) Behavioral enhancement of visual responses in
monkey cerebral cortex. II. Modulation in frontal eye fields specifically related to
saccades. J Neurophysiol 46:773–87.
Goldman-Rakic PS (1987) Circuitry of primate prefrontal cortex and the regulation
of behavior by representational memory In Handbook of Physiology. The Nervous
System., Vol. V, pp. p. 373–417. Bethesda, MD: Am. Physiol. Soc.
Goldman-Rakic PS (1988) Topography of cognition: parallel distributed networks in
primate association cortex. Annu Rev Neurosci 11:137–56.
Gottlieb JP, Bruce CJ, MacAvoy MG (1993) Smooth eye movements elicited by
microstimulation in the primate frontal eye field. J Neurophysiol 69:786–99.
Gottlieb JP, MacAvoy MG, Bruce CJ (1994) Neural responses related to smooth-
pursuit eye movements and their correspondence with electrically elicited smooth
eye movements in the primate frontal eye field. J Neurophysiol 72:1634–53.
Graziano M, Taylor C, Moore T (2002) Complex movements evoked by microstimu-
lation of precentral cortex. Neuron 34:841–851.
Hanes DP, Schall JD (1996) Neural control of voluntary movement initiation. Sci-
ence 274:427–430.
134
Hasegawa RP, Blitz AM, Geller NL, Goldberg ME (2000) Neurons in monkey pre-
frontal cortex that track past or predict future performance. Science 290:1786–9.
Hatsopoulos N, Joshi J, O’Leary J (2004) Decoding continuous and discrete motor
behaviors using motor and premotor cortical ensembles. J Neurophysiol 92:1165.
Hays AJ, Richmond B, Optican L (1982) Unix-based multiple-process system, for
real-time data acquisition and control. Electron Conventions, El Segundo, CA.
Hochberg LR, Serruya MD, Friehs GM, Mukand JA, Saleh M, Caplan AH, Branner
A, Chen D, Penn RD, Donoghue JP (2006) Neuronal ensemble control of prosthetic
devices by a human with tetraplegia. Nature 442:164–71.
Horwitz GD, Newsome WT (1999) Separate signals for target selection and movement
specification in the superior colliculus. Science 284:1158–61.
Horwitz GD, Newsome WT (2001) Target selection for saccadic eye movements:
prelude activity in the superior colliculus during a direction-discrimination task. J
Neurophysiol 86:2543–58.
Judge SJ, Richmond BJ, Chu FC (1980) Implantation of magnetic search coils for
measurement of eye position: an improved method. Vision Res 20:535–8.
Kalaska JF, Scott SH, Cisek P, Sergio LE (1997) Cortical control of reaching move-
ments. Curr Opin Neurobiol 7:849–59.
Kalaska J (1991) What parameters of reaching are encoded by discharges of cortical
cells? In Motor control: concepts and issues: report of the Dahlem Workshop on
Motor Control: Concepts and Issues, Berlin, 1989, December 3-8, p. 307. John
Wiley & Sons.
Kalmar R, Gilja V, Santhanam G, Ryu S, Yu B, Afshar A, Shenoy K (2005) PMd de-
lay activity during rapid sequential movement plans. In Abstract Viewer/Itinerary
Planner. Program No. 519.17., Washington, DC. Society for Neuroscience.
Kalmar R, Reppas J, Ryu S, Shenoy K, Newsome WW (2010) Ensemble activity un-
derlying movement preparation in prearcuate cortex In Computational and Systems
Neuroscience (COSYNE) meeting, Salt Lake City, UT.
135
Keele S (1968) Movement control in skilled motor performance. Psychological Bul-
letin 70:387–403.
Kim JN, Shadlen MN (1999) Neural correlates of a decision in the dorsolateral
prefrontal cortex of the macaque. Nat Neurosci 2:176–85.
Kolenikov S, Bollen K, Savalei V (2006) Specification tests with Heywood cases In
Proceedings of the American Statistical Association, Social Statistics Section [CD-
ROM], Alexandria, VA. American Statistical Association.
Kutas M, Donchin E (1974) Studies of squeezing: handedness, responding hand,
response force, and asymmetry of readiness potential. Science 186:545–8.
Lecas JC, Requin J, Anger C, Vitton N (1986) Changes in neuronal activity of
the monkey precentral cortex during preparation for movement. J Neurophys-
iol 56:1680–1702.
Levinsohn G (1909) Uber die beziehungen der grosshirnrinde beim affen zu den
bewegungen des auges. Graefe’s Archive for Clinical and Experimental Ophthal-
mology 71:313–378.
Levy R, Goldman-Rakic PS (2000) Segregation of working memory functions within
the dorsolateral prefrontal cortex. Exp Brain Res 133:23–32.
Louie K, Glimcher PW (2010) Separating value from choice: delay discounting ac-
tivity in the lateral intraparietal area. J Neurosci 30:5498–507.
Machens CK, Romo R, Brody CD (2010) Functional, but not anatomical, separation
of “what” and “when” in prefrontal cortex. J Neurosci 30:350–60.
McCoy AN, Platt ML (2005) Expectations and outcomes: decision-making in the pri-
mate brain. J Comp Physiol A Neuroethol Sens Neural Behav Physiol 191:201–11.
Medalla M, Barbas H (2006) Diversity of laminar connections linking periarcuate and
lateral intraparietal areas depends on cortical structure. Eur J Neurosci 23:161–79.
Messier J, Kalaska JF (2000) Covariation of primate dorsal premotor cell activity
with direction and amplitude during a memorized-delay reaching task. J Neuro-
physiol 84:152–65.
136
Miller EK, Erickson CA, Desimone R (1996) Neural mechanisms of visual working
memory in prefrontal cortex of the macaque. J Neurosci 16:5154–67.
Miller E (2000) The prefrontal cortex and cognitive control. Nature Reviews Neuro-
science 1:59–65.
Moschovakis AK, Gregoriou GG, Ugolini G, Doldan M, Graf W, Guldin W, Had-
jidimitrakis K, Savaki HE (2004) Oculomotor areas of the primate frontal lobes: a
transneuronal transfer of rabies virus and [14c]-2-deoxyglucose functional imaging
study. J Neurosci 24:5726–40.
Mott F, Schaefer E (1890) On associated eye-movements produced by cortical
faradization of the monkey’s brain. Brain 13:165.
Mulliken GH, Musallam S, Andersen RA (2008) Decoding trajectories from posterior
parietal cortex ensembles. J Neurosci 28:12913–26.
Musallam S, Corneil BD, Greger B, Scherberger H, Andersen RA (2004) Cognitive
control signals for neural prosthetics. Science 305:258–62.
Newsome WT, Britten KH, Movshon JA (1989) Neuronal correlates of a perceptual
decision. Nature 341:52–4.
Nicolelis MA (2001) Actions from thoughts. Nature 409:403–7.
Padoa-Schioppa C, Li CSR, Bizzi E (2002) Neuronal correlates of kinematics-to-
dynamics transformation in the supplementary motor area. Neuron 36:751–65.
Pare M, Munoz DP (1996) Saccadic reaction time in the monkey: Advanced prepara-
tion of oculomotor programs is primarily responsible for express saccade occurrence.
Journal of Neurophysiology 76.
Parthasarathy HB, Schall JD, Graybiel AM (1992) Distributed but convergent or-
dering of corticostriatal projections: analysis of the frontal eye field and the sup-
plementary eye field in the macaque monkey. J Neurosci 12:4468–88.
Pellizzer G, Hedges JH (2003) Motor planning: effect of directional uncertainty with
discrete spatial cues. Exp Brain Res 150:276–89.
137
Petrides M (1994) Frontal lobes and behaviour. Curr Opin Neurobiol 4:207–11.
Posner MI, Snyder CR, Davidson BJ (1980) Attention and the detection of signals.
J Exp Psychol 109:160–74.
Preuss T (2007) Evolutionary specializations of primate brain systems. Primate
Origins: Adaptations and Evolution pp. 625–675.
Preuss TM, Goldman-Rakic PS (1991) Myelo- and cytoarchitecture of the granular
frontal cortex and surrounding regions in the strepsirhine primate galago and the
anthropoid primate macaca. J Comp Neurol 310:429–474.
Reppas J, Kalmar R, Ryu S, Shenoy K, Newsome W (2010) Characterizing response
properties in prearcuate cortex (in preparation) .
Reppas J, Ryu S, Newsome W (2010) Hold-period activity in prearcuate cortex. (in
preparation) .
Riehle A, Requin J (1989) Monkey primary motor and premotor cortex: single-cell
activity related to prior information about direction and extent of an intended
movement. J Neurophysiol 61:534–49.
Riehle A, Requin J (1993) The predictive value for performance speed of preparatory
changes in neuronal activity of the monkey motor and premotor cortex. Behav
Brain Res 53:35–49.
Rivera Alvidrez Z, Kalmar R, Afshar A, Santhanam G, Yu B, Ryu S, Shenoy K (2007)
Single-trial representation of uncertainty about reach goals in macaque pmd. In
Society for Neuroscience annual meeting.
Rivera Alvidrez Z, Kalmar R, Afshar A, Santhanam G, Yu B, Ryu S, Shenoy K (2008)
Single-trial representation of uncertainty about reach goals in macaque pmd. In
Computational and Systems Neuroscience (COSYNE) meeting, Salt Lake City, UT.
III-52.
Rivera Alvidrez Z, Kalmar R, Ryu S, Shenoy K (2010) Low-dimensional neural fea-
tures predict muscle EMG signals In Proceedings of the 32nd Annual International
Conference of the IEEE EMBS, Buenos Aires, Argentina.
138
Robinson DA, Fuchs AF (1969) Eye movements evoked by stimulation of frontal eye
fields. J Neurophysiol 32:637–48.
Roesch MR, Olson CR (2003) Impact of expected reward on neuronal activity in
prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J
Neurophysiol 90:1766–89.
Roitman JD, Shadlen MN (2002) Response of neurons in the lateral intraparietal area
during a combined visual discrimination reaction time task. J Neurosci 22:9475–89.
Romanski LM (2004) Domain specificity in the primate prefrontal cortex. Cogn Affect
Behav Neurosci 4:421–9.
Romo R, Hernandez A, Zainos A (2004) Neuronal correlates of a perceptual decision
in ventral premotor cortex. Neuron 41:165–73.
Rosenbaum DA (1980) Human movement initiation: specification of arm, direction,
and extent. J Exp Psychol Gen 109:444–74.
Rosenkilde CE (1979) Functional heterogeneity of the prefrontal cortex in the monkey:
a review. Behav Neural Biol 25:301–45.
Roweis S (1998) EM algorithms for PCA and SPCA. Advances in Neural Information
Processing Systems 10 (626).
Santhanam G, Ryu S, Yu B, Afshar A, Shenoy K (2006) A high-performance brain-
computer interface. Nature 442:195–198.
Santhanam G, Sahani M, Ryu S, Shenoy K (2004) An extensible infrastructure for
fully automated spike sorting during online experiments. Conf Proc IEEE Eng Med
Biol Soc 6:4380–4.
Schall JD (1991) Neuronal activity related to visually guided saccadic eye movements
in the supplementary motor area of rhesus monkeys. J Neurophysiol 66:530–58.
Schall JD (1995) Neural basis of saccade target selection. Rev Neurosci 6:63–85.
Schall JD (2001) Neural basis of deciding, choosing and acting. Nat Rev Neu-
rosci 2:33–42.
139
Schall JD, Hanes DP, Thompson KG, King DJ (1995a) Saccade target selection
in frontal eye field of macaque. i. visual and premovement activation. J Neu-
rosci 15:6905–6918.
Schall JD, Morel A, King DJ, Bullier J (1995b) Topography of visual cortex connec-
tions with frontal eye field in macaque: convergence and segregation of processing
streams. J Neurosci 15:4464–4487.
Schall JD, Thompson KG (1999) Neural selection and control of visually guided eye
movements. Annu Rev Neurosci 22:241–259.
Schall J, Hanes D (1993) Saccade latency in context: Regulation of gaze behavior by
supplementary eye field. Behavioral and Brain Sciences 16:588–589.
Schall JD (1997) Visuomotor areas of the frontal lobe In et al. R, editor, Cerebral
Cortex, Vol. 12, chapter 13, pp. 527–638. New York, Plenum Press.
Schall JD, Stuphorn V, Brown JW (2002) Monitoring and control of action by the
frontal lobes. Neuron 36:309–22.
Schiller PH, True SD, Conway JL (1979) Paired stimulation of the frontal eye fields
and the superior colliculus of the rhesus monkey. Brain Res 179:162–4.
Schiller PH, True SD, Conway JL (1980) Deficits in eye movements following frontal
eye-field and superior colliculus ablations. J Neurophysiol 44:1175–89.
Schlag J, Schlag-Rey M (1987) Evidence for a supplementary eye field. J Neurophys-
iol 57:179–200.
Schwartz AB (2004) Cortical neural prosthetics. Annu Rev Neurosci 27:487–507.
Scott S (2003) The role of primary motor cortex in goal-directed movements: in-
sights from neurophysiological studies on non-human primates. Curr Opin Neuro-
biol 13:671–677.
Seidemann E, Arieli A, Grinvald A, Slovin H (2002) Dynamics of depolarization and
hyperpolarization in the frontal cortex and saccade goal. Science 295:862–5.
Shadlen MN, Newsome WT (1996) Motion perception: seeing and deciding. Proc
Natl Acad Sci USA 93:628–33.
140
Shadlen MN, Newsome WT (2001) Neural basis of a perceptual decision in the
parietal cortex (area LIP) of the rhesus monkey. J Neurophysiol 86:1916–36.
Smith W (1940) Electrically responsive cortex within the sulci of the frontal lobe.
Anat. Rec 76:75–76.
Snyder LH, Dickinson AR, Calton JL (2006) Preparatory delay activity in the monkey
parietal reach region predicts reach reaction times. J Neurosci 26:10091–9.
Stanton GB, Bruce CJ, Goldberg ME (1993) Topography of projections to the frontal
lobe from the macaque frontal eye fields. J Comp Neurol 330:286–301.
Stanton G, Deng S, Goldberg M, McMullen N (1989) Cytoarchitectural characteristic
of the frontal eye fields in macaque monkeys. J Comp Neurol 282:415.
Stoet G, Snyder LH (2003) Task preparation in macaque monkeys (macaca mulatta).
Anim Cogn 6:121–30.
Sugrue LP, Corrado GS, Newsome WT (2004) Matching behavior and the represen-
tation of value in the parietal cortex. Science 304:1782–7.
Suner S, Fellows M, Vargas-Irwin C, Nakata G, Donoghue J (2005) Reliability of
signals from a chronically implanted, silicon-based electrode array in non-human
primate primary motor cortex. IEEE Transactions on Neural Systems and Reha-
bilitation Engineering 13:524–541.
Suzuki H, Azuma M (1983) Topographic studies on visual neurons in the dorsolateral
prefrontal cortex of the monkey. Exp Brain Res 53:47–58.
Tanji J, Evarts EV (1976) Anticipatory activity of motor cortex neurons in relation
to direction of an intended movement. J Neurophysiol 39:1062–8.
Taylor CSR, Gross CG (2003) Twitches versus movements: a story of motor cortex.
Neuroscientist 9:332–42.
Tehovnik EJ, Sommer MA, Chou IH, Slocum WM, Schiller PH (2000) Eye fields in
the frontal lobes of primates. Brain Res Brain Res Rev 32:413–48.
141
Thompson KG, Hanes DP, Bichot NP, Schall JD (1996) Perceptual and motor pro-
cessing stages identified in the activity of macaque frontal eye field neurons during
visual search. J Neurophysiol 76:4040–4055.
Wagman I, Krieger HP, Papatheodorou CA, Bender MB (1961) Eye movements
elicited by surface and depth stimulation of the frontal lobe of macaque mulatta.
J Comp Neurol 117:179–88.
Weinrich M, Wise SP (1982) The premotor cortex of the monkey. J Neu-
rosci 2:1329–45.
Weinrich M, Wise SP, Mauritz KH (1984) A neurophysiological study of the premotor
cortex in the rhesus monkey. Brain 107 (Pt. 2):385–414.
Wise SP (1985) The primate premotor cortex: past, present, and preparatory. Annu
Rev Neurosci 8:1–19.
Wise SP, Boussaoud D, Johnson PB, Caminiti R (1997) Premotor and parietal cor-
tex: corticocortical connectivity and combinatorial computations. Annu Rev Neu-
rosci 20:25–42.
Wurtz RH, Goldberg ME (1972) Activity of superior colliculus in behaving monkey.
3. Cells discharging before eye movements. J Neurophysiol 35:575–86.
Wurtz RH, Mohler CW (1976) Enhancement of visual responses in monkey striate
cortex and frontal eye fields. J Neurophysiol 39:766–72.
Yu B, Afshar A, Santhanam G, Ryu S, Shenoy K, Sahani M (2006) Extracting
dynamical structure embedded in neural activity. In Weiss Y, Scholkopf B, Platt
J, editors, Neural Information Processing Society (NIPS), Vol. 18, pp. 1545–1552,
Cambridge, MA. MIT Press.
Yu BM, Cunningham JP, Santhanam G, Ryu SI, Shenoy KV, Sahani M (2009)
Gaussian-process factor analysis for low-dimensional single-trial analysis of neu-
ral population activity. J Neurophysiol 102:614–35.
Zaksas D, Pasternak T (2006) Directional signals in the prefrontal cortex and in area
MT during a working memory for visual motion task. J Neurosci 26:11726–42.